I'm trying to receive a UDP stream with Gstreamer in Java, which is sent from a Raspberry PI.
The stream should be displayed in a JavaFX window.
However, there's only a black screen appearing and the pipeline state is set to "PAUSED".
even though I can see the stream of my local webcam in a JavaFX window with the second class below.
Below my code of the UDP stream which doesn't work:
package experimental;
import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.Background;
import javafx.scene.layout.BackgroundFill;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.CornerRadii;
import javafx.scene.paint.Color;
import javafx.stage.Stage;
import org.freedesktop.gstreamer.*;
import org.freedesktop.gstreamer.fx.FXImageSink;
public class NotWorking extends Application {
private Pipeline pipeline;
@Override
public void start(Stage stage) throws Exception {
Gst.init(Version.BASELINE, "FXCamera");
pipeline = new Pipeline();
FXImageSink imageSink = new FXImageSink();
imageSink.requestFrameSize(640, 480);
Caps caps = Caps.fromString("application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96");
Element src = ElementFactory.make("udpsrc", null);
src.set("port", 1234);
src.setCaps(caps);
Bin bin = Gst.parseBinFromDescription("rtph264depay ! decodebin ! videoconvert ! autovideosink", true);
pipeline.addMany(src, bin, imageSink.getSinkElement());
Pipeline.linkMany(src, bin, imageSink.getSinkElement());
stage.setTitle("FX Camera");
BorderPane pane = new BorderPane();
pane.setBackground(new Background(new BackgroundFill(
Color.BLACK, CornerRadii.EMPTY, Insets.EMPTY)));
ImageView view = new ImageView();
pane.setCenter(view);
view.imageProperty().bind(imageSink.imageProperty());
view.fitWidthProperty().bind(pane.widthProperty());
view.fitHeightProperty().bind(pane.heightProperty());
view.setPreserveRatio(true);
stage.setScene(new Scene(pane, 640, 480));
stage.show();
pipeline.play();
new AnimationTimer(){
@Override
public void handle(long now) {
System.out.println(pipeline.getState());
}
}.start();
}
}
Code of the local webcam which works:
package experimental;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.image.ImageView;
import javafx.scene.layout.Background;
import javafx.scene.layout.BackgroundFill;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.CornerRadii;
import javafx.scene.paint.Color;
import javafx.stage.Stage;
import org.freedesktop.gstreamer.Bin;
import org.freedesktop.gstreamer.Gst;
import org.freedesktop.gstreamer.Pipeline;
import org.freedesktop.gstreamer.Version;
import org.freedesktop.gstreamer.fx.FXImageSink;
public class WorkingClass extends Application {
/**
* Always store the top-level pipeline reference to stop it being garbage
* collected.
*/
private Pipeline pipeline;
@Override
public void init() throws Exception {
Gst.init(Version.BASELINE, "FXCamera");
}
@Override
public void start(Stage stage) throws Exception {
/**
* FXImageSink from gst1-java-fx wraps the native data from the
* GStreamer AppSink in a JavaFX image. Requested dimensions will be set
* on the caps for the AppSink.
*/
FXImageSink imageSink = new FXImageSink();
imageSink.requestFrameSize(640, 480);
/**
* Parse a Bin to contain the autovideosrc from a GStreamer string
* representation. The alternative approach would be to create and link
* the elements in code using ElementFactory::make.
*
* The Bin uses a videoconvert element to convert the video format to
* that required by the FXImageSink, a videoscale in case the source
* does not support the required resolution.
*
* The bin is added to a top-level pipeline and linked to the AppSink
* from the image sink.
*/
Bin bin = Gst.parseBinFromDescription(
"autovideosrc ! videoscale ! videoconvert",
true);
pipeline = new Pipeline();
pipeline.addMany(bin, imageSink.getSinkElement());
Pipeline.linkMany(bin, imageSink.getSinkElement());
/**
* Set up a simple JavaFX window with ImageView.
*/
stage.setTitle("FX Camera");
BorderPane pane = new BorderPane();
pane.setBackground(new Background(new BackgroundFill(
Color.BLACK, CornerRadii.EMPTY, Insets.EMPTY)));
ImageView view = new ImageView();
pane.setCenter(view);
/**
* Bind the ImageView to the image from the FXImageSink, and scale the
* view within the window.
*/
view.imageProperty().bind(imageSink.imageProperty());
view.fitWidthProperty().bind(pane.widthProperty());
view.fitHeightProperty().bind(pane.heightProperty());
view.setPreserveRatio(true);
/**
* Show the window and start the pipeline.
*/
stage.setScene(new Scene(pane, 640, 480));
stage.show();
pipeline.play();
}
}
I'm trying to build the pipeline like this:
gst-launch-1.0 -v udpsrc port=1234 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink
The command for the Raspberry PI to send the stream:
sudo raspivid -gps -hf -vf -n -w 640 -h 480 -b 1000000 -fps 60 -t 0 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=xxx.xx.x.x port=1234