Streaming h264 encoded video over Serial port from Colibri imx6

Hi,

I have a Colibri imx6 device and I am trying to stream h264 encoded video from imx6 board to a PC. So far I have succeeded in streaming the videotestsrc element over UPD with RTP. Now I encoded the video to a bitrate of 5Kbps as per GStreamer imx plugins and tried transmitting this over serial port with 115200 Baudrate. This should be posssible considering that the bitrate is 5Kbps.

However, at the receiving end, I see a lot of packet losses and the videostestsrc is not correctly displayed.
Following are the commands:

imx6:

gst-launch-1.0 -v videotestsrc pattern=18 !
video/x-raw,width=100,height=50,framerate=10/1  ! imxvpuenc_h264 bitrate=5 ! h264parse !
filesink location=/dev/ttyUSB1

PC:

CAPS= video/x-h264,width=100,height=50,framerate=10/1 
gst-launch-1.0 -v filesrc location=/dev/ttyUSB1 ! $CAPS ! h264parse !
avdec_h264 ! autovideosink sync=true 

I am attaching the log file explaining the error with this question.

I have verified that the baud rates are same at both the serial ports. I also understand that RTP is useful in using UDP streaming and which greatly helps in streaming video. However is such a similar thing available for serial port.

Can you guys suggest any solutions for the same? I do not expect to rewrite the whole h264 decoder to reconstruct video from lost data packets. :wink: Perhaps this is better handled by the GStreamer plugins already.

The reason I am using a serial device, is that I need to transmit this video over a custom wireless device, which is accessible via serial port. The next part of my test would be to send the camera captured data and see it stream on a PC.

hi @soc_learner

Could you provide the version of the hardware and software of your module? Which carrier board are you using?

If you want to transmit this video to serial port then you could use also the regular UART Ports of the module.

Alternatively you can use PPP over UART on Host and Module. An example for using PPP can be found here.

Best regards, Jaski

HI Jaski,

I am using a Kolibri evaluation board. We are using a custom built kernel Debian image using 4.1.15 kernel. I do not intend to use the UART ports directly, since the wireless device I am using has an usb-serial interface. So the wireless Tx/Rx can only be accessed using the serial ports.
PPP sounds promising, but would be bounded by the baud rate. But perhaps I can use the same UDP stream with RTP over PPP and find out if it helps. As far as I have read and googled, RTP allows for the syncrhonization of encoded packets with the UDP packets matching the sequence numbers with the RTP packets. Such a feature is not available with UART and hence the packets are lost and the frame is decoded incorrectly at the PC side.
But I wonder if using PPP, would solve this issue, because at the receiving side, the UART might still not be able to receive all the packets and hence the RTP might lose out the frames again.
What are your views on this? Do you suggest to try something else?

  1. I am trying to stream video from imx6 board to a PC using wireless Tx/Rx. Here is the device which I am using. This wireless device is visible as a serial port and can be accessed using /dev/ttyUSBx

  2. Correction I am using the Apalis evaluation board.

  3. I am using the Apalis imx6 board.

  4. GStreamer 1.0 is installed on PC and apalis imx6 with imx6 plugins.

  5. Image running on the imx6 is a custom Debian image with kernel 4.9 (verified)
    #1 SMP PREEMPT Debian 4.9.74-2.2

Please let me know if you need more information.

Your use case is still completely unclear to us. What exactly is it that you are trying to achieve. How exactly does your hardware and software setup look like? What exact hardware and software versions of things are you using? That wireless device you are mentioning, what exactly is it?

Hi @soc_learner

As you are not using standard openembedded kernel and standard imx6-uart interface, it will be difficult to reproduce your issue. However we can give some recommendations

PPP sounds promising, but would be bounded by the baud rate. But perhaps I can use the same UDP stream with RTP over PPP and find out if it helps.
Why do you think you are bounded by baud rate. Which baud rate do you need? You should try PPP.

As far as I have read and googled, RTP allows for the syncrhonization of encoded packets with the UDP packets matching the sequence numbers with the RTP packets. Such a feature is not available with UART and hence the packets are lost and the frame is decoded incorrectly at the PC side. But I wonder if using PPP, would solve this issue, because at the receiving side, the UART might still not be able to receive all the packets and hence the RTP might lose out the frames again. What are your views on this? Do you suggest to try something else?

If you want to have sequence numbering, you could also try SLIP.