Gray overlay on video

Ole_Gulbrandsen

Hi, I've been using UAVMatrix on my Nano Talon with a RPi Zero W, a 4G modem and ZeroTier VPN. I'm using Mission Planner on a PC with the video displayed in the Mission Planner HUD.

The telemetry and MAVLink communication works great and is super stable. I really like the UAVMatrix software! It works very well for me and made the setup extremely simple.

However, the GStreamer video is quite bad and a gray overlay seems to enter the image half of the time, especially when the connection is not perfect. ( Sometimes when the connection is super fast, all is good )

I've tried using the 5601 port and use a custom Gstream configuration in MP as suggested here:
https://docs.uavmatrix.com/pages/camera/Configuration/

udpsrc port=5601 caps = "application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96" ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert ! video/x-raw,format=BGRA ! appsink name=outsink

It did help a little perhaps, but it is still not possible to fly using the video, I rely on the map and cruise mode in MP.

What can I do to improve on this? Is there some way to increase the buffering? I would be happy to live with a higher latency. I do not think it is a matter of just bandwidth, I have a feeling it is related to VPN, network latency and data packets arriving at irregular intervals, but I'm just guessing. Is there some way to specify a buffer, for lets say 2 secs?

I've tried dialing down the bandwidth, framerate, resolution to a minimum, the problem is the same.

Anyone with similar problems? Any suggestions to what I could try?

Thanks!!

Sinamics

Hi Ole_Gulbrandsen
Good to see you are still in the business. 👍️
The rtpjitterbuffer string in mission planner usually solves these gray overlay`s.
I think there is an option to add choices for rtpjitterbuffer for diffrent modes, example rtpjitterbuffer=1
but i need to look up that later on.

I have a feeling it is related to VPN, network latency and data packets arriving at irregular intervals

If you use APN network with multiple NAT`s, the VPN struggles to Punch though the UDP network, hence the grey overlay issue. OpenVpn for example, does not suffer this problem.

Do you use telenor? check which APN you use. im using the internet.public and this works without any issues at all.

Ole_Gulbrandsen

Hi, thanks for the reply! After looking more into it, it turns out the modem I was using gave a really bad connection. It was fast for 3 sec, then slowing down or basically pausing before starting up again. Switched to a Huawei modem ( E3372H ) and now everything is much better, stable fast connection and video looks good, at least on the bench. I will report how it is in once I get to try it out with the plane. Thanks for the help!

Ole_Gulbrandsen

Video is better now: 😀

https://www.youtube.com/watch?v=U3ABE0tefqI

Sinamics

Ole_Gulbrandsen Awesome stuff. Thank you for sharing.
You use the virtual desktop with Quest while flying? Clever idea.
Was it a good experience?

isoos73

Sinamics Hi!

I built in to the gcs a hdmi-usbc converter and a hdmi switch so I can watch the video on a phone in a modified vr goggles.

Ole_Gulbrandsen

Well, just for fun, but it worked! 😀 Quite immersive with such a big screen, and much better than trying to look at the laptop screen outdoors

rma153

I have the same issue. My UAVcast camera setup uses GoPro/B101 RPi4/Pixhawk4, Zero Tier VPN over 5G Tmobile network, and I see annoying random spikes of grey screen in my windows 10 video stream (1080p/30), but very low latency. My windows pipeline =

gst-launch-1.0 -v udpsrc port=5600 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false

If using a different rtpjitterbuffer variant can help solve this, that would be most excellent!

Sinamics

rma153 this is usually related to multiple NAT layers from Cell vendors causing zerotier to struggle punching through.

You could try using openvpn, this will probably solve it.

Im re-writing the whole UAVcast application from scratch, and im also writing new gstreamer binaries with more customize functionalities. This could as well improve your video , but it wont be released until end of year

slippyjeep

I think this has more to do with the h.264 encoding, if I send the video via jpeg stream I don't see the issue. Which makes some sense because the gray distortion looks like mpeg compression. One thing I have noticed is when you send the same video to Qground Control you don't see the grey interruptions. Its almost like a newer version of gstreamer on the receiving sid will fix the issue. Mission planner is opening its own version of gstreamer, which does match the 1.14.4 version Sinamics Sinamics implemented. For example if I send the video to the PC version 1.18 outside of MP, it also works fine. I am using an rpicamera on the CSI port. I have tested it on a Pi Zero 1/2 and Pi 3b all running Buster OS. All devices running on local wifi together. I gave up on this project two years ago due to this issue alone, I was hoping running the new Bullseye OS would have fixed this issue. I really want to use the Mission Planner HUD overlay. If anyone has any ideas I am all ears.. I have been playing with the gstreamer pipeline for a few days and can't seem to come up with a fix..
Also I have to fiddle with the gstreamer in MP to get the video in the HUD at all, versus it comes up immediately in QGC. Best I could get to was launching MP with a CLI string in force the gStreamer pipeline, then I can just right click and select stop stream and then the video will pop up in a second or two. But with two video issues at the build stage it makes it difficult to trust the solution enough for flight.

rma153

![
](https://)

To further describe the issue, the greyed-out Image attached shows that when the stream reverts momentarily to an all greyed-out image, it typically will only partially display colored objects that are in motion, while the static objects in the surrounding background all display as greyed out (perhaps related to the I-frames?)

rma153

Hi Ole, did you solve the issue with seeing Gray overlay on video? I have the same issue, and I have tried both openVPN and Zero Tier, but I still have the same annoying issue. I tried lowering the bitrate and that showed slight improvement. I have not tried adjusting rtpjitterbuffer or other adjustments yet. Do you have any suggestions?