Raspberry Pi Video Overlay on OMXPlayer via Kivy

In one of my hobby projects I needed to display additional information on top of a running video. The whole project is based on the Raspberry Pi and Python.

This article describes the approach I’ve taken to display arbitrary content over the video with the help of the very flexible (and admittedly awesome) Kivy framework.

I use OMXPlayer to play back the video because it uses the Raspberry Pi hardware efficiently enough to play back 1080p video in real time.

Using DISPMANX layers #

DISPMANX is the layer between EGL (OpenGL ES) and the BCM graphics chip in the Raspberry Pi. This API allows the control over the display quite directly and is ultimately used for most drivers that communicate with the display(s) attached to the Raspberry Pi.

The DISPMANX API supports “layers”, which allow different content to be placed for very basic compositing. The best part about this is that layers have alpha channels, so can be (partially) transparent.

Using Kivy for the Overlays #

For video overlays, we need to achieve two points:

1. Define a layer on top of the video player to render the overlay
2. Keep part of that layer transparent, so we see the video underneath.
3. Define all of the overlay elements as widgets in Kivy.

Kivy and DISPMANX Layers #

Kivy 1.9.2 and newer has a rendering backend specifically for the Raspberry Pi, based on the BCM DISPMANX API, which allows direct access to the screen without the need for window managers, etc.

Currently, the selected layer is always 0 but can be added easily in window_rpi_egl.py. I’ll make a pull request to Kivy to add a config variable to define the DISPMANX layer easily.

Someone made one already here: Pull Request for adding RPI DISPMANX Layer configuration

Let’s see when it’s accepted.

Keeping the Kivy Background Transparent #

The Kivy Window will be “cleared” with a black color by default and basically enforces a full-screen black image, on which any widget is rendered.

To see the video underneath the Kivy window, we need to disable this clearing behaviour, so Kivy leaves the background transparent.

# transparent background color
Window.clearcolor = (0, 0, 0, 0)

The best thing about this is that you can define any color and any alpha intensity (level of transparency). The color (1, 0, 0, 0.5) would make the background semi-transparent red for example.

Defining UI Elements in Kivy #

Kivy is a powerful UI framework with lots of documentation and examples. Defining the UI you want overlaid over the video should be reasonably simple.

These two links should help further:

• Kivy documentation, overview of widgets
• Kivy: Rendering a background color for labels

Integrating OMXPlayer #

OMXPlayer is integrated via the pyomxplayer project, with the slight modification that it defines the layer, on which to display the video.
pyomxplayer will create a background thread that checks on the OMXPlayer process and provides callbacks to various states.

Synchronising Playback and Overlay #

For my project I need time-synchronised overlays. pyomxplayer monitors the video’s progress. I’ve extended it a bit to propagate the current playback position via callback. More on this in a dedicated article of my extensions to pyomxplayer.

Why not the Kivy Video Player? #

I’ve tried various alternatives to make my life easier with this.

The first attempt was of course to use the video player that is provided with Kivy. We play back 1080p video, which works great with OMXPlayer, but is just too slow when decoding frames, putting them into a texture and displaying it (I’ve seen between 1-4 frames/s on the Raspberry Pi 2).

That is the approach that Kivy takes and is the regular approach that performs well on more powerful hardware. The Raspberry Pi is just barely fast enough to decode the 1080p video frames and put them on screen with its hardware decoder. There is no room left for working with textures.