Application Development

From Boundary Devices Wiki

Revision as of 06:37, 10 November 2024 by Chris@eyenamicdesigns.com (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

//

This section will detail some details about some of the application development frameworks we’ve came across in the past.

Linux

This section will described the different options for our supported Linux distribution (Yocto, Buildroot, Ubuntu/Debian).

Qt

Maybe the most famous UI framework nowadays, offering many multimedia/connectivity features.

Some demos / blog posts have been published about Qt:

A common question about Qt is its licensing which can be unclear.

Therefore the QtCompany has created a webpage that should answer all your questions:

Crank

Another very attractive option is Crank which offers a unique UI experience.

Many of our demos leverage this framework:

GStreamer

GStreamer is the de-facto Linux Multimedia framework:

It can be used along with any of the above UI framework to process/control multimedia content.

It is therefore the preferred framework for Video decoding/encoding/streaming.

For those not really familiar with Gstreamer and its tools, here is a non-exhaustive list of tools that can help you get started.

gst-inspect

Provides information about a particular plugin

  • Very useful to find and debug capabilities
  • Useful to see which GObject properties or which signals an element supports

It can list all the plugins present on the target:

$ gst-inspect-1.0
lame: lamemp3enc: L.A.M.E. mp3 encoder
videoscale: videoscale: Video scaler
volume: volume: Volume
videotestsrc: videotestsrc: Video test source
...

Or it can give specifics about one plugin:

$ gst-inspect-1.0 imxvpudec
Factory Details:
 Rank primary + 1 (257)
 Long-name Freescale VPU video decoder
...
gst-launch

Used to create and test pipelines (prototyping).

$ gst-launch-1.0 filesrc location=/root/tears_of_steel_1080p.webm !
matroskademux ! imxvpudec ! imxipuvideosink

This tool will also allow you to easily troubleshoot issues by adding some log using the GST_DEBUG shell variable. It will then change the log level whose value range between 0 and 5 (5 will show all messages).

$ GST_DEBUG=5 gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm

You can specify which plugin you want to add traces to:

$ GST_DEBUG=imxvpudec:5 gst-launch-1.0 playbin
uri=file:///root/tears_of_steel_1080p.webm
Examples

Camera (MIPI) Streaming to display using 3D GPU:

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video1 ! imxeglvivsink

Camera (Parallel) Streaming to display using IPU:

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 !
imxipuvideosink use-vsync=true

Camera Streaming to display using PXP (i.MX7 / i.MX6SX):

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video1 !
imxpxpvideosink use-vsync=true

Movie playing:

$ gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm

Movie playing + Camera Streaming at once:

$ gst-launch-1.0 playbin uri=file:///root/tears_of_steel_1080p.webm
video-sink="imxipuvideosink window-width=640 window-height=480"
imxv4l2videosrc device=/dev/video0 ! imxeglvivsink window-x-coord=600
window-y-coord=400

h.264 stream over UDP with RTP and camera as source (courtesy of shondll):

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video1 fps-n=30
capture-mode=4 ! imxvpuenc_h264 bitrate=1000 ! h264parse !
rtph264pay ! udpsink host=192.168.11.11 port=5001

For more examples, please read this blog post and especially the comment section since some people shared their own pipeline commands.

Yocto Eclipse IDE dev

If you want to develop and debug an application using the Eclipse IDE on Yocto, here are a couple of documents to read:

Android

Android is famous for its easy to use IDE/SDK combination that offers flexible and well documented API.

Although our platforms supporting Android act just like any other Android phone, this topic has been covered in a previous articble.

Android API/Documentation:

QNX

QNX also provides an easy to use IDE/SDK combination:

QNX API/Documentation: