VSERPI pi 4b/ general esgl and oF environment updates

video_waaaves
1

PI 4b support update

after literally weeks (possibly months in total at this point) of work trying to configure environments properly i’ve got basic functionality for nearly all the VSERPI on the latest Buster (legacy) raspbian os for pi 4b. have only scratched the surface of potential hardware acceleration but heres some initial bonuses happening on a pi4b with 1g ram, running at SD

-WP can double its framebuffer length and stay 30fps, meaning at least 4 seconds of delay time. alternately this could mean having two seperate buffers of ~2 seconds to draw from, so with some control reconfiguration could mean having basically 2 wp running at once

drawbacks

-temporal vortex doesn’t run as of yet, but haven’t really dug into the memory allocation sides of things and could be quite trivial

-artificial life has noticable screen tearing. guessing this has to do with multiple heavy duty shader passes per draw cycle

-hello world is broken as seems to be no native windowing input for ‘enter/carriage return’ and ‘backspace’

-biggest one is that so far i haven’t found an easy way to autodetect and hotswitch hdmi/analog outputs, nor have i heavily tested really anything with analog output. much of the hardware acceleration pi4s have is due to the internal clock not being necessarily enslaved to cvbs video output timing signals unless specifically configured to, thus much of the acceleration might disappear when analog output is forced

either way i’m backing up my current pi4b dev image and can share with anyone looking to experiment with openFrameworks & shaders on pi4bs who doesn’t want to run through a whole rigamarole to get things set up, will have a link somewheres on my site soonish for diyers

8 Likes
2

I’d be happy to try and debug next week when I’m back on the studio. :slight_smile:

3

iwould test ur image on a pi4 when u share it - thanks for the insights !

i also would be interested what you needed to do to configure environemnts correctly for pi4 if u documented it / feel like summarizing @andrei_jay

4

the quick answer was that i used the newest buster image (not bullseye), the latest oF build, and edited some of the oF makefile stuff to disable glfw windowing and force egl windowing instead, which is certainly not without some repercussions.

heres the image, like i said theres still some issues with screen tearing and lack of complete keyboard support but otherwise is looking good
https://drive.google.com/file/d/1GNOyq6xGjN5OmWOhBi1A-UJoJz67DYu0/view?usp=sharing

this is the file i edited to remove all glfw forcing, i don’t remember offhand all of what i did but all the changes are in this section of this file

2 Likes
5

Sweet! Thanks Andrei! Here’s hoping some stock of rpi4s start appearing in the wild like pokemon! I’d love to get this setup!

6

pi4bs 1g and 2g are all i think anyone should expect to see produced this year on a semi regular basis, keep yr eyes peeled at official resellers and you’ll see some pop up here and there for a 1 per customer kind of deal

1 Like
7

Hey, cool seeing this thread!

Feeling guilty because I did some research into these issues in 2020 but never shared the findings. This year I’m starting a project to create a generalised C++ / OF backend for video synths that targets both 3B / 4B. For both of these I’d like to use the new 64bit version of RPi OS (circa Feb 2022) in Bullseye (circa Nov 2021).

The conclusions I came to in 2020 were:

RESOURCES
=========
https://www.raspberrypi.com/news/raspberry-pi-os-64-bit/

VLC as replacement to OMXPlayer (hardware acceleration)

For this I forked an older addon ofxVLCPlayer and got it working on OSX - in some cases VLC needs to be recompiled with a hardware acceleration flag first. The addon interacts with the dylib and should display the texture zero-copy :sparkles: So fingers crossed closer in performance to OMX…

RESOURCES
=========
https://github.com/autr/ofxVLCVideoPlayer
https://forums.raspberrypi.com/viewtopic.php?f=29&t=257395

Libcamera as replacement for OMXCamera (hardware acceleration)

I made an excited fuss about this a while back, and some example apps were released in 2020 - but I couldn’t yet grok them.

After getting on the IRC channel someone gave me a really helpful low down of how the API works - explained some API conventions which can be confusing. I saved the chat log for future reference and will attach it to this post - it’s definitely the solid starting point for an ofxLibcamera addon.

RESOURCES
=========
https://github.com/raspberrypi/libcamera-apps
https://github.com/raspberrypi/libcamera-apps/blob/main/apps/libcamera_vid.cpp

libcamera_chat.txt (9.0 KB)

ISF for shaders (cross-platform simplicity)

ISF is great but the specification is fuzzy and there are now two versions (of which it isn’t made clear which is which - or what the implementation should be).

I wrote a JSON schema into this fork last weekend and am now trying to grok how it should be implemented via the compiled output of ISF_Editor (which isn’t open sourced).

ofxISF is what I forked previously and what I think is the best starting place - the initial addon was first written about 6 years ago but I think is 90% of the way there. It basically just needs uniform updates for the V2 spec and some cross-platform conditionals (ie. use this function name if ES2, use this function name if GL2 / 3 and so).

VVISF-GL is the way to go for ISF!

RESOURCES
=========
~~https://github.com/autr/ofxISF~~
https://github.com/mrRay/VVISF-GL
https://github.com/autr/ISF_Spec
https://isf.vidvox.net/desktop-editor

Dual screens & desktop configuration

I did some research into SPI / I2C LCDs alongside composite video output, and came up with some notes and configurations to have the desktop environment spread across both. Tim also uses the SPI LCD for r_e_c_u_r so I figure it could be useful for instances where we want a touchscreen GUI alongside composite output.

The repository piconf backs up and restores various config files as defined in the conf.js file.

RESOURCES
=========
https://github.com/autr/piconf

Addêndüm

I haven’t reached out to any of the maintainers or creators of the libs mentioned above yet (except for libcamera) - so this could be good to do. Also lots of notes spread all over the place. In the coming months I will try to tidy up and organise them!

MXB

Working title for this project is MXB

3 Likes
8

Awesome!!
Really helpful info and notes.
Thanks a lot!

1 Like
9

Please contribute to MXB! There is a working repository here:

https://github.com/autr/mxb

It’s starting off very ground-up, working on a good base API structure before filling in the features (where some amount was written in previous incarnations).

Re.ISF, I totally overlooked VVISF-GL which is the official VidVox implementation that works cross-platform and on RPi. Also wrote some notes on implementation and V3 ideas here.

1 Like