update todos

src/2023-06-21.festl_is/main.frag

@@ -47,6 +47,25 @@ vec3 ditherBayerLut(vec3 ref) {
 }
 #include "../../lib/lygia/sample/dither.glsl"
 
+//custom helpers -- TODO check if lygia can do this?
+// https://gist.github.com/983/e170a24ae8eba2cd174f
+vec3 rgb2hsv(vec3 c)
+{
+    vec4 K = vec4(0.0, -1.0 / 3.0, 2.0 / 3.0, -1.0);
+    vec4 p = mix(vec4(c.bg, K.wz), vec4(c.gb, K.xy), step(c.b, c.g));
+    vec4 q = mix(vec4(p.xyw, c.r), vec4(c.r, p.yzx), step(p.x, c.r));
+
+    float d = q.x - min(q.w, q.y);
+    float e = 1.0e-10;
+    return vec3(abs(q.z + (q.w - q.y) / (6.0 * d + e)), d / (q.x + e), q.x);
+}
+
+vec3 hsv2rgb(vec3 c)
+{
+    vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
+    vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
+    return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
+}
 
 
 void main() {

src/2023-06-21.festl_is/notes.md

@@ -1,7 +1,7 @@
 # notes
 
 
-## video streaming
+## video streaming notes
 
 * using glslShader is buggy, setting time with `stream,ID,time,0` buffers and then changes playback speed. not good
 * idea: use v4l to stream to `/dev/video2` and read from there
@@ -15,4 +15,26 @@
             * dont want to deal with on the fly reload and mapping the controls?
                 * but wouldn't mapping the controls be easier this way anyway? m)
         * example v4l2 processing export [TODO test]: https://gist.github.com/UriShX/fc34825843c36849af2dbc49afa1d0a6a
-        * 
+            * would need to build control for everything, meh
+    * idea: use OBS-Studio
+        * yeah that works. also might give some audio info
+
+## prelim setup
+
+* OBS-Studio
+    * plays with vlc video source
+    * streams through v4l2 with "start virtual camera"
+        * check which output it is with `v4l2-ctl --list-devices`
+    * also collects audio from line in for loudness control
+* qpwgraph
+    * routes line in to obs
+    * routes obs to processing
+* processing
+    * collects audio in
+    * runs audo analysis and midi control
+    * sends osc to glslViewer
+* glslViewer
+    * runs shader, listens to osc
+
+This is, admittedly, bullshit. Running obs should not be necessary, but it provides the most stable stream that I can control without programming my own controls (ffmpeg, gstreamer don't do realtime skipping and vlc doesn't do v4l2 streaming). glslViewer might be silly here, but it allows to adjust the effect in realtime and provides includes for the shaders, no idea if processing would support it. also, doing the audio processing in glslViewer is silly because then every friggin pixel would calculate on the audio - no thank you very much. Sending osc 30/s is overwhelming glsl tho, so we might have find another way of them communicating. Processing can do shaders and possibly read from v4l2 video and recursive shaders (see lygia examples). But hot reloading would mean fixing and changing things would all happen in the same sketch, and if the audio processing needs work we can't do that. There's an idea how to do it over at https://github.com/processing/processing/issues/5648 tho.
+

src/2023-06-21.festl_is/todo.md

@@ -1,27 +1,55 @@
 
 * preparation
-    * [ ] make sure videos are in right format
-    * [ ] control playback of mp4 in glslviewer?
+    * [X] make sure videos are in right format
+    * [X] control playback of mp4 in glslviewer? -> no, we use obs for that now
+    * [ ] put videos on laptop
 
 
 * Makefile
-    * [ ] add command to extract ratio from mp4
+    * [ ] add command to extract ratio from mp4 -> is this necessary
+    * [X] disable magenta error screen
     * [ ] make command to use other monitor + fullscreen
 
 
 * code
-    * [ ] barebones shader with lygia effects
-        * [ ] needs a texture to apply effect -> stacking them decision
+    * [X] barebones shader with lygia effects
+        * [X] needs a texture to apply effect -> stacking them decision
             * [ ] option1: all FX goes on raw video, then combine
                 * [ ] probably easier and more modular, seperates code for FX and order
-            * [ ] option2: gigure out how to write to texture so pipeline is possible
-    * [ ] test communication via osc with processing
-        * [ ] processing sketch with g4p knobs that control shader values
-        * [ ] how does it default if no input is coming? can I adjust from glslViewer cli?
-    * [ ] what are the weird issues with the glslViewer cli? is it an ncurses problem?
-    * [ ] processing
-        * [ ] processing.sound not very powerful, but joshuas trick for normalization maybe possible? try that.
-        * [ ] minim offers logarithmic scaling -> likely much better for analysis and what I want to do
+            * [ ] option2: figure out how to write to texture so pipeline is possible
+    * [X] test communication via osc with processing
+        * [X] processing sketch with g4p knobs that control shader values
+        * [X] how does it default if no input is coming? can I adjust from glslViewer cli? -> black or last frame, doesn't matter
+    * glslViewer
+        * [.] passes
+            * [X] brigthness pass
+                * [X] POC
+            * [ ] edge detection pass
+            * [ ] something that slow it down, some time-based blur
+            * [ ] some feedback
+            * [ ] some color shift
+            * [ ] noise threshold - dithering
+            * [ ] ideas from shadertoy:
+                * [ ] glitch: ngMir5 https://www.shadertoy.com/view/XtSGRG
+                * [ ] glitch: mpeg artifacts https://www.shadertoy.com/view/Md2GDw
+                * [ ] blur: fire ghosting https://www.shadertoy.com/view/XsdGWj
+                * [ ] blur: motion blur https://www.shadertoy.com/view/Xs33DS
+    * processing
+        * [X] processing.sound not very powerful, but joshuas trick for normalization maybe possible? try that.
+        * [X] minim offers logarithmic scaling -> likely much better for analysis and what I want to do
+            * yes we go with that
         * [ ] make gui for sound analysis
-            * [ ] fft settings etc
-        * [ ] make midi ctrl for settings so we don't have to implement drag+drop
+            * [ ] beat detection output
+            * [ ] fft settings from midi
+                * [ ] 2 bands
+                * [ ] spectrum viz?
+            * [ ] 
+        * [ ] pause-osc button in g4p
+
+* bugfixes
+    * [X] DEBUG: what are the weird issues with the glslViewer cli? is it an ncurses problem?
+        * [X] SOLUTION: use --noncurses 😅
+    * [X] FIXME: change default color on error from hot pink, this is horrible
+        * [X]  howto: see constant at https://github.com/patriciogonzalezvivo/vera/blob/ed5076fcb28db4b376b2ba4e8a22d264a8661db9/include/vera/gl/shader.h#L19C1-L19C1 and fix in https://github.com/patriciogonzalezvivo/glslViewer/blob/main/src/core/sandbox.cpp#L68
+        * [X] actually implement
+        * [X] deploy on laptop and desktop