I was testing out some real-time video matting in Max/MSP, that we developped for an ongoing project, and realized that “sunset.jpg” demo file was a nice match for this memorable dialogue in Romero’s “Night of the living dead”… If it weren’t for all those zombies outside, it could have been such a lovely campfire scene!
We coded an ONNX wrapper with my friend Anthony Rushforth to achieve real-time video matting in Max, using neural networks on trained models. So far it doesn’t work too bad though there’s probably room for optimization: around 10FPS to process Full-HD on the CPU of a MacBook Pro M2. Next step will probably lead us to use the GPU.
Still some visible ghosts and zombies (like “Barbara” Judith O’Dea disappearing hand, or below the arm of “Ben” Duane Jones at the end of the above example), but a pretty decent result otherwise. I was fearing an algorithmic bias on Duane Jones’ dark skin (also simply because of poorer contrast with the background at this timecode) but it does work nicely.
