Vimeo - Compilation of social media video posts

Rcam is an experimental project that uses a real time volumetric video capture system for live performance visuals.

Rcam consists of the following technologies:

• Unity
  • Visual effect graph
  • High definition render pipeline
• Intel RealSense
  • D415 depth camera
  • T265 tracker
• Backpack PC (MECHREVO Vest PC II)
• TouchOSC
• NewTek NDI

Volumetric data is captured in real time and visualized on the backpack PC carried by the camera operator. Rendered frames are send to a projector via ethernet link using NDI video transfer protocol.

This system was used in a live performance with umio at Channel 20. Here is a video compilation of social media posts.

• Unity 2019.1
• Desktop system with HDRP support
• RealSense D415 and T265
• TouchOSC

To control visuals, upload Rcam.touchosc to TouchOSC app using TouchOSC Editor. Please note that Windows Defender Firewall has to be turned off to receive OSC signals on a Windows system.

Also you can try the effects in the Editor by manually enabling visual effects from the Inspector.

I designed a custom bracket for this purpose, which is available on Tinkercad.

If the camera is fixed at a single point, T265 tracker isn't needed. It's used to capture the movement of the camera into the 3D scene. It also used to compensate jitter caused by hand. In other words, it implements a software motion stabilizer.

Special thanks to @z_zabaglione for lending me the backpack PC.