In this Episode
Ruben Frosali, also known as RubenFro, is a visual effects artist known for his pioneering work with volumetric capture and point clouds. Ruben delves into his artistic journey, from explaining his deep passion for point clouds and more recent experiments with Gaussian Splatting. He sheds light on the technological advances that have shaped his work, including his preference for Unity in creating complex VFX and his contributions to augmented reality content at Jadu AR. Join us to discover how Ruben blends technology and art to create captivating, immersive experiences.
Interview with Ruben Frosali
Host: Ruben, can you share how your fascination with particles and effects started?
Ruben: I’ve always been drawn to the intersection of technology and art. My journey began as a developer, and I developed a particular interest in photography and video making. In 2018, I delved into photogrammetry and point clouds through an interactive installation project. This technology, which captures reality in a unique way, really captivated me. Each point in a point cloud is like an atom, and using them in particle effects feels like playing God with reality. It's an exhilarating experience, transforming and creating new things from these fundamental elements.
Host: Can you explain the differences between point clouds, Nerf, and G Splat technologies?
Ruben: Point clouds are the simplest form of volumetric capture, representing scenes with precise points in space. However, they can have imperfections, like missing details on uniform surfaces. Nerf uses neural processing to generate images, providing more detail and reflections but is harder to implement in real-time engines. G Splat, a newer technology, renders points as transparent blobs to create realistic images, balancing detail and ease of manipulation.
Host: How did you navigate the challenges of working with new technologies like Nerf and G Splat in real-time engines?
Ruben: Nerf and G Splat represent significant advancements in capturing and rendering reality. Nerf offers advantages like reflections, but it’s quite challenging to use in real-time engines, especially for dynamic effects like particle systems. G Splat, on the other hand, is more accessible and manageable in real-time engines. I adapted my libraries for point clouds to work with G Splat, allowing for more realistic and detailed visual effects that are also easier to manipulate.
Burning Gaussian Splat in Unity3D. pic.twitter.com/kwYlkNf7lU
— 𝐑𝐔𝐁𝐄𝐍🥽𝐅𝐑𝐎 (@Ruben_Fro) October 5, 2023
Host: You've mentioned using Unity over Unreal for your VFX work. What led to this choice?
Ruben: Unity offers greater flexibility in coding and shader creation, which is crucial for my work with point clouds and effects. While Unreal has better rendering and lighting systems, I’ve found Unity to be more customizable and suitable for the unique requirements of my projects. I’ve built a substantial library of effects in Unity, allowing me to render and manipulate point clouds effectively.
Host: How do you integrate sound into your visual projects, and what role does it play?
Ruben: Sound is an integral part of my creative process. I often compose music and layer samples that complement the visual aspect of my work. For projects like “Dissolving Realities,” I even converted point cloud data into audio wave files, adding another dimension to the experience. This process, though abstract, allows me to create a more immersive and holistic artistic expression.
Host: As Head of Cinematic at Jadu AR, what’s your approach to creating augmented reality content?
Ruben: At Jadu AR, we focus on creating immersive AR experiences, including a multiplayer AR fighting game. My role involves creating cinematics and concept art for AR interactions. We’re pushing the boundaries of what’s possible in AR, especially in terms of realism and interactivity. Working with volumetric content in AR presents unique challenges and opportunities, which we’re excited to explore.
Host: How do you envision the future of XR and AI in your creative work?
Ruben: XR and AI are rapidly evolving fields that offer endless possibilities for creative expression. I’m interested in exploring mixed reality and more interactive experiences, especially with volumetric content. The development of technologies like G Splat and advancements in VR and AR hardware will enable more detailed and realistic experiences. I’m excited to see where these technologies take us and how they’ll shape the future of digital art and interaction.
Host: Ruben, thank you for sharing your insights and experiences with us today.
