Exploring Physics and Sound: A Deep Dive into WebGPU and TSL in "Garden Anomaly"

The "Garden Anomaly" project exemplifies how modern graphics and audio rendering technologies like WebGPU and the Three Shading Language (TSL) can create immersive interactive experiences. Developed by Frank Reitberger, this unique experiment highlights the intricate relationship between physics simulations, visual effects, and sound design, all wrapped inside an aesthetically pleasing interactive environment.

The experiment first sets the stage with a translucent shell effect and representative elements that give the feeling of depth and fluidity. Reitberger crafted the demo using the WebGPURenderer with TSL node materials, pushing the boundaries of what can be achieved with web-based graphics technology today.

Conceptual Framework

Reitberger's path into this project stemmed from a prior experiment he had conducted a couple of months ago concerning physics and sound. Having a foundational understanding of TSL and WebGPU allowed him to rapidly iterate and innovate within this new project. Rather than relying solely on traditional resources, he embraced a hands-on experimentation approach, drawing inspiration from various existing Three.js examples and the TSL documentation. This iterative process cultivated a strong visual outcome, although technical hurdles often emerged during development.

Rendering Techniques

A closer look at the rendering setup reveals the meticulous attention to detail in crafting the visuals. The environment incorporates a MeshPhysicalNodeMaterial for the glass-like shell, which features various settings such as transmission, thickness, and iridescence. Each visual element incorporates custom TSL nodes that enhance the overall rendering quality:

• Position Node: Utilized for creating bulging impacts through vertex displacement, driven by an array of amplitude vectors.
• Normal Node: Developed with an analytic perturbation model, enabling more realistic light interaction.
• Emissive Node: Features generated visual elements that create an engaging aesthetic, particularly through the use of dynamic UV offsets.
• Instanced Bubbles: These bubbles are represented as opaque for the time being, keeping in line with the rendering limitations currently imposed by transmission passes.

Physics Simulation

The core interaction in "Garden Anomaly" revolves around a Position Based Dynamics (PBD) simulation that meticulously governs the bubble interactions. One of the standout features is how user input creates randomized forces, allowing participants to influence the simulation dynamically. Reitberger calibrated several physical parameters:

• Gravity and Air Drag: Implemented within a semi-implicit Euler framework to enhance realism.
• Sphere Collisions: Three iterations refine the interaction dynamics based on the mass, ensuring smaller bubbles yield to larger ones seamlessly.
• Velocity Calculation: Position changes compute velocities, a hallmark of the PBD method that ensures a more organic settling process.
• Interaction Damping: Refined damping methods create a natural friction effect when bubbles are in contact with each other or the glass shell.
• Impact Dynamics: These detect interactions and couple visual deformation with audio signals for a more immersive experience.

Procedural Audio Elements

Distinct auditory feedback is integral to the "Garden Anomaly" experience, built entirely via the Web Audio API. This approach ensures that all sounds are generated procedurally, allowing for a dynamic range of auditory interaction:

• Trigger System: Each audio cue corresponds to visual interactions, specifically when bubbles impact the glass, ensuring a synced experience.
• Pitched Tones: By linking pitches to bubble sizes, the project utilizes a two-octave major pentatonic scale that supports rich harmonic structures.
• Voice Design: Each note is crafted from multiple sine oscillators, allowing for greater resonance and character that evolves with audiovisual interactions.
• Master Mix: Integrates both dry and flanged signals to create a layered soundscape while managing dynamic range for clarity.

This synthesis of sound and physics amplifies the viewer's experience, fostering a compelling connection between what they see and hear.

The environment's appeal further benefits from a generative skybox produced with AI, offering a stunning backdrop to the bubble-filled experiment. It's clear that each design choice contributes to the engaging atmosphere that "Garden Anomaly" encapsulates.

This project not only serves as a playful interaction but also highlights how creative coding and procedural generation can intersect to create unique web experiences. Reitberger's journey exemplifies the potential held within modern web technologies while pushing the boundaries of what's possible.

To explore this engaging project firsthand, visit the live demo and access the source code on GitHub:

As developers continue to weave together these technological threads, the discussions surrounding potential directions for these tools become increasingly vital. The pathways explored in "Garden Anomaly" are only the tip of the iceberg, suggesting a bright future for web-based creative coding.