Oobleck: Non-Newtonian Fluid Synthesizer

Oobleck is an interactive, browser-based audio synthesizer that translates physical motion into sound. Mimicking the properties of a non-Newtonian fluid, the audio engine reacts differently depending on the velocity of interaction: move slowly for lush, liquid textures, or strike fast for hard, abrasive, solid sounds.

🌊 Features

• Physics-Based Synthesis: Velocity and "hardness" modulate filter cutoff, distortion, and FM synthesis in real-time.
• Generative Patching: "Chaos Mode" instantly generates unique patches with randomized oscillators, physics, and visuals, powered by Google Gemini.
• Motion Control: Integrated webcam tracking divides the air into 4 interactive zones for hands-free FX triggering and modulation.
• Visualizer: GPU-accelerated particle system that simulates fluid dynamics, complete with particle trails, glow, and reactive camera shake.
• Rhythm Engine: 4-track step sequencer with genre-specific presets (Boom Bap, House, Trap, Dubstep, Metal) and multiple drum kits (808, 909, Acoustic, Industrial).
• Gamification:
  • Score System: Earn points for rhythm precision, chaos triggering, and exploring FX.
  • Leaderboard: Compete for the top 3 spots on the local high score board.
• Patch Management:
  • "The Big Save": Save your chaotic creations into 10 user slots (1-0).
  • Chaos Lock: Lock the randomizer to prevent accidental patch loss during performance.
• Performance Tools:
  • "GRRRR!" Button: Instant bass growl fills and drops with visual glitches.
  • "Chop It Up": Instant 1/64th note gating effects for stutter transitions.
  • "Run Back": Instant state revert/undo for live performance transitions.

🚀 Getting Started

Prerequisites

• Node.js (v18 or higher)
• npm or yarn

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/oobleck-synth.git
   cd oobleck-synth

2. Install dependencies:

   npm install

3. Configure API Key: This project uses the Gemini API for generative AI features.

   • Get a free API key from Google AI Studio.
   • Create a .env file in the root directory.
   • Add your key: API_KEY=your_api_key_here

4. Run the development server:

   npm run dev

   Open your browser to the local host link provided (usually http://localhost:5173).

🎹 Controls

Mouse / Touch

• Move: Modulate pitch, filter, and harmonic spread.
• Click: Trigger "strike" hardness (distortion boost) and transient effects.

Keyboard Shortcuts

Key	Action
Space	Chaos Mode (Randomize)
Alt	Trigger Growl ("GRRRR!")
Esc	Run Back (Undo/Revert)
Enter	Start / Toggle Drums
\	Toggle Camera
P	Toggle Audio Recording (.webm)
Z - M	Play Synth Notes (Chromatic/Scale mapped)
1 - 0	Load User Patches
Q / W / E / R / T	Gate Speed (1/64 - 1/4)
U / I / O	Toggle Arp / Gate / Drums
[ / ]	Master Volume Down / Up
Arrow Left / Right	Crossfade (Drums vs Synth)
Arrow Up / Down	Octave Shift

Webcam Zones

• Top Left: Chop It Up (Gate 1/64)
• Top Right: Grrrr! (Growl Fill)
• Bottom Left: Run Back (Undo)
• Bottom Right: Chaos Mode

🛠️ Architecture

Built with React, TypeScript, and the Web Audio API. The visualizer utilizes HTML5 Canvas with custom physics integration.