SimTouch

SimTouch

Environment for Embedded Particle Simulation and Actuator Array Control

Live Demo and Full project on Github

This project’s aim is to develop a particle simulation engine designed to generate organic and coherent control signals for actuator arrays (servos, LEDs, etc.) on touch-enabled embedded systems. It serves as an addon to a parent Haptic/VR project but could be used for wildy different purpose such as a movement reactive pendant (emu/graphics), sound reactive art installation (hardware/audio/PWM) etc. But most importantly, it enables rapid protyping development/testing platform for embedded devices.

The development follows a two-phase strategy:

  1. Phase 1 : JavaScript Prototyping Environment (Sim/)
    • This environment uses JavaScript and WebGL2 for rapid development and visualization of the core simulation algorithms. It leverages the immediate feedback loop of web development for efficient tuning of physics and behaviors, which is difficult in typical embedded workflows (compile, upload, flash).
    • The core simulation logic is written with portability in mind, minimizing external dependencies, suitable for refactoring to C++/embedded environments.
  2. Phase 2 : Embedded Deployment
    • The validated core logic from Phase 1 will be ported to a target embedded system C++/Arduino. The touch-enabled embedded device will run the simulation natively to create novel, organic control methods for actuator arrays (LEDs, servos, etc.).

While the Waveshare ESP32-S3-Touch-LCD-1.28 is used as the default reference target for parameters, the simulation core is designed for any touch-enabled devices.

Phase 1: JS Prototyping Environment Features

  • Purpose: Rapidly develop, test, visualize, and tune core simulation algorithms intended for the embedded target. Acts as a functional specification.
  • Simulation: Real-time particle-based simulation including:
    • Fluid dynamics (PIC/FLIP methods)
    • Collision detection and response
    • Force fields (Turbulence, Voronoi, Gravity)
    • Noise driven effect field.
    • Boundary interactions
    • Organic behaviors (Swarm, Automata, Chain – planned/experimental)
  • Rendering: WebGL2 rendering for visualization and debugging (Particles, Grid, Boundaries, Data) and use instantiation for grid rendering performance.
  • Portability Focus: Core simulation logic is separated from web-specific rendering/UI components. (See memoLTM/goals.md for detailed scope).
  • Input Handling:
    • Mouse interaction (simulating touch input + receive UDP)
    • External Input Integration (EMU forces via UDP)
    • Microphone input processing with a comprehensive band filtering/visualisation + parameter modulation control
    • Joystick simulating gravity (act as an abstraction and tunning for external inputs)
  • UDP Communication: Sends simulation state (grid specs + cells data) to an external device (acting as a “Slave” display in Phase 1). Each cell’s data in the grid conceptually represents the state for a corresponding external actuator (potential 1:1 mapping), or specific cell indices can be selected for targeted data extraction. Also sends touch and Emu data back to Js Sim.
  • UI Controls: Interactive UI (lil-gui) for real-time parameter tweaking.
  • Modulation: Includes a Pulse Modulator system synced on a Beat Per Minutes.
  • Embedded device: Simple firmware, acting as slave, mirroring Sim’s grid and sending back Touch and Emu data via UDP. Serial is used for runtime debugging.

Phase 2: Embedded System

  • Objective: Port the core simulation logic (developed in Phase 1) to a target embedded system (e.g., ESP32-S3) providing a flexible system adaptable to various embedded targets and actuator types.

  • Functionality:

    • The embedded device runs the simulation natively.
    • Handles inputs directly (touchscreen, sensors like IMU, audio).
    • Outputs control signals for connected actuator arrays.
    • The JS application (Sim/) transitions to a remote control/configuration tool via UDP/API.

Inspirations

Heavily inspired by mitxela and he’s amazing fluid pendant as well as Ten Minute Physics for his high quality recources and informations.

License

This project is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0). See the full license text here: https://creativecommons.org/licenses/by-nc-sa/4.0/

Details

Date:  4th May 2025
Skills:  Embedded Systems
URL:  Launch