A desktop app that turns your webcam into a MIDI controller: chords, strums and melodies from your hands, expression from every tilt and pinch, straight into any DAW.
Five stages. No sound, only MIDI.
720p or 1080p, 30 or 60 frames, hands from the waist up.
Hands and face, tracked on your own machine. Nothing uploaded.
A pinch, a finger count, a fist. Roll, height, a head turn.
Each mapping is one message on a channel you choose.
Route it to any instrument you already own. Your synths are the sound.
One camera. Many instruments.
Count a chord.
Fingers up is the degree. A fist cuts.
Strum it.
The chord hangs as strings. Pinch, and cross them.
Draw a melody.
One voice on a ladder of the scale, right beside your hand.
Bend it. Tilt it.
Every roll, pinch and turn of the head rides a CC or a pitch bend.
Every card is a MIDI message you choose.
A card is a gate with rows inside. The gate is the pose that opens it. Each row is one movement wired to one MIDI message, sending only while the gate is open. That is the whole model.
Set the gate
A gate is a pose that opens the card: always, a pinch, a fist, three fingers up, a slap. While you hold it the rows inside send. Let go and everything the card started stops. A gate can also play a note itself: on when it opens, off when it closes.
Pick a movement
Palm height, roll, a pinch distance, a head turn. The menu meters the movement live while you choose.
Pick the message
A CC number and channel, a note, or pitch bend. Whatever your synth already listens to.
Shape the response
Input range, output range, curve, direction. Deadzone and smoothing if you want them.
One port. Any track input.
No plugin, no setup in the DAW. The app creates one virtual MIDI port and every host lists it as an input. Pick it on a track and play.
Product names and marks belong to their owners. Listed to say the port shows up, nothing more.
Nothing leaves your device. Nothing waits on a server.
Tracking, mapping and MIDI run inside one desktop app. No account, no cloud, no upload. The Perform view shows camera-to-MIDI latency live, so you never have to guess whether it feels late.
Hear about the beta first.
One mail when the beta opens, one when it ships, and the occasional clip of something new the hands learned. That is the whole list.
- Beta
- Closed. The list gets the download link first.
- Platforms
- Windows and macOS, one license for both
- Camera
- Any webcam, 720p or 1080p, no extra hardware
Try the idea in your browser first.
GestureMIDI grew out of gesture.live, a free instrument that plays in a browser tab. Same hand grammar, same blueprint. One makes sound on its own. The other plays yours.
- Drums, chords, bass, melody, effects
- Nothing to install, nothing uploaded
- Record a take and share it
- Your DAW, your synths, your effects
- Map any gesture to any message
- Presets, solo, learn, latency readout
Made by Robin van Soelen, a musician and developer in the Netherlands, building on the hand tracking in MediaPipe and the JUCE framework. Progress and clips on Instagram.
- Runs on
- Windows 11 · macOS on Apple Silicon
- Camera
- Any webcam, 720p or 1080p, 30 or 60 fps
- Output
- One virtual MIDI port, MIDI 1.0 messages