{"schemaVersion":1,"record":{"key":"mechanic/spatial-interfaces--t305","type":"mechanic","id":"spatial-interfaces--t305","lang":"en","title":"Musical instruments and specialized controllers","summary":"A familiar performance gesture sends commands to an app. AudioSwift’s demo uses musical keys to trigger notes and a trackpad to shape sound.","updatedAt":"2026-10-22","status":"active","url":"https://gamemechanics.pro/en/knowledge/mechanic/spatial-interfaces--t305/","jsonUrl":"https://gamemechanics.pro/api/v1/knowledge/en/mechanic/spatial-interfaces--t305.json","markdownUrl":"https://gamemechanics.pro/content/knowledge/en/mechanic/spatial-interfaces--t305.md","aliases":["Musical and specialised controllers"],"tags":["Spatial and alternative interfaces","Mechanics and variants"],"sourceIds":["source-3e8db7ac782a4997","source-fd215230c062deb7"],"sources":[{"id":"source-3e8db7ac782a4997","title":"W3C: Web MIDI API","url":"https://www.w3.org/TR/webmidi/","verifiedAt":null},{"id":"source-fd215230c062deb7","title":"AudioSwift","url":"https://audioswiftapp.com/","verifiedAt":null}],"sections":[{"id":"principle","title":"How it works","body":"Specialized controllers translate familiar performance gestures into game actions through explicit event mappings."},{"id":"variants","title":"Variants","body":"### Mechanic variants\n\n<a id=\"t305-variants\"></a>\n\n- Notes select actions through a mapping table. <a id=\"t305-v1\"></a>\n- Event strength modulates values when supported. <a id=\"t305-v2\"></a>\n- Pedals change modes or sustain effects. <a id=\"t305-v3\"></a>\n- Calibration aligns input latency with music. <a id=\"t305-v4\"></a>"},{"id":"example","title":"Design example","body":"A music controller's keys open gates in sequence while a pedal holds an opened passage.\n\nPrototype check: Test actual event formats, simultaneously held notes and a keyboard substitute when the controller is absent."},{"id":"context","title":"Context and limits","body":"Spatial and specialized input requires calibration, clear action confirmation and a way to continue after signal loss. Each overview separates player actions, state changes and alternative rules. Examples are hypothetical design scenarios, not claims about a named game. A prototype check states an observable question for the next iteration. Existing detailed references remain separately linked from the direction map."}],"relations":[],"placements":[{"catalogId":"spatial-interfaces","id":"t305","title":"Spatial and alternative interfaces","url":"https://gamemechanics.pro/en/mechanics/spatial-interfaces/#t305","categoryIds":["mechanics"]}],"supersededBy":[],"attributes":{"preview":{"updatedAt":"2026-10-22","title":"Musical and specialised controllers","description":"A familiar performance gesture sends commands to an app. AudioSwift’s demo uses musical keys to trigger notes and a trackpad to shape sound.","image":{"src":"images/mechanics/control-audioswift-3.webp","width":1024,"height":576,"game":"AudioSwift","sourceUrl":"https://audioswiftapp.com/","originalUrl":"https://www.dropbox.com/scl/fi/fc2ycw35tusq9f0joj5s5/04-Core-Features-XY-Pads.mp4?rlkey=hc7s0jir9zb5du0z08f9xipon&st=nltffpdt&raw=1","sourceFrameSeconds":3,"credit":"Nigel Ríos; host apps and instruments retain their respective rights","rights":"Imagery © Nigel Ríos; host apps and instruments retain their respective rights. Developer-distributed screenshot or frame; all rights reserved. Excluded from MIT and CC BY 4.0.","caption":"AudioSwift XY: a trackpad shapes sound during keyboard performance","alt":"AudioSwift XY: a trackpad shapes sound during keyboard performance","markers":[{"x":24,"y":55,"label":"The XY point maps two motion axes to sound parameters"}]},"tips":["Show how physical controls map to notes or application actions."],"examples":[{"title":"AudioSwift","url":"https://audioswiftapp.com/"}]}},"endsAt":null,"provenance":"Original bilingual editorial overviews authored for gamemechanics.pro on 2026-10-20 and reviewed on 2026-10-21. Scenarios are hypothetical design examples; prototype checks describe work to perform, not completed playtests. Primary sources support the stated techniques and implementation context, not every proposed variant.","license":"CC-BY-4.0","codeLicense":"MIT","attribution":"gamemechanics.pro contributors","catalogKey":"catalog/spatial-interfaces"}}