FaceMusicMISK

Play music using your face. Tilt your head up and down, and vary the shape of your mouth. Connect to the MISK music software for more options.

This app allows you to play music using your face. It has basic sounds built in, but to get a wider range of musical options it needs to be connected to the MISK music software, which can be found at https://goteborg.se/wps/portal/enhetssida/eldorado/kunskap/misk/bygg-din-egen-misk . For the connection, either an open wifi (like a home wifi) or midi can be used. To use midi, the device has to be activated as a midi device in AudioMidiSetup, and has to be selected under sound as a midi unit in the MISK music software. The app was developed in the MISK project, and is provided as is (for free).

The app needs a device with a front facing camera with true depth for the face tracking. This camera is used to track your face using the Apple ARKit, extracting parameters for things like head position, mouth width etc. These parameters are used to control the music in the built in music software, or - if you select this in the settings – are sent via OSC or midi to the MISK MaxMSP music software patch on a computer for a more advanced music experience. You need to input the IP number of this computer, in order for the OSC connection to work. The control is in real time, and the face parameters are not saved or stored. The app does not save or send any personal data, or share it with any 3rd party.