Moving Digits technology is made available free as open-source. The following technology repository will be updated throughout the project, but software and code can already be accessed and downloaded:
https://github.com/hci-dance-visuals/
At the moment, it contains 4 tools:
- Sensor-Drawings
- Interactive-Structure
- MoCap-Avatar
- MoCap-Abstract
Interactive-Structure – visualize structure, using pre-prepared animations or video, allowing to make decisions, through mocap and/or a Graphical User Interface (GUI), for visualizing a choreographic structure (using Isadora) that “directs” the actions of the performers, simulating a fictional digital character;
Mocap-Abstract – playing with temporal elements, using abstract generative graphics (with Processing), that are loosely connected to the movement and dynamics of the dancer, tracked via mocap;
Mocap-Avatar – also playing with time, using mocap and resulting 3D avatar to create an onscreen “double” of the dancer, that can be manipulated in terms of speed, appearance and background (using Isadora); and
Sensor-Drawings – reveal the internal, using visuals generated by the machine learning engine developed, trained with a database of bodily map drawings by the dancers (using Max/MSP and TensorFlow).
The first three prototypes use mocap as the main interaction technology (in addition to a GUI on Interactive-Structure), whereas Sensor-Drawings uses BITalino R-IoT on-body sensors.