KnittedKeyboard: Digital Knitting of Electronic Textile Musical Controllers

Irmandy Wicaksono and Joseph A. Paradiso

Responsive Environments Group, MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA 02139, USA


“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” Mark Weiser

In this work, we explored intarsia, pocket patterning, and a collection of yarns in digital knitting to create a piano-patterned textile for expressive and virtuosic sonic interaction. We combined functional with common yarns to develop “KnittedKeyboard”, both with its physical properties and responsive sensing and display capabilities. The KnittedKeyboard demonstrates a vision towards seamless fabric-based interactive surfaces. The framework would enable further exploration and customization of soft, malleable, and on-body musical interfaces that leverage unique mechanical qualities of the material, as well as electrical properties of the sensors.


Concept

The KnittedKeyboard (Figure 1a) is a machine-knitted, interactive textile surface with multimodal sensing and display capabilities. As illustrated in Figure 2, it consists of 60 capacitive touch and velocity-sensitive keys that can be transformed into five large proximity sensing pads. In addition, it has a piezo-resistive pressure sensing layer underneath for aftertouch expression. The patterns are also thermochromic and can change color with applied heating to visualize different modes of play (Figure 1b). As a second iteration of our previous work [1], this piano-patterned, electronic textile can be connected to any audio synthesis or sequencer software, and mapped to any instrument or sound effects [2]. 


Fabrication

We employed a two-bed flat digital knitting machine (Figure 1e) and fed this machine with the following yarns: two cones of silver-plated conductive yarns, two cones of thermochromic yarns, and two cones of high-flex polyester yarns spun together with melting yarns (Figure 1c) [3]. Each cone connects to a different yarn carrier, with a total of four carriers used. As illustrated in Figure 1d, each carrier knitted a different section of the fabrics. Intarsia knitting was instructed on sections where more than one yarn type is needed, such as in any line where the piano key is plotted. Since it is a two-layer knitting machine, we performed interlock knitting to fuse the front and the back-fabric into a single layer. It took the machine 1 hour and 40 minutes runtime to knit the entire prototype, which has five octaves of diatonic piano-keys pattern (Figure 2d). The resulting knitted fabric was steamed at the end to activate the melting yarn, giving structural rigidity to the final prototype.


Musical Mapping

In Ableton Live 9, there are two active channels. We assigned the

touch events and velocities to Channel 1 (synth keys and leads, such as suburban or brassicana lead or a classic piano rack). The pressure-sensing values as aftertouch or expressions are also mapped into Channel 1, for either track volume or filter frequency. A user can go into the proximity mode for sensing hand’s waving and hovering, which we assigned to Channel 2. In this case, we applied ambient and evolving sound effects (spacefolder or cosmos scape). Amplitude modulation can also be mapped into this channel as our hands get closer to the surface.



Project Link and Demo 

https://www.media.mit.edu/projects/knittedkeyboard/

References

[1] Wicaksono, I. and Paradiso, J.A., 2017. Fabrickeyboard: multimodal textile sensate media as an expressive and deformable musical interface. In NIME (pp. 348-353).

[2] Paradiso, J.A., 1997. Electronic music: new ways to play. IEEE spectrum, 34(12), pp.18-30.

[3] Ou, J., Oran, D., Haddad, D.D., Paradiso, J. and Ishii, H., 2019. SensorKnit: Architecting textile sensors with machine knitting. 3D Printing and Additive Manufacturing, 6(1), pp.1-11.