This work aims at investigating the relationship between creation and fruition of art, in works that show  intelligent responsiveness to viewer input. Which is your brass voice? is an interactive installation in which the audience members are partners in constructing meanings and producing contents. Using artificial intelligence techniques, the artwork analyses the individual nuances of people’s voices and creates a new music composition in real time on the base of its vast musical knowledge. 

Users’ voice is transformed into sound and coloured lights for exploring rich aesthetic interactions. 

Keywords: computational creativity, intelligent systems, software art, digital media, artificial intelligence,   interactive installation,  installation art, embedded electronics, public engagement, open work, audience experience, interdisciplinary collaboration, co-creation.

The piece invites people to interact by speaking, singing, shouting or whispering into microphones; each of the five microphones represents a specific brass instrument: trumpet, trombone, tuba, muted trumpet and french horn sounds are generated in response to the captured input of people voices. 

The artificial intelligence behind this piece records all the notes performed and creates a new music composition continuing the musical sentences collected.

The sequence of notes gathered from the microphones becomes the seed of new compositions generated from the large musical knowledge-base of hundreds of brass scores a note at a time. The brass sounds played back through surrounding speakers are synchronized with the light events of an array of coloured LEDs to create a fully immersive aesthetic experience. [1] 

Which is your brass’s voice? surrounds the visitors with sounds and colours that respond to their performance and therefore enable experiences that are unique for each participant.

The installation creates a cycle of interaction that transforms viewers into active spectators [2], immersing them in an ambience consisting of sound, lights and real-time processing; its behavior is entirely based on the involvement of the public thus embodying interaction [3] and encouraging social participation. 

Can the exhibition become a site for artistic co-creations? How can such an environment trigger creativity in the participants?  

The last part of the following video shows a spontaneous performance of a street artist who was just passing by and  started to improvise in front of the artwork, amazingly the day after he came back along with his  performance band, and started interacting  with the piece in a natural action of ‘doing and receiving’.

During the exhibitions hundreds of people of different ages and backgrounds have interacted with the piece and each others, bringing their own personal story; with our creative practice we are trying to research  how co-creation and collective processes can transform spectators into partners in constructing meanings and producing contents. 

Technical Description

The installation’s software builds on a large pool of algorithms, libraries and standards both from commercial and open source environments, bringing together music generation and interaction [4].

Five real time frequency analyzers convert the input from the microphones into MIDI messages that the C# application receives and analyses through a number of deterministic and stochastic algorithms

• Analysis of musical scales is performed deterministically;
• Stochastic classification of the input is performed against a knowledge base to select a musical style to adopt in the composition stage;
• an AI predictor is then used iteratively to generate the music determining duration, frequency and velocity of a note at a time [5];
• dynamic lighting behaviours are generated deterministically.

The output stage relies on MIDI commands for the music and the open hardware platform Arduino for the control of a custom multi channel light controller. 

A real time interaction console allows to contrrol the behaviour of the installation that was also used as a testbed to analyse the reaction of participants and gather important feedback for further development of this open and participatory work  [6].

As this system is a musical meta-co-creation, evaluation of such a system’s output is itself a challenging research area. Our future will investigate and try to evaluate the methodologies to do so.

The system is capable of co-generating thousands of soundscapes, with possibly varying levels of success, but there is room to improve the system and extend its use in creative performances.

Acknowledgements

Initially designed for the Durham International Brass Festival for celebrating its musical heritage,Which is your brass voice? is the result of an interdisciplinary collaboration between IT Architect Claudio Benghi and Artist Gloria Ronchi.

References

1. Dewey, J., 1934. Art as experience. G.P. Putnam

2. Shearman, J.K.G., 1992. Only Connect: Art and the Spectator in the Italian Renaissance. Princeton University Press.

3. Dourish, P., 2004. Where the action is: the foundations of embodied interaction. MIT Press.

4. Pachet, F., 2003. The Continuator: Musical Interaction with Style. Journal of New Music Research, Vol. 32, pages 333-341.

5. Begleiter R., El-Yaniv R., Yona R., 2004. On Prediction Using Variable Order Markov Models, Journal of Artificial Intelligence Research (JAIR), Vol. 22, pages 385-421.

6. Eco, U., 1989. The open work. Harvard University Press

Copyright

The article on this page is copyright of Aether & Hemera (2013) and cannot be reproduced or reused in any form totally or partially without our written consent. High resolutions photographs can be freely downloaded and  reused from here.