Introduction

This site describes a sound installation work supported by spatial sensing system with a Personal Area Network (PAN), which may be applicable for such areas as dance performances and mobile music.

The term "Sound Jewelry" was coined by Iwatake, one of the authers of this paper. The original idea is to create interactive musical objects which are worn like necklaces by people. Each of the "Sound Jewelry" then automatically generates "melody" and according to the distance between the persons wearing it the sounds of "melodies" may be changed or exchanged. On the other hand, Yamauchi, the first author, has been researching into real-time localization systems for multiple agents in a PAN. So it was only natural for the 2 ideas come together. .

Figure shows the localization system. There are two "listeners" on the wall and eight "senders"to be held by the participants. The "senders" transmit supersonic waves to the "listeners" and the master server which is connected to the "listeners" measures the locations of the "senders" based on the distance estimation subroutine in the middleware. The results are sent using OSC protocols to the sound generating application, whose outputs are heard from 4 surrounding speakers in the environment.

The actual "Sound Jewelry" turned out to be an environment that consists of two layers of sounds. In the foreground, "melodies" are dynamically generated by measuring the distances between the participants. In the background, ambient sounds are automatically generated using the distance data. When many people move in a 4-by-6 m space, the sound only changes based on the nearest relative distance. However, as the number of people increases, the sound changes become more complex. Sound complexity was used as part of the installation. Users recognized sound changes as they moved in real time in the space.

System Architechture

The system consisted of slave servers and a master server. After the widgets on the slave servers had received the identification and distance information from the sensors, the information was sent to the aggregator. The aggregator on the master server estimated the userfs location in the PAN, as well as calculated the relative distance from the nearest user. Figure shows the distance estimation in the PAN. We assume that there are N node users in the PAN and two listeners on the x-axis located at (t, 0) and (t+|c|, 0). The location of each node can be given as

node1(x,y) = (t + |b1| cos(A1)+b sin(A1))

node2(x,y) = (t + |b2| cos(A1)+b sin(A2))

:

:

nodeN(x,y) = (t + |bN| cos(AN)+b sin(AN))

Summary

This site proposes a real-time location system for multiple agents in a PAN. The system is composed of a location system that estimates the location of the "listener" on the wall in an indoor environment, the positions of users with "senders," and a sound system that feeds back into the speakers. In the case that the user enters a zone of the location system, the user's "sender" transmits supersonic waves to the "listeners" on the wall, and the master server measures the position based on the distance estimation subroutine in the middleware layer. After the master server has calculated the position of the users in the location system zone, the results are sent using OSC protocols to the media application located on the application layer. In order to apply the location system media content, we considered the design and implementation of the location system. This system was demonstrated using a sound installation. "Sound Jewelry" is an environment that consists of two layers of sound. In the foreground layer, "melodies" are dynamically generated by measuring the distances between the participants that are wearing sensors. In the background layer, "accompaniments" are automatically generated using the distance data in order to support the ambient environment. When many people move in a 4-by-6 m space, the sound only changes based on the nearest relative distance. However, as the number of people increases, the sound changes become more complex. Sound complexity was used as part of the installation. Users recognized sound changes as they moved in real time in the space.

Demo

"Daikanyama 2006"

Demo demo

Acknowledgement

This project is supported in part by a grant from CREST, JST. We would like to thank the members of CREST for their helpful suggestions.

Publications

(1) "An Interactive Musical Installation through Spatial Sensing"

2007 4th International Mobile Music Workshop poster

(2) rhizome.org rhizome.org/object.php?46823

(3) Neural.it www.neural.it/art/2007/06/sound_jewlery_networking_with.phtml

Contact

yamauchi(at)sfc.keio.ac.jp, yamauchitakuya(at)yahoo.co.jp

Update: 07/05/01