Technical Stuff
The installation will use a kinect to track the location of the user,
the same kinect will be used for arm/hand recognition.
This is an example that Theo Watson made that illustrates this technology:
To track the direction a user is looking, i will be using either a digital compass built into the headphones, or face recognition software.
In the testing stages i’ll use a wizard of oz technique.
Users will perform actions in the room and i’ll be changing the variables of their location and actions on the fly in software. A smartphone attached to the headphones will act as the viewing direction sensor, sending compass data to the computer.
Nice video that illustrates beautifuly, how different sounds can be linked to similar sounding actions & objects.
Frequency “Sounds” Commercial
Moving sound(s)
Moving sounds is something we really wanted to put into the application, we had some really nice ideas like virtual musicians walking around, playing a guitar that you could walk next to on your way through the park.
Unfortunately, moving a sound can be a bit disorienting. Since you have no visual reference that would help with locating it, your imagination has to come up with one.
Apparently, mentaly placing the sounds with a fixed location is difficult enough on the brain, the extra variable of a moving sound just makes the experience confusing.
We decided to use this technique only for smaller loops and only when the movement of a sound is more important than it’s location.
Fortunately this is not as relevant for my installation since i Am using visual references and the movements are of a much smaller scale. This is however one of the first things i will be testing.
Distance and different algorithms for fading
After some small tests, we eventually used a logarithmic formula since that sounded more realistic.
The difference between the two is in the way they link audio volume with the distance from its source.
Multiple sounds
We found that using many sources of information at the same time, such as people talking to you, is impossible to follow unless they are part of one conversation(taking turns).
“Splitting” audio into different parts(one event different sound sources) works best.
We used this effect to create a virtual Pink Floyd concert, where we split the audience and the band into different sound sources.
In this demo you can walk through the crowd and around the playing band.
I made some important choices in the last week, did some tests, made a rough planning, booked a ticket to Greece…
But all this will be in tomorrow’s post.
While you wait, here are a couple of links to a some awesome installations.
Dark Matter by David Rokeby:
http://homepage.mac.com/davidrokeby/Dark_Matter.html
Audio Space by Theo Watson:
http://www.theowatson.com/site_docs/work.php?id=15
Personal Project
Next to the smartphone app i will be creating with Michael and Geert, i’ll also be making an installation.
I will be using the same technology(directional audio) but my space, target audience and premise will be different.
The user will be wearing wireless headphones out of which directional sound will be played as they walk in a white/neutral space with black geometric objects inside it.
Sensors will “follow” the users around the room and observe their position, orientation and interaction with the space and the objects within it.
The idea is that using this system, I can create a virtual world that exists only in sound and the mind of the user wearing the headphones.
First demonstration
This was the first Demo of our group Project on the 25th of february, the results were very promising.
The software (which we created in two weeks) proved to be very robust and our client Eric Velleman from Stichting Accessibility was very impressed.
Building the first demo
We promised our client that we could deliver a working prototype in two weeks.
We did this because, we really believed we could, and we wanted to push ourselves to keep a tempo for the project.
Small sprints with a lot of umph so that we can test our ideas as quickly as possible. http://en.wikipedia.org/wiki/Rapid_application_development
In the first few days we did some initial research and chose to work with iOS because of it’s openAL support.
This meant that we also had to learn the basics of a new programming language, objective-c in these two weeks.
Thanks to some help from the awesome tutorials on http://www.thenewboston.com/ and our previous experience with C++, that proved to be easier then we expected.
Our demo was finished with time to spare and with almost no overtime.
The Start of the Project
We will be using Directional Audio to create an audio app for smartphones that helps people who are visually impaired experience their surroundings.
Directional audio is a technique using stereo headphones to simulate the placement of sound in a spatial(3d) surrounding. http://en.wikipedia.org/wiki/3D_audio_effect
There are different ways to record directional audio: http://en.wikipedia.org/wiki/Binaural_recording
Faking the effect is a bit more complex.
The software would have to do most of the calculations of the effect of the physical space to create the same effect.
Thankfully there are a few ways, mostly used in game development to produce the same effect from pre-existing “standard” mono or stereo recordings.
