How can type communicate sound? Can simply abstract forms suggest melodies?

These were some of the questions I came up with while developing this sound type visualizer .

see this experiment (the audio in the beginning is quite low so please turn up the volume of your speakers)

Design

For this piece of interactive motion type I was inspired by the work of the Swiss graphic designer Niklaus Troxler: I was particularly inspired by the poster he designed for the Willem Breuker Kollektief. Here type is dissected, disassembled while retaining though, the fundamental shape of the letter. When I first saw it I suddenly associated this image with the sound of a trumpet and a clarinet, and just when listening to the band I realized how the design was carefully matching the style of the musicians.

After a couple of thoughts and a walk down the alley I sat down on Saturday sketching some prototypes for putting this idea in motion. My first idea was to get the pieces of the letter further away from the center in an ordered fashion, more than an irregular offset as in Troxler’s work.

I wanted to give the letters a broken look more then a dissected one. I sketched out a quick prototype setting each part of a letter with the registration point in the middle. Then using a little bit of trigonometry I managed to give to the letters an implosion/explosion effect.

Here is my very first test with keyboard input.

Using the keys to expand the letter wasn’t really exciting though. Therefore I came up with the idea of linking the animation to a sound output and finally set the words alive dancing with the music.

I realized a first prototype with the sound using the whole alphabet and playing a little bit with transparency: take a look at it here.

I still see some potential in this experiment and I will like to expand it in the future with different type, weights and motion.

Images

Some snapshots taken during development phase: I love those shapes!

Code

Process

The main idea behind this experiment is: repulse an object positioned on a circle from the circle center with a distance related to sound values.
To each object is assigned a proportional value of the whole sound wave. Object on the right part of the circle will have values from the right channel and objects on the left part will have values from the left channel
Actionscript can give two representations of the sound by setting different values to the parameter FFTMode (Fourier transformation) of the computeSpectrum function in the SoundMixer class:

• if set to “true” the method returns a frequency spectrum where low frequencies are represented on the left and high frequencies are on the right.
• if set to “false” the method returns the raw sound wave.

To apply this concept to type I created for each letter a class and using the Flash IDE I sectioned each letter in 17 pieces (average) and transformed each of this pieces into a movieclip with its registration point in the center.

You can see the difference in this example

Classes

I’m going to write here, for the sake of code readability, just the main function for each class and their responsibilities.

• Main.as
• SpectrumAnalyzer.as
• DancingLetter.as

Main
Provides to initiate the sound data model. Once the sound has been loaded the data model is passed to each DancingLetter object on stage.
It rotates and scales the letters’ container.

SpectrumAnalyzer
It is the data model for the visualizer.
Loads and provides sound information. Displays the graph.

Here is the function that provides and displays sound values:

 
private var _values:Array=[];
private var _wave:Boolean=false; 
private var music:Sound = new Sound; 
private var ba:ByteArray = new ByteArray();       
private var sc:SoundChannel;
 
......
 
private function processSound(ev:Event):void { 
	SoundMixer.computeSpectrum(ba, _wave); 
	//creates the graph
	graphics.clear(); 
	graphics.lineStyle(0, 0xFFFFFF); 
	var counterY:int=0;
	var counterX:int=0;
	_values=[];
    for (var i:uint = 0; i < 512; i++) { 
		var lev:Number = ba.readFloat(); 
		if(i==512/2) { //split channels graph visualization
			counterX=0;
			counterY=20;
			graphics.moveTo(counterX,-(lev*10)+counterY); 
	}
	_values.push(lev);
	graphics.lineTo(counterX,-(lev*10)+counterY); 
	   counterX++
	}        
}

DancingLetter
It is the view model for our visualizer.
Stores the initial values for each sprite that contains and animates the sprites according sound values.

public function DancingLetter() {
	//populate a multidimensional array with all the object in its displaylist and their initial position
	for (var i:Number=0; i<numChildren; i++) {
		var part:Object = new Object();
		part.point = new Point (getChildAt(i).x, getChildAt(i).y);
		part.object= getChildAt(i);
		parts.push(part);
	}
}
 
//This is the function that animates the letter parts
 
private function changePosition(e:Event) {
	for each (var dl:Object in parts) {
		//assign sound values proportionally to each parts
		var soundValue:int =_sa.values[Math.round((_sa.values.length)*parts.indexOf(dl)/parts.length)]*20;
		dl.object.scaleX=dl.object.scaleY=Math.round(Math.abs(soundValue)/5)+1;
 
		//get distance from starting point
		var dx:Number=dl.point.x;
		var dy:Number=dl.point.y;
		var distance:Number=Math.sqrt(dx*dx+dy*dy)+soundValue*25;
 
		//get angle 
		var angle:Number=Math.atan2(dl.point.y,dl.point.x);
 
		//move according its orignal angle
		dl.object.x+=((Math.cos(angle)*distance)-dl.object.x)/2;
		dl.object.y+=((Math.sin(angle)*distance)-dl.object.y)/2;
 
		//change colors
		var color:uint;
		ct.color =colors[int(soundValue>1)];
		dl.object.transform.colorTransform=ct;
		dl.object.alpha=1-Math.abs(soundValue)/10;
 
		/*
		dl.object.rotation=soundValue;
		dl.object.scaleX=dl.object.scaleY=dl.object.alpha+1 
		*/
   }                                                      
}