faceCloth – sur(ta)face
Multi-touch implementation of faceCloth.
Collaboration with prisonerjohn.
Cast: smallfly, prisonerjohn
faceCloth – sur(ta)face
Multi-touch implementation of faceCloth.
Collaboration with prisonerjohn.
Cast: smallfly, prisonerjohn
Some sketches from the first day of Ben Fry’s Processing class here at Anderson Ranch. What was striking to me is that you really can cover the essence of setup and coding syntax in a day, even for people not familiar with programming/Java. You’re instantly translating code into visuals, so there’s immediate feedback — not a big slog through how the environment works before something actually happens.
Since I have spent some time with Processing, I took the opportunity to try to push some of the examples in a different direction. Speed is not necessarily my forte (with anything, really), but it was fun to try to throw together a sketch as quickly as possible. I didn’t even worry about checking for errors; I actually decided that if the compiler was regularly throwing errors because I accidentally left out a parenthesis, that meant I was moving fast enough. Even though we’re starting at the beginning, though, I’m rapidly filling up holes in my knowledge about Processing and picking up endless tips — it’s really extraordinary to get the chance to work directly with Ben Fry himself! And not only is it an opportunity to get close to the source, but, as I expected, he’s a fantastic teacher, as well.
I’m also trying a new way of working, which is to regularly keep a visual log of what I’m doing. Plaqs’s Skitch, a hot new Mac app currently in beta, takes care of that nicely. It allows me to quickly take screen grabs and post them either to my Skitch page or (as here) Flickr, so I have a record of various iterations — some successful, some less so. More on that and how TextMate makes life easier with Processing coding on Mac soon.
Stay tuned for more…
© Peter Kirn for Create Digital Motion, 2007. |
Permalink |
5 comments
Add to del.icio.us
Want more on these topics ? Browse the archive of posts filed under News.
Creating audio reactive software with processing is not the problem but making a video from it can become a painfull work. Everything works great in real time but when you want to save frames as it takes some time for each to save, the sound is quickly out of sync with the image.
After some headhache I came across a wonderfull topic in the Processing forum where Dave Bollinger gives one of his great pieces of knowledge. The method is simple: just read the spectrum step by step instead of real time analysis and everything will work fine.
I’ve done some video with this trick, one on Murcof music and another on a friend composition:
There’s only you – Processing Visuals.
Murcof Processing Visuals.
The two sketches use almost the same code.
I don’t know how I came across this one but it really was a cool experience. I gave a little workshop on Processing in the Art school La Cambre, here in Brussels.
A friend was talking about future workshops with Pierre Huyghebaert (teacher of Art numeriques) and they found that they hadn’t anybody to talk about Processing. And that was the moment I came in! (or my friend talked about me).
And so I gave my very first workshop. I was kind of afraid but Pierre is a very nice and interesting guy and everything was cool and easy. I prepared some exercises for the students from very basic programmation to some physics stuffs. But the workshop was much too short so it was just a little initiation for people who never heard about Processing. But it seemed the students (and Pierre too) liked my talk. Hope they all take some more time too look at this wonderfull tool.
Technique: A bike handlebar mounted mounted webcam is processed by a laptop, creating a time lapse where the exposures overlap (no movement is lost). The audio comes from pitch-shifting recordings from a large solenoid (modified bass speaker) which were capturing most low-frequency vibrations. More here: http://rpi.edu/~mcdonk/AmbientLapse/ I went to and returned from Radioshack, around 23 minutes away, and composited the return path in reverse using overlay mode. Going and returning are panned slightly left and right, respectively.
I added some mass to the bass speaker, effectively introducing a low-pass filter. I wanted to use the bike like a record needle, and play the street and sidewalks.
Cast: Kyle McDonald
Recently I’ve been keeping things very simple – eight frequency bands driving all my graphics. My previous experiments used 64 bands, split into four groups of sixteen with Ess kindly working out the average for these groups. My past methods were based completely on event triggering – when a spectrum band peaks above its group’s average by a certain amount something happens.
I’ll definitely work a feature like this back into the mix. If tamed correctly it finds drum hits (particularly the snare) and highlights big dynamic changes really well.
For now, I’ve stripped things back to find more complex ways of interpreting these spectrum values. The plan is to have these always changing spectrum values controlling movement and speed. Turns out it’s quite tricky – even with audio dampening, getting smooth results will take a bit more work. Until then I’ve been thinking up ways to visually highlight structural change beyond a keyboard or time based trigger.
Here is St. Anthony’s Fire (the song I will be using, right click to download) split into eight frequency bands and averaged every 60 frames (two seconds). Not exactly a sexy piece of data visualisation I know but useful enough.

(right click to see a larger version)
The song is very heavy and a total mess of frequencies but windowed averaging has made the structural changes quite clear to the trained eye. I’ve circled the points in which I could set thresholds or even better link a windowed average to a particular variable to get a nice gradual transition rather than an instant shift. Hopefully I can give these six sections a completely different look/structure and provide smooth transitions to create a clip that’s always flowing and moving. Generative animation’s beauty is its capacity to provide output, shapes and movements beyond the programmer’s initial intention – visuals that are somewhat ‘alive’. I’d like to turn the visuals into a character of sorts, a complex and wild beast that twists, turns and changes with the music. Avoiding hard cuts and instant transitions will help the visuals to live and breathe.
A tree representation of a collection of music. The root is the library which branches out into artists then albums then songs.
Cast: traer

I’ll be out next week at the Anderson Ranch Arts Center in Aspen, Colorado, where I’m taking a workshop led by Processing co-creator Ben Fry.
In this workshop we will focus on Processing’s video library, learning how to play with pixels from a live camera by analyzing their makeup, altering colors, and experimenting with effects. We will also explore video as an input to control graphic elements on-screen, whether by controlling their movement or generating dynamic, abstract compositions.
I’ve done work with video and Processing before, but naturally not with Ben Fry himself nearby, so I’m quite excited! I’ll be interested to see how he teaches it, since video performance is not a strong suit of Processing (largely the fault of Java, not Processing), though it is an interesting tool for using video for other tasks.
I’ll try to “liveblog” some of it next week, but if that doesn’t work, I promise some code examples of my own and other tidbits when I return. CDM, code, events, generative, java, learning, processing, processing.org, video
© Peter Kirn for Create Digital Motion, 2007. |
Permalink |
One comment
Add to del.icio.us
Want more on these topics ? Browse the archive of posts filed under News.
I saw Stéphane Cousot on my way to the atelier this morning and he suggested I use abstract classes instead of inheritance for my experiments. This too is a concept I know pretty well from other languages, but haven’t really used in Java yet. As it turns out, it’s pretty easy.
Abstract classes are classes that cannot be instantiated. I.e. you can’t turn an abstract class into an object directly. It’s sorta-kinda like a half-finished prefab house: parts of it are already built, but you still have to lay down your own foundation. In my example from yeasterday, the Animation class could have easily been an abstract class, since it was never transformed directly into an object: there was no Animation myAnimation = new Animation();, only Animation myAnimation = new Opening();, or Animation myAnimation = new Middle();.
To actually write an abstract class you just add the word abstract in the class declaration, as well as in the methods that you know will need to be added by the class that inherits it:
abstract class Animation {
// this is a provided method that will work without creating
// a new method inside of the subclass. it is not abstract:
// as you can see it actually contains working code
void setup() {
background(255);
}
// this is absolutely necessary and is incomplete by design
// meaning that the object that inherits this abstract class
// will absolutely positively have to create its own draw()
// method, or else the java compiler will yell at you
abstract void draw();
}
If you try to create a concrete object out of your abstract class (Animation), the compiler will yell at you: Semantic Error: Attempt to instantiate an abstract class "Temporary9837080$Animation”. But you can very well tell it to create a new object out of a class that inherits from your abstract class:
void keyPressed() {
switch(key) {
case 'a':
sketch = new SketchA();
break;
case 'b':
sketch = new SketchB();
break;
}
}
Technique: A windshield wiper mounted webcam is processed by a laptop, creating a time lapse where the exposures overlap (no movement is lost). The audio comes from pitch-shifting recordings from a large solenoid (modified bass speaker) which were capturing most low-frequency vibrations (the higher-pitched sounds are music playing in the car). More here: http://rpi.edu/~mcdonk/AmbientLapse/
Highlights: circles around star park at :34, motorcyclist trying to show off at 1:21, passing at 1:49.
Notes: The original length was about 26 minutes. There is a cut near the beginning due to some corrupted video, but it’s hard to see. My sister Kiera McDonald drove, chose the path and music.
Cast: Kyle McDonald
I’ll be working with Wolf Ka this week on a new choreography he’s preparing; it’s got a lot of promise, I’m quite happy with what it’s turning into. Anyway, I mention this because he asked me a question that I didn’t really have an answer to: how do you jump from sketch to sketch during a performance when running Processing, i.e. how to build a program-switcher if you will. This is essential for performances where you really want to have controllable chapters or sets or call-them-what-you-will to keep the performance moving.
When he first asked me about it, I made a quick look on the Processing forum but couldn’t find anything to sink my teeth into (let me know if I missed anything). So today we finally took the time to actually investigate the matter, and it turned out to be a pretty easy solve: polymorphism. I knew this method of programming from other languages, but not in Java which I’m only now really getting used to. And so far, at least syntax-wise, I find Java to (thankfully) be clear even enough to show students how it works, which we did this afternoon to my utter delight — and gosh golly, they seemed to get it.
If you don’t know what Polymorphism is, Daniel Shiffman has a great link to a very short page that perfectly explains the concept to anyone who knows a bit already about object-oriented programming: How My Dog Learned Polymorphism.
But if you don’t want to read even that, here’s my take: Polymorphism allows you to make code that calls an object without knowing exactly which object will be called, at least not when you write the code. It allows you to solve that problem later, while the actual code is running. In the case of Wolf’s project, the advantage is pretty clear: he needs the main draw() method to constantly call the draw() method of one of his objects, only he wants the object he calls to change from time to time, depending on the needs of the performance. Early in the performance he wants to tell object no.1 to draw itself, then later object no.2, then object no.3, and you get the point. All at 60 frames per second, which is why you need the program to do it all by itself.
Now, you could do it the ugly way, like this:
int which_sketch = 0;
void draw() {
switch(which_sketch) {
case 0:
doDatThang();
break;
case 1:
doDatUdderThang();
break;
case 2:
goFreakOut();
break;
default:
letItAllHangOut();
break;
}
}
So that’s the wrong way. Why is it wrong? Well, it’s okay actually to do it that way, but it’s not good for switching from one program to the next. If you do it this way, you really need to control the flow of your entire code which makes it very sloppy if you want to change anything (which you will). The other problem is that the global variables of doDatThang() are probably going to get mixed up with the global variables of doDatUdderThang(), which is bad. Why don’t you just let the computer do all the work for you, and encapsulate all your variables for one part, or another part, of your program? Then let the computer do all the initialization of your variables for you, and finally let it get rid of those variables whenever you’re ready to move on to the next part of your program. I.e. polymorphism.
There are two ways to do this cleanly in object-oriented programming: through what is called « inheritance », or through what is called « interfacing ». The former is easier so I’ll only talk about it here, but the latter is just as cool so check it out if you’re looking to solve a similar problem. But to understand the later, you need to understand the former, so start at the beginning.
I don’t have the patience to explain everything here, but the idea is basically that you create a non-descript Object that knows two basic methods: void setup() and void draw() (I could have called it startup() and goDoIt(), but I like Processing’s terminology, so I’ll stick with that). Then, once you’ve created that vanilla-flavored Object, you tell all your little sub-programs to all inherit the functions of your vanilla-flavored object. The code looks a little like this:
class Animation {
void setup() {
}
void draw() {
}
}
class Opening extends Animation {
void setup() {
background(255);
}
void draw() {
// do stuff here
}
}
class Middle extends Animation {
void draw() {
background(random(255));
}
}
class Ending extends Animation {
void setup() {
background(0); // take inspiration from the Sopranos
}
void draw() {
// draw stuff here
}
}
We now have three classes, all of them extensions of the class Animation. And all of them either inherit the method draw() or directly override the version of Animation with their own. Which means that we can call any of these classes and be sure that something will be drawn, all without error.
But the cool thing about Java is that we can create objects out of the Beginning, the Middle, or the Ending, but program them all as if they’re just another type of Animation, letting the program actually decide what type of Animation it will be at run-time.
Animation myAnimation;
void setup() {
myAnimation = new Opening();
}
void draw() {
myAnimation.draw();
}
void mousePressed() {
int random_choice = (int)random(3);
switch(random_choice) {
case 0:
myAnimation = new Opening();
break;
case 1:
myAnimation = new Middle();
break;
case 2:
myAnimation = new Ending();
break;
}
}
The cool thing there is that you just have to tell the program that you’ll be using one of three Objects, all of them containing at least one setup() and one draw(), only you’ll let the computer decide while the program is running which object will actually be the one to run.
The Java trick is to actually declare myAnimation as an Object of the class Animation, whereas in reality it is an Object of the class Beginning, Middle, or Ending. This is the polymorphism part. You are telling it that you don’t know which object you are going to be running at any one time in your program, but that at least all three of the different parts will adhere to the two methods available in Animation, namely setup() and draw().
The final nice thing about this trick is that the new Middle(); or new Ending(); takes care of creating and destroying all the variables for you. No need to get rid of all the variables you created with the first object, and no need to worry about global variables stepping on each other.
Well, re-reading all that, I realize it’s a bit dry. Also, be careful, it’s pseudo-code, as I don’t have Processing in front of my from this terminal (my computer just died). You’ll just have to trust me that I’ll make a full-scale class out of it later on. At least the main details are there, and I’m always willing to accept walks-in to explain it to you on-site, only then you have’ll have to come to lovely Aix-en-Provence and I’m just about to leave for my summer vacation
Crap. My MacBook just died. I’ll have to take it in tomorrow. Luckily it’s got a 3-year warranty on it. Great machine, don’t get me wrong, quite brilliant. But hard-disks really are a daily gamble, especially considering how much of our lives we put on it. Too fragile. We need to move to solid-state. Anyway it sure sounds like the hard-disk died — yeah, go ahead and laugh, sure I played the Dr. Dougie and put my ear up to the damn thing. It sounds like its the mechanism itself, and not even a bad sector which could be easily fixed — but we’ll see what the magician says tomorrow. I have far less fairy dust (and spare time) than those fellows who are by the way paid to figure it out anyway…
So, long story short, anyone who just sent me email, know that my computer died right after it grabbed about 100 emails off my server. So unfortunately, I have know way of knowing what you just said, nor that you even said it. Sorry, but that’s part of the equation: missives in general must structurally be designed to occasionally lose themselves, otherwise no message would ever find its way…







