If you haven’t seen it yet, Jamie and Adam did what may be the greatest illustration of a computing concept onstage ever, using an 1100-barrel paintball gun:

The basic idea: GPUs, by using parallel processing, are able to render graphics more effectively than CPUs. And while the illustration is something of an oversimplification, it is pretty literal in terms of showing people what’s going on — and why GPUs are uniquely well-suited to computing graphics. It’s really one of the most brilliant demos I’ve ever seen.

There are just a couple of problems — and, amusingly, this demo makes them visible, as well.

We should recognize that the demo took place at NVIDIA’s NVISION show, so there was a certain bias expected.

But first, there’s the problem of standards. With graphics, we’ve already seen what happens with competing APIs like DirectX and OpenGL (and other wrinkles, like the Mac-only Core Image, which is in turn built on OpenGL). Now, NVIDIA and other GPU makers want to sell us on the idea of using their chips for general computing tasks (aka GPGPU). There still isn’t a standard for doing that. NVIDIA is pushing its CUDA 2.0 technology, of course, but it’s not clear whether others will adopt it. I’d put more faith in OpenCL; aside from the backing of Apple and AMD, that spec looks like it has a good shot at being adopted by the Khronos Group, the folks who brought you OpenGL’s standard implementation. (Not that everything Khronos has touched has turned to gold — see the largely failed OpenML.) But the bottom line is, to put it in the terms of this demo, it’d be a lot nicer if you could take the same instructions from one paintball gun to another.

Then there’s the bigger problem: what works for graphics doesn’t work for everything else.

In fact, you can see in the demo why, in general terms, parallel processing doesn’t work as well for audio tasks. Interestingly, this week as I chatted about a new release of a dedicated DSP-chip platform called the UAD-2 on Create Digital Music, people brought up this question of whether GPU chips might be the future of audio processing. You can see the fundamental problem here: audio, as a real-time operation that occurs in extremely tiny slices of time (think 44,100+ samples each second), tends to want to be processed more like the first paintball gun. That’s not to say that some tasks, particularly those that require parallel operations like granular synthesis, wouldn’t benefit from some parallel processing. But even then, the round-trip to the GPU has to compete with simply staying on the CPU for the task. These aren’t necessarily insurmountable problems, but suffice to say CPU makers are working on this, too. GPU makers are removing architectural barriers between the CPU just as CPU makers are working on smarter parallel processing, so the placards in this demo are likely to say “serial versus parallel”, not “CPU versus GPU” in the near future. (What do you want? It was an NVIDIA-sponsored event.)

That said, readers of Create Digital Motion have quite a lot to look forward to in the increasing power of GPUs. Obviously, 3D graphics capabilities continue to improve, and despite what you may hear about the PC game market dying, both the niche PC game and 3D production industries seem interested in continuing to push the envelope here.

But that’s just the beginning. Video decoding at higher resolutions and qualities is getting more efficient, which is not only good for video but leaves more room for other tasks. Decoding on the GPU (or a dedicated chip) is becoming increasingly powerful, and a number of GPGPU implementations do decoding on the GPU. (If we get really lucky, we’ll see a convergence of an open-source codec with an open-source GPGPU implementation in a standard spec like OpenCL, finally making video accessible to all. A boy can dream.)

I think the most interesting application may actually be in the area of computer vision. Doing analysis of video to determine motion has always been costly and difficult on the CPU. But processing a bunch of pixels in parallel is ideally suited to GPGPU implementations. Again, if we could get an open-source implementation, the work built on that could be incredible. Intel’s open-source OpenCV library is the basis for countless computer vision projects, including many in academia that would be unable to license a proprietary library. OpenCV runs on the CPU (written in C/C++); imagine something similar running on a standard GPGPU implementation.

If anyone was at NVISION, I’d love to hear about any developments on those fronts.

Gizmodo has a similar combination of enthusiasm with a small dose of technical skepticism (but plenty of gratitude to these guys for making the demo so clear):
MythBusters Build a 1100-Barrel Paintball Gun to Paint the Mona Lisa, Instantly

They also did a great explanation of what the parallel processing business is about, particularly in regards to Apple’s upcoming Snow Leopard OS:
Giz Explains: Mac OS 10.6 Snow Leopard Parallel Processing and GPU Computing

So, there’s my semi-uninformed opinion, at least. Usually what happens is, when we touch a subject like this, people who know more than me come out of the woodwork to share what they know. If you’re there, I’d love to hear from you.

I believe that coding is an essential skill for people making live digital visuals. At the same time, there’s no question that learning to code has been a big obstacle for visually creative people — especially as they have plenty of other things on their mind. You need somewhere to start, and you need to make the learning curve manageable. Processing has been a great tool for doing that, but the point isn’t to learn Processing — it’s to learn how to code.

For that reason, I’m thrilled that Dan Shiffman’s book Learning Processing is now in print and available. I’ve been waiting for this for some time. There are already a couple of great books out there for Processing, but Dan’s book is unique in that it’s entirely focused on teaching you to code visuals step by step, even if you haven’t coded before. Dan teaches coding to creative-minded non-coders at NYU’s ITP program, and the book comes out of that teaching technique.

I know the book very well as I served as a technical editor during its development. We’ll be running an exclusive set of excerpts this week, but here’s a look at what’s included:

• Basics of code structure, pixels, interaction, and fundamentals like how the coordinate system works
• Using arrays to make lots of stuff appear on the screen, including particle systems
• Basics of images, video, data, and networking
• How to use object-oriented programming to make coding easier and more efficient
• Extending Processing with Java and more advanced coding techniques

By the time you’re done, you’ll be processing pixels, drawing generative visuals, and writing well-organized code.

To me, one of the real strengths of this book for teaching and learning is its strong emphasis on object-oriented programming, in a way people can actually understand. Explained properly, objects can really help keep your code clean. For some reason, this is often viewed as an “advanced” concept, but on the contrary, I’ve found using objects actually helps keep beginners from getting tripped up. And, to put it in visual terms, understanding objects is a terrific skill for getting lots of glitzy eye candy up on the screen.

Here’s what Dan has to say about the book (emphasis mine):

My goal for “Learning Processing” was to write something for the complete and total programming beginner. If you’ve never written a line of code before in your life, but want to get started creating your own digital media tools then I wrote this book for you. There are several other wonderful Processing books out there and I hope mine will complement them nicely. A special thanks to Casey, Ben, and Ira who kept encouraging and inspiring me as their books were being published.

The book is also geared towards the teacher. It’s not my belief that such a person will necessarily learn any new skills from the book (assuming they have a programming background), however, my hope is that the book will encourage and help facilitate the teaching of programming. It is structured with 10 lessons (complete with examples and exercises) and can act as a ready-made syllabus for a beginner interactive media / programming class. In fact, the book is modeled exactly on ITP’s Introduction to Computational Media course.

The book is available on Amazon.com. It shows out of stock, but those kind of screw-ups are common when a book has just come out (speaking as a published author here).

Stay tuned for those excerpts later this week; I’m editing them now!

See also: Dan Shiffman’s blog, Facebook Page, Official Site, free download of TOC and first chapter

This illustrates why people are jumping for Processing.js, even if it cripples many of the things that make Processing cool. Paul Downey inadvertently hits the nail on the head:

… somehow hadn’t got around to actually doing anything with it. You see it’s the whole Java thing that puts me off; when it comes to playtime life’s far too short to wrangle a CLASSPATH or compile an applet.

Ah-hah — there’s the reason: developer laziness, and fear of Java. And rightfully so. Configuring a full-blown IDE for Java can in fact be some effort — I think it’s well worth it when you’re doing lots of work over time, but what if you just want to sketch?

Here’s the good news: Processing’s “founding fathers” Ben Fry and Casy Reas agree with you.

The thing is, these JavaScript developers I think haven’t bothered actually trying Processing — the real Processing in Java. The creators of Processing understood that traditional Java development could be a pain. So the whole point of the Processing IDE that you get when you download — a simplified text editor that understands the Processing language — is saving you exactly that trouble. You almost never, ever have to deal with “wrangling a classpath.” It just doesn’t happen, certainly not with the included libraries (which do a lot more than Processing.js can). In fact, there’s even a download for Windows that takes care of installing Java for you. Nor do you have to worry about the effort involved in “compiling an applet.” Again, Processing does the work for you.

In case you’re interested, what Paul did is reasonably cool — he set up Processing.js inside TiddlyWiki, the personal notebook tool.

But, Paul, in the time it took you to do that, you could have been off and running in the Processing editor. Give it a shot, really. As I said, I think the hack for JavaScript is very much in the spirit of Processing as an open platform. But if you don’t experience it in Java, you’re missing out on a lot of what it can do.

© Peter Kirn for Create Digital Motion, 2008. |
Permalink |
3 comments

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under News.

Continuing our current vvvv love-in. Aforementioned generative AV project “Va” will be performaing at the Node08 festival in Frankfurt, which started on the 5th and continues until the 12th. Lots of exciting workshops and lectures happening, and concluding with “vvvvinisage”, featuring visualists from around the world.

© Jaymis for Create Digital Motion, 2008. |
Permalink |
No comment

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under Asides.

I used to be resistant to the idea of coding. It wasn’t just fear that I couldn’t do it, though that was part of it; it was also the sense that I wouldn’t be able to get to the actual art and music making if I got too involved in programming. And, actually, that bit can be true. But a group of pioneers, working on projects like Processing, OpenFrameworks, and other intelligent development frameworks, has been working really hard to make code an elegant an expressive tool rather than a hindrance. Processing has reached widespread popularity because it does this really, really well — even if you’ve never programmed before.

I’ll be teaching a three-part class on Processing at Harvestworks in New York next month. If you’re in the area, there should still be openings if you’d like to sign up (and if you’re enrolled, feel free to holler hi here — if I hear from you in advance, I can help tailor the course to your needs).

For intermediate digital artists, even those who have never coded before, we will introduce techniques in Processing. Processing is an elegant, high-level, Java-based tool designed to make coding friendly to artists. We will learn how to create generative art in just a few lines of code, building interactive works in minutes. We’ll also look at some of the deeper possibilities for manipulating data, video, images, sound, and MIDI and other I/O. The emphasis will be on basic sketches that help introduce fundamental coding skills.

Wednesdays, March 5, 12 and 19, 6:30 – 9:30pm
$325/$385

Class page / signup @ Harvestworks

The class will specifically focus on how to make video, 3D visuals, MIDI, and sound work for performance. Making Processing a performance tool definitely involves some particular skills. But I’ll also use this as an opportunity to teach very basic coding techniques so that unfamiliar programming topics can immediately generate something on the screen or some sound, since that’s part of the appeal of the whole tool.

But what if you’re not in New York?

We’ll soon have CDM Labs up, which will include examples from the team at CDM, plus other stuff from around the Web, not only in Processing but related tools, as well. I’ll use this as a playground for the course, so what I share with them, I can share with you. And, honestly, we hope this will help discipline us here to keep coding and keep documenting. More on that soon.

I’m also hoping to refine this course into something that can be offered elsewhere; if you’re interested, get in touch.

More on Processing:

Random sketchbook of mine, the kind of stuff you can put together in minutes

Flickr Processing pool

Processing videos on Vimeo

Processing tag on Create Digital Motion

Official Processing exhibition page

© Peter Kirn for Create Digital Motion, 2008. |
Permalink |
No comment

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under News.

Some sketches from the first day of Ben Fry’s Processing class here at Anderson Ranch. What was striking to me is that you really can cover the essence of setup and coding syntax in a day, even for people not familiar with programming/Java. You’re instantly translating code into visuals, so there’s immediate feedback — not a big slog through how the environment works before something actually happens.

Since I have spent some time with Processing, I took the opportunity to try to push some of the examples in a different direction. Speed is not necessarily my forte (with anything, really), but it was fun to try to throw together a sketch as quickly as possible. I didn’t even worry about checking for errors; I actually decided that if the compiler was regularly throwing errors because I accidentally left out a parenthesis, that meant I was moving fast enough. Even though we’re starting at the beginning, though, I’m rapidly filling up holes in my knowledge about Processing and picking up endless tips — it’s really extraordinary to get the chance to work directly with Ben Fry himself! And not only is it an opportunity to get close to the source, but, as I expected, he’s a fantastic teacher, as well.

I’m also trying a new way of working, which is to regularly keep a visual log of what I’m doing. Plaqs’s Skitch, a hot new Mac app currently in beta, takes care of that nicely. It allows me to quickly take screen grabs and post them either to my Skitch page or (as here) Flickr, so I have a record of various iterations — some successful, some less so. More on that and how TextMate makes life easier with Processing coding on Mac soon.

Stay tuned for more…

(…)
Read the rest of Processing Workshop Day 1: You, Too, Can Learn to Code (0 words)

© Peter Kirn for Create Digital Motion, 2007. |
Permalink |
5 comments

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under News.

I’ll be out next week at the Anderson Ranch Arts Center in Aspen, Colorado, where I’m taking a workshop led by Processing co-creator Ben Fry.

In this workshop we will focus on Processing’s video library, learning how to play with pixels from a live camera by analyzing their makeup, altering colors, and experimenting with effects. We will also explore video as an input to control graphic elements on-screen, whether by controlling their movement or generating dynamic, abstract compositions.

I’ve done work with video and Processing before, but naturally not with Ben Fry himself nearby, so I’m quite excited! I’ll be interested to see how he teaches it, since video performance is not a strong suit of Processing (largely the fault of Java, not Processing), though it is an interesting tool for using video for other tasks.

I’ll try to “liveblog” some of it next week, but if that doesn’t work, I promise some code examples of my own and other tidbits when I return. CDM, coding, events, generative, java, learning, processing, processing.org, video

© Peter Kirn for Create Digital Motion, 2007. |
Permalink |
One comment

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under News.

Shaders, snippets of code for processing pixels and 3D points on your 3D card’s GPU, are cool — that much you may know. You may even know that you can use shaders — designed for 3D applications — to perform powerful video-processing tricks, as well, at high speeds, even on a relatively lowly laptop. How to actually build your own — that may be elusive. So, at long last, Cycling ‘74 has published a great, beginner-friendly (even for non-programmers) tutorial on building your own shaders:

Your First Shader

The author is Andrew Benson, who is my hero as far as coming up with great Jitter examples. (Every time I’m looking for some model for a technique I have in mind, I keep stumbling on his sample patches in the Jitter folder and still more in his By the second page, you’re already building your own custom, glitchy visual filters. Great stuff.

Now, of course, this tutorial isn’t limited to users of Max/MSP/Jitter, but Max is really an ideal environment for testing shaders. (It can even work well as a prototype environment before going elsewhere.) This code will work in Processing, though, and I could see using a combination of those two tools (still working on my own workflow there).

We’ll be practicing our shader chops, because there’s definitely a need for more information like this. This sentence says it all: “If you want to learn more, I highly recommend poking around the “jitter-shaders” folder and grabbing the official GLSL specification(PDF) or the GLSL Orange Book.” Good advice, and the example Jitter shaders included with the program will already do a lot of what you’d like need. But, as fair warning, the Orange Book and other official OpenGL documentation can make your head hurt, fast. (There’s a reason “… for Dummies” isn’t on the end of the title.) I still recommend picking up a copy, but there’s definitely a need for at least intermediate documentation. Being something of a dummy myself, I may be able to help. GLSL, GPU, Jitter, learning, Max/MSP, opengl, processing.org, programming, shaders, software, tutorials, video

© Peter Kirn for Create Digital Motion, 2007. |
Permalink |
10 comments

Add to del.icio.us

Want more on these topics ? Browse the archive of posts filed under News.