Synesthesia

Ever since seeing Fantasia as a boy, I’ve been fascinated by animated renderings of music. Poking around online today I found two very different animations of J.S. Bach’s Brandenburg Concerto #6. Both are complete mappings of the musical score onto a scrolling visual display, and so both express the same information. I can’t decide, though, which one I think does a better job of communicating visually the flow and interplay of the composition’s several voices.

The comparison is not ideal, because the first clip animates the concerto’s first movement, and the second one animates the third. Also, the first clip features a far better performance than the second, which is obviously the output of a computer playing a programmed version of the score.

Both animations use vertical position to indicate pitch, and scroll horizontally through the score so that the notes currently being played are always in the center of the screen. Both renderings also color-code the individual musical voices. And there the similarities end.

The first animation, which is graphically far more sophisticated, uses colored circles to represent each note, strung along lines that mark off the half-dozen or so separate voices. As each note is struck, it appears as a circle whose initial size indicates the duration of the note. Each circle shrinks at the same rate, so that it has just shrunk to zero by the time the next note is struck. The circles bubble up on the right side of the screen, have their moment in the sun, then rush off to the left — solid before they are struck, and hollow afterwards. The visual effect is lively and effervescent, and in particular the vertically stacked circles that mark off the quarter-note pulse throughout most of the piece have a beautifully propulsive effect. It’s a marvelous piece of work.

The second animation is far simpler, but clearer too: it’s a simple graphic representation of exactly the same sort that one might see on a MIDI display of the score. The notes are horizontal bars, with vertical position representing pitch, and horizontal extension representing duration. As each note plays, it is is highlighted, then goes dark again. That’s it! There’s none of the fancy eye-candy we had in the first clip: the expansion and contraction of the horizontal scale as the circles approach the center, the changing sizes of the circles to represent the lifespan of each note, the passage from solid to hollow circles as each note is played — but the eye is much better able to follow the representation of each note, and to connect the visual input with the music. As much as I admired the creative ingenuity of the first animation, I found the second to be more effective in bringing eye and ear together in a unitary experience.

Have a look. Which do you like better?

(Also: was Bach a genius, or what?)

Oh, and lest we forget: here’s another way of doing this sort of thing.

Related content from Sphere

10 Comments

  1. MikeZ says

    They’re both quite good.

    – M

    Posted May 14, 2012 at 10:36 pm | Permalink
  2. Kevin Kim says

    “Also, the first clip features a far better performance than the second, which is obviously the output of a computer playing a programmed version of the score.”

    Uh-oh. If you look at the ending credits for the second vid (i.e., 3rd movement), you see:

    Elias Goldstein, viola

    Elizabeth Choi, viola

    Anna Steinhoff, violoncello

    Advent Chamber Orchestra

    Maybe they’re robots…?

    Posted May 14, 2012 at 10:36 pm | Permalink
  3. Malcolm says

    Ha! Well, there you go. The dynamics of that second performance sounded so stiff to me that I didn’t even bother to look.

    That’s funny, really – as music sampling and sequencing got better and better (and I’ve been recording that stuff in the studio since the beginning) I was sometimes fooled into thinking that a well-programmed piece was a human performance when it wasn’t. This is the first time I’m aware of that I’ve made the opposite mistake — thinking a live ensemble was a computer.

    How embarrassing. My sincere apologies to the players. (But next take, try to let it breathe a little, guys!)

    Posted May 14, 2012 at 10:43 pm | Permalink
  4. Malcolm says

    Anyway, which animation did you like?

    Posted May 14, 2012 at 10:44 pm | Permalink
  5. Kevin Kim says

    I’m familiar with the “musanim” and “smalin” animations; my first experience was with a YouTube vid “showing” Debussy’s Clair de lune back around 2006. I agree with you that the second vid’s animation is easier to follow; it certainly feels more straightforward. But the first vid’s animation is more lively, albeit harder to follow. Aesthetically speaking, I like the first animation better.

    As for being fooled: sound quality on YouTube may have thrown you off. I don’t have the best ear for sound or music, but even for me, the sound on YouTube always seems a bit flat — almost tending toward mono instead of stereo.

    Posted May 14, 2012 at 11:42 pm | Permalink
  6. Malcolm says

    No, it wasn’t the sound quality; it was the performance itself, in particular the opening few bars. It’s just very plodding in exactly the way that sequenced music can be. It didn’t even occur to me that it was anything other than a MIDI file — and I guess once that impression was set in my mind, that was that. If anything biased me, I guess it was that the animation looks very much like the display a digital workstation such as Pro Tools would show as it plays the file — a very familiar sight, but one that always means the computer’s doing the work.

    An embarrassing mistake for a professional recording engineer to make, but there it is!

    Posted May 14, 2012 at 11:59 pm | Permalink
  7. Churchgoer says

    A genius and an intellectual. And a wage earner to boot, like so many intellectuals of today. Hope somebody was keeping an eye on him!

    Posted May 15, 2012 at 7:15 am | Permalink
  8. Dom says

    It’s the same orchestra on both vids, and I bet the animators are the same, since the descriptions on the two vids are identical. I like the first better, but I like Disney’s animations better than both. I’ve been thinking about Fantasia now — remember those Hippos and Crocs?

    Posted May 15, 2012 at 7:56 am | Permalink
  9. Eric says

    Bach was a genius indeed. It reminds me of the story behind the Musical Offering: When Frederick the Great invited Bach to the king’s residence in Potsdam, Frederick presented Bach with a long and complex musical theme, and invited Bach to improvise a 3-part fugue on the theme. Bach took this long theme, constructed to be difficult to counterpoint, and wrote the 3-part fugue on the spot.

    Frederick then challenged Bach to write a *6* part fugue on the same theme. Bach accepted the challenge, and two months later presented the result, now known as the Musical Offering.

    Posted May 15, 2012 at 11:43 am | Permalink
  10. David Brightly says

    Malcolm, if you liked Fantasia I think you would like the Italian pastiche, Allegro Non Troppo, even more.

    Posted May 16, 2012 at 5:07 am | Permalink