This article describes the inspiration, design, and construction of an abstract animation music video using Blender.
This was done as an assignment for the course Music Technology 2A at UCT.
Below is the animated music video I made:
You can download the FL Studio (FLS) project file to see how the audio was made. FL Studio 20 runs on Apple & Windows and the project can be viewed and played for free (but not saved with the free FLS version).
The Blender animation file is also available below for download. It needs Blender version 2.79 (the latest at the time of writing). Later versions of Blender use a new rendering engine that won't work with this file.
I began my search for ideas by looking for purely abstractly animated music videos.
I knew I didn't want to use illustrations or photographs as I had done that last year for another music video project. Instead this work would be done manually on computer or programmatically in code.
My first inspiration was early abstract animation music videos. The best example of this is Norman McLaren's Dots (1), shown below:
This video has some noteworthy features:
I then looked for 3D abstract animation and found SLICED by dxmiq (2).
This looks pretty and impressive but I imagine is quite simple to make. I would simply draw a long line of shapes and then automate the camera to move down the line, with a clipping plane "slicing" the shapes as the camera passes along them.
An example of Hollywood level commercial production is the fantastic work by Milk VFX on the film Annihilation, created in Houdini (3).
This is obviously the ultimate goal for a musical animation, but is possible only with a paid experienced VFX team, and weeks of paid rendering on a server farm.
This video also is the basis for my musical inspiration. It is synthesized, complex, and vague, but has clear melodic elements and rhythm.
I had made an application for TEDx Cape Town a few years ago that pulled tweets from Twitter that were tagged with the conference hashtag and displayed them on a giant projector on the wall. I thought about something similar for a music video and searched to see if anything like this had been done before. It has, by Brightly for their song Preflight nerves (4):
The video above is static, but the one on their website pulls fresh tweets to match the song lyrics in realtime each time you play it.
I then discovered the most impressive video yet. Dennis is a procedurally generated 3D animated, gesture interactive, beat synchronised music video (5).
The animation was programmed in three.js.
It is very smooth and quick to render as it uses modern web browsers' support for the graphics processing unit (GPU) by using WebGL. Finally, the animation was synchronised to the beat of the song by using WebAudio's native analysis capability.
This video shows the power you have when you don't have to animate every shape manually, but can write code to create and manipulate the shapes and camera for you.
After getting some idea of what could be created I began to look for free software tools to work in.
For 2D vector animation the best tool I could find was Synfig Studio (6).
Synfig allows you to draw complex vector shapes directly into the application or import external images. Each object's properties (e.g. size, rotation, outline, color, etc.) can be independently altered at any point (keyframe) in an animation of any length. Synfig will then automically fill in (tween) the animation between each keyframe. It also supports advanced functions like masking layers, blend modes, duplication and time manipuation, and skeleton animation.
Synfig would be the perfect tool to make something simple like Dots and I was able to work through all the tutorials in a day.
For 2D programmatic animation the best framework I found was two.js (7).
two.js would be best for creating a video like Dots but with much more complexity and movement, or for creating something with text, like Preflight nerves.
Blender is the undisputed champion of free 3D modelling and animation software. It has been around for years and its features rival industry standards like Maya.
Recently Blender has also added powerful capabilities for beautiful 2D illustrated animations through its grease pencil tool (which used to be a simple way for animators to annotate their work quickly for later use) (8).
Plain Blender would be the best tool for making a video like SLICED.
Blender with Grease Pencil would be the best tool for making something like Dots, but with 3D effects.
The problem is that Blender takes at least a week of learning and practising to become proficient at the basics, and I didn't have the time or skill for that.
The best framework I could find for 3D programmatic animation was three.js (9). It was used for the Dennis video above and supports almost anything you could imagine.
Move your mouse through the birds in this demo to scatter them.
There is a competitor made by Microsoft, called Babylon.js, which is more suited to game development as it simplifies commonly used functions like lighting, but three.js is well established.
I decided to write a little story about an acoustic bass meeting a modern synthesiser and having a conversation/song. The Annihilation movie inspired me to finish the story by having the bass being transfigured into a more evolved creature by meeting the synthesiser.
I wrote the audio in FLS as it is an excellent choice for making electronic music fast.
I particularly enjoy the ability to put snippets of MIDI and automation into a library of clips which can then be cloned, reused, and rearranged easily. As far as I know no other DAW other than Tracktion's Waveform and possibly Studio One can do this.
It also has a fully scalable vector graphic interface, and the hotkeys enable me to switch between the playlist and each MIDI clip incredibly quickly.
Below is an image of the track. You can zoom in to see the labels of each section.
The disadvantage of FLS compared to the other DAW I use, Reaper, is that it has poor routing. To overcome this, and route one MIDI clip to multiple instruments to create the complex sounds for each character in the animation, I used Patcher for almost every instrument track.
Finally, Effector is plugin that provides useful special effects, like bitcrushng, vocal formants, and flanging.
I wrote the music at 120 bpm so it was easy to synchronise with the video (where each second is simply two beats).
For the sounds themselves I used the Sytrus and Harmor synths with preset sounds to save as much time as possible. I used only synthesizers, no samples, as I find it easier to work with MIDI.
I spend a few days writing the video in three.js but quickly encountered difficulties. It's hard to programmatically specify what you want to happen on screen if you can't see it and are completely new to 3D graphics. Finding errors and fixing them is difficult. And documentation is lacking.
So I decided to switch to Blender. This took a week to learn from reading a couple of tutorials. But the good thing was all my questions could be answered within an hour by asking on the forum, so I never got stuck on a problem.
Putting two shapes on the screen and using keyframes to animate them was easy. The time consuming task was synchronising the video and audio. I also had to learn some intermediate techniques:
Finally, I didn't have time to do interesting textures or particle effects, which would have greatly improved the impressiveness of the evolved bass in the second half of the video. I tried these, but couldn't get them to work with my limited skills.
Blender outputs each frame as a png rather than exporting video, so if your computer restarts you don't lose hours of rendering.
To convert the .pngs and my mp3 file to a video I used the command:
ffmpeg -framerate 60 -i temp%4d.png -i a.mp3 -start_number 0000 -vcodec libx264 -crf 16 -r 60 output.mp4
Overall this was a useful experience and I learnt a great deal about animation and the available options.
The audio part took only a few hours and suits the idea I was trying to convey quite well. The video is suited to the audio too, but is clearly the work of an absolute beginner in modelling and animation. I think using simple abstract shapes allows the audience member's mind to freely interpret the experience.
However, I don't think this is much of a contribution to art overall. While the idea is fun, it is not original, and the graphical execution would need a lot more work to make it of a professional standard.
All Internet resources were accessed on 25 September 2018.
