Soundscapes – The Future of Music

In order to glimpse what the future of music might hold we need only look at Thief 3

Have we perfected sound yet? No, not by a long shot. We imagine that graphics will be “done” when we can create anything we can imagine without hitting a hardware limit. Whether it’s a perfect recreation of reality or whatever our imagination can conjure. Considering the intangibility of sound, it might be hard to imagine how you can make it “better”. Graphics and sound compete for computing resources. At the moment we expect a level graphics from games that doesn’t allow any major advances in sound and music. The next generation of consoles will make developments possible as there will be additional resources to work with. There are still great advances to be made, and, thanks to audio demos, we can get a glimpse of what the future might hold. With the next generation of consoles a year or two away this might be a preview of what you’ll get to hear from them. This is part two of a three part series. In part one we looked at the future of sound, this time we will look at the future of music.

Using music in new ways

Music in games today generally consists of two or three tracks that fade into each other as appropriate based on action. This works well for most situations today, but if the developers wanted to get more ambitious with the music design it would result in compromises in graphics. With an unlimited amount of resources available to developers, games will be able to feature hundreds of tracks in the background if they wanted to. The point of this would be to, for example, have an environment where the music would subtly change depending on where you are on the map in order to make the music work better with the ambient sound. So a city level could have all sorts of sound sources from road works, clubs, markets, etc, and the music would change in tempo and style to match all of them.

“Music” is a broad term, so the nature of what this music would sound like could be anything: strings, drums or just mechanical noises that add rhythm and a sense of drive to the game and world. This could be done today, as Thief 3 did something similar to my example back in 2003, by using many unique music ques that played only in specific areas of missions. The clip below is from the first level of the game where this music design is obvious. As the player moves across a trigger a new set of music appropriate for that area plays. Ambient sounds seem to blend in with the music making the line between sound effects and music blur. The tone of the music is that of a horror game and some ques are reminiscent of Aliens soundtrack. Every level has a different musical and sound theme appropriate for the story of the location. One level takes place at an ocean side manor where a storm is about to hit. The sound of lightning blends well with the music which sounds like it’s building pressure in anticipation of the thunder storm. Another level takes place in an abandoned mental asylum where the now undead patients still roam. The music design of this level blends perfectly with the noise of the twitching heads of the patients and ticking clocks that seem to be everywhere. The quality and thought put into the audio design of Thief 3 is on a level rarely seen in games, or anywhere for that matter. Sound and music work in perfect harmony to create an oppressive atmosphere where you’re afraid to disturb anything in the world, no matter how ineffective the AI might turn out to be in practice at stopping the player. The audio by itself becomes a critical gameplay component which motivates player actions.

Borrowing the audio design from Thief 3 and taking it to the next level is something which will inevitably happen. Doing so today, however, might come with an unacceptable cost to the graphics. Every multi platform game made today has very little memory to work with on the consoles. The more music and sound you put into a game directly takes away from the memory budget shared with the graphics and results in lower quality models, smaller levels etc. Thief 3 for example had tiny levels compared to games of its time. At this point a game attempting this would have to either use a simplistic visual design, or be PC exclusive. Combining ideas and technology from music based games such as Rock Band or DJ Hero with other genres such as an open world role playing game wouldn’t be out of the question if the hardware resources can cope with it.

Music of the future would in many ways be similar to what we are used to today as we would still have musical themes to identify the game and the characters in it. Where we will see these more experimental approaches is in the incidental music during moments of exploration and action where the game is at its most unpredictable and open to player expression. It’s this music in between the scripted sequences where there is massive untapped potential. With enough tracks and cues waiting in the background to play when the correct trigger is made by AI or the player the music could become an orchestra directed by the player with the help of the developers and music designers. There are many ways this style of music design could cause artifacts and problems with unpleasant dissonant effects such as too many samples playing at once. But with good enough tools and time to fine tune the effect it could create something which cannot be done in any medium other than games.

This is part two of a three part series. Part one covered the future of sound, part three the future of virtual surround.

Warren Spector hinted at Epic Mickey 2 having ambitious music design. Will it set a new standard?