Ed Del Castillo and The Next Evolution in Fight Games

Edward Del Castillo likes to play things close to the chest. When his PR rep read my query about implementing reactive audio in fighting games Castillo’s first thought was “did this guy get a leak?” Without revealing the title, the man behind the Command & Conquer series spoke with me on the sound design processes that went into creating what he hopes will be the next evolution in the fighting genre – reactive audio.

Such change, of course, couples with looming clouds of doubt in a genre resistant to change since Yu Suzuki boldly took fighters into the three dimensional realm. Fight games have long since struggled to define emotion through gameplay in an interesting way, but Castillo’s audio system just might change that by handing the emotional keys of the soundtrack to the player.

Well aware of the challenges he and his team face, Castillo explains in this interview why his reactive audio system will make gamers care about each punch and why you absolutely shouldn’t call it a rhythm fighting game.  

Nightmare Mode: So tell me a little about this secret project you’re working on.

Ed Del Castillo:  It’s a one-on-one combat fighting game where you move through spaces fighting opponents. And in those scenarios the hero has music that supports him, the enemy has music that supports them, and in addition to that we have figured out an entire vocabulary of musical instruments, and um, musical – well, yeah, I guess musical rhythms and musical instruments – and kind of short verse the music in the form of links or riffs, whatever you wanna call ‘em.

And we’ve made it so that – focusing on that from the very beginning – we made it so that those pieces, those component parts –  if you want, think of it as Legos – all kind of go together. And so there’s this kind of very long flat green piece, y’know that baseboard Lego that sits underneath it all that’s always playing, but then we have Lego bits of different lengths, but they’re all uniformly fitting into the integers that form the Lego bits.

And what happens is a very dynamic – I mean – if you were to close your eyes while someone is playing and just listen, you would hear a unique piece of music that would never ever ever be heard again in your, or anybody else’s, lifetime. The interesting thing there is that we’ve also figured out how to, in addition to that, do sound effects in a way that many of the sound effects also fall into this as well.

So, you end up with this awesome scenario where, in my opinion, a game for the first time is appreciable on multiple separate categories. Whereas when we usually play these games we talk about how great the music is, or we talk about how great the design was, or we talk about…but like a movie soundtrack, y’know, we kind of acknowledge them separately, and we don’t acknowledge them as each one of them contributing to the other, and this is absolutely the case with this game.

I mean, the music is being built by you and by us together in unison, and that result is not only unique every single time you play, but it’s something that flavors…Just like you can no longer hear the soundtrack and give it accolades on its own, and you also can’t do that for the fighting.

NM: So would you categorize this as a music fighting game, or is it more of an evolution of the fighting genre?

Ed: Umm, yeah, I’m gonna say it’s an evolution because… [rhythm fighting game] implies there is a rhythm like Dance Dance Revolution or Parappa the Rapper, and hitting on the rhythm makes you better or worse…because what it does is it immediately puts – it tells people – “well, I’m just following the music,” and that’s not the case; you’re actually collaboratively making the music together with the existing music track. And so, I see it as an evolution, I see it – or maybe a better way of saying it is – it’s an exploration.

Y’know, a little while ago, a couple of years ago, we had this social and mobile boom, and I don’t think it’s gonna go bust, I think it’s here to stay. But, y’know, I looked at that and said games have always been social – we had leaderboards in Space Invaders, we just didn’t call them leaderboards, we called them high-score boards. And I see this entire social phenomenon of Facebook and everything else as a very natural exploration of alternative avenues to express our art. And I’m kind of happy that it happened, because to be quite honest, in the last ten years we’ve just been pounding on the better graphics drum. And, I hate to say it, y’know, we need a change.

It got to a point where all the AAA console titles were just starting to very much look the same to me. And I know that the pundits will tell me “how can you compare Gears of War with Assassin’s Creed with God of War?” But I’m like, well, think about what you’re fundamentally doing in all of those – it isn’t very different. What new things am I learning out of those? And I think what you’ll see is the biggest draw of those [games] is graphics – the biggest draw is unique eye candy that I can’t get anywhere else.

And I think that when we had the social boom and we had the mobile boom it did two things: One, it encouraged us to go back into our libraries and pull out great game mechanics that had been forgotten because of high budgets, but it also allowed us to explore another side of our humanity. Y’know, as I’ve said a thousand times here in the building, humans are social and mobile, why are we surprised that our technology has become the same?

And I think that this is yet another exploration in that music affects our emotions, music affects our chemistry, music affects our biorhythms, and the sounds also do the same thing. So why are we, y’know, why shouldn’t we explore this deep impact on our psyche, and how that can be married and created dynamically into a game? Y’know, and how that can be brought out in this kind of game.

And it’s funny because, for us, a fight game is the perfect place for that, or a dedicated music game like “DJ Mix-a-Lot” or something. But a fight game is a perfect place for that because a fight game’s tempo lends itself to music’s tempo. And so I’m just surprised no one’s done it earlier to be honest.

NM: Would you say that this type of reactive audio system builds a narrative through the emotional context of music? Instead of just like a storyline, does the music help push the narrative forward in a more ludic fashion?

Ed: Okay, well, so a couple of things I need to lay the groundwork for: One of things the game doesn’t do is a lot of talking, it actually lets the experience speak for itself. And what ends up happening is we have a lot of very kind of, y’know, Kurosawa-like scenes with very, y’know, stoic poses and things like that.

And what happens when you do that, is what sound is present becomes even more important.  And, what I feel is I’ve always been emotionally attached to music – I listen to some music and it makes me cry, I listen to some other music and it makes me laugh, or it makes me feel very patriotic.

And what happens is – what I find is – it feeds your character’s narrative in the fact of “who is my character?” Like, the sounds for blocking are very different from the sounds for attacking. And so, depending on the mix that you’re using, you form a soundtrack for your character that, in my opinion, starts to influence your thinking.

It’s like, well I’m attacking a lot so that gives me these kind of high tempo drum hits in some cases, or, I’m kicking a lot, y’know, that gives me some very sharp bass tones and things like that. So you end up with a completive fight.

If your style is similar, you end up with this reoccurring – although never the same because your timing is never exactly the same as it was in the previous fight – you end up with this kind of album that you’re forming. Y’know, where the music might vary from fight to fight or track to track, but the “album” has reoccurring themes, or the level has reoccurring themes, because you’re fighting the same way. And conversely, if you take a completely different approach to the next opponent that you took to the previous opponent, well, your soundtrack changes.

I think what it does is – if you’re a big music guy, if you’re a big sound guy – it backfills your feelings around this character. I don’t know this will ever tell you a story, because we have some very cliché and western thoughts about how you tell a story as a console industry, but I think it absolutely alters the tempo of your heart, and alters the emotions you might or might not be feeling. And if shown, for example, that the deeper, the harder, the beat, the more it has a likelihood of changing the tempo of your heart; your heart actually tries to match the beats that are around it.

So if you’re doing some kind of fast-paced, constant attacking, well, you’re going to shoot some adrenaline into you. You’re actually gonna biochemically alter your body to be more amped. And if you’re, y’know, choosing a very passive style, you’re going to have a slower body tambour – you’re going to have a slower bio rhythm. And I think that’s a interesting thing that people aren’t fully exploring yet, but we are, and that’s a key place to be with this game.

NM: Did you use specific instruments and sounds pertaining to individual characters? Or is it a just one cohesive sound all around the board?

Ed: No, I think it’s, again, thinking of – I don’t want to belabor that Lego metaphor so let me try a different one: So, you have music, and music can be presented in many different parcels. So you have, for example, “I’m listening to an entire opera,” or “I’m listening to an entire album.” Then you have “I’m listening to a track in that album.” Then you have, well, “I’m listening to a particular piece of a track” – y’know, whether it’s the drum solo or the bridge or whatever. And then, one step further, “I’m listening to the individual instruments in that riff, or in that segment,” and then “I’m listening to the individual notes of an instrument.”

And so, we really kind of parsed all those out and considered all those separately. And what we do is…okay, let me just take an example, I’ll pick something outrageous: So, the background music is country music, right? And, not that it is in the game, I’m just picking something noticeably different; so, y’know, it’s country music, some line dancing tempo in the back, well, the instruments we choose for the player and for the enemies are all country. We’re gonna have steel guitar, and we’re gonna have banjo, and we’re gonna have harmonica, and we’re gonna have, y’know, the things that you would expect to hear in that style of music.

And then what happens is the individual inputs that you are inputting – the blocks, the attacks, the movements – all are key to hitting on rhythms that are congruous with the background music and with instruments that make sense with that background music.

Now, we take a little bit of liberty in that we might increase the falloff of some of the notes, we might increase the pitch of some of the notes and things like that. But you get to a point – and this is the really cool part – it’s a fighting game and, I mean, who hasn’t had that dream of, like, that whole movie dream of “I close my eyes and I’m a ninja,” y’know? “I close my eyes and I can hear where the enemy is…” Well, that’s one of the really cool things about this – you get really good at our game and you can actually play it with your eyes closed. And that’s really cool.

Like, you can close your eyes and you can just listen to the music and the sound, and you can actually play an entire fight game with your eyes closed. And for us that was the proof in the pudding. That was the “wow, we’re on to something really cool here,” y’know? Maybe one percent of the people who play it will get that good, but there are already people in this building who are that good already. And that’s really cool for us.

NM: How much of the music would you say was composed on the granular level? Like, individual notes that come together or are triggered by certain actions?

Ed: A lot of it. I mean, I have to say that was the focus. I mean, we’re using a famous composer for some of the cut scenes and we certainly enlisted him in some of the underlying tracks of the music, but, absolutely, a fundamental understanding of how beats and how tempo works, and creating instruments that could cohesively work together and create a tapestry of usable inputs was the core of this thing.

So, y’know, we – think of it this way – we didn’t start with “it’s a piece of music and now we’re gonna break it apart,” we started with “what happens if every time I punch it’s a drum?” “What happens if every time I kick it’s a flute, kind of?” And “what does that sound like when I start fighting?” Okay, “what does it sound like if blocking is a harp?” “What does it sound like if moving is a guitar riff?” Y’know, and how does that all…we started from there – we started from the very basic block, which is a single note, and created other congruent instruments, but also figuring out what the right fall-offs were, what the right introduction of the note is, what the right decay of the note was, to make it feel like they weren’t stepping over each other. To make it feel like they weren’t…like they felt like they were supposed to be together.

And there’s so much good hardware out there…along with graphics, all these consoles have also improved their sound hardware, and thus far that only gets used for, like, “okay, one of these tracks is gonna be music, okay, one of these tracks is gonna be hero sound effects, okay, one of these tracks is gonna be enemy sound effects,” y’know? And “one of these tracks is gonna be environmental sound effects.” And so, we’re using those tracks as – as in traditional console gaming – independent autonomous stimuli that’s all happening in a cacophony together. And what we did is we said, “Okay, what if each one of those tracks is a line in the stanza?” “What if each one of those tracks was, y’know, a much more granular approach to sound making?” And we blend all these together.

NM: How do you think this approach affects the fluidity of the gameplay? What I mean by that is does the system influence the player to live up the expectations of the music cues or are the music cues more player influenced?  

Ed: Yeah, it’s a give and take. I would say that at a micro-second level there is some fudging that we do. I mean, at a micro second level there is a, y’know, music does have a tempo, and even if the tempo is “babababababababababa” there’s still a tempo to that. And so if a player is hitting even faster than the tempo for example that I just exemplified we might fudge it into that tempo.

And what we found is if you look at combat games across the world, especially fight games, but not just fight games but action games as well, there is a certain maximum speed at which an animation driven move ceases to look good. And across the board most fighting games avoid that. Y’know, most fighting games avoid allowing you to go beyond the animation’s fluid limits, because they understand that the break-up of the animation makes the game fall apart, because if you literally have to pop from one thing to another it starts tearing apart the fabric of the fight game.

So what you end up having is, y’know, even if I’m button mashing – let’s say in a Street Fighter game – even if I’m mashing that button faster than the animation can draw, the animation is going to draw at its rate, and it’s gonna drop the inputs that are out of sync with the possibility with its range, right? So let’s imagine that the fastest attack from Guile in Street Fighter II is a fifth of a second, so you can do five of these in a second. If someone’s hitting that button faster than five times a second, you’ll still only get five punches within that second. You’ll never go faster.

And this is a very typical anomaly – not anomaly – but this is a very typical characteristic of all action games and fight games. There are parameterized constraints. All we’ve decided to do is [make] the parameters of our constraints fall on traditional music beat lines. So, what happens is we have a sixteenth note, a very fast play note – y’know, “dadadadada” –  if you can hit buttons that fast we can shoot attacks out that fast, but if you go faster, that’s as fast as you’re gonna get to go.

But most people will never be able to punch that fast or kick that fast, much less be able to go beyond that, so we haven’t really found a problem with it only because we’re thinking of it at that micro level. If we were thinking about it like a PaRappa The Rapper or a Dance Dance Revolution where it’s like, “okay, hit the punch, punch, block, kick, punch, punch, block, kick” well, that’s a rhythm fighting game and that’s not we’re not doing. We’re saying how fast is the fastest music? Let’s allow that to be our input windows. And we found that was more than fast enough.

NM: When you recorded the sound, did you mic the actually thud of a punch and mix it to be more musical or did you use drum kits to map over the punches and kicks, or use music instruments in place of true sound effects?

Ed: The answer is yes to all of them. So yeah, in some cases the musical version was the best and in some cases – in many cases – we found that having some kind of punch sound mixed in with music was better just so a player could be grounded in the reality of the game, y’know, because if it was just music what you end up with is “am I hitting or am I not?”

Y’know, unfortunately, because it’s an exploration we’re bridging the gap between what people expect and the music, so we couldn’t just take the leap to completely the new thing. And we found that it just worked better when we mixed in a sound effect with the musical stuff too. But in some cases the music was enough, or the instrument or the note was enough, and so we went with that.

NM: What possibilities do you see coming from this type of reactive audio system as it evolves throughout the fighting genre?

Ed: Well, let me be pragmatic for a moment. I think to start; it has a lot to do with adoption. I mean, if people really come to value that then others will do more of it. If the Japanese fight game companies like Capcom and all these other guys look at that and go, “eh, it’s a gimmick,” well then, y’know, on our own I don’t know that we will change the world of fight games.

But I do think that what this does is it gives a whole new flavor to fighting and I would hope that people will explore how music can intensify the fight feeling. Because really at the end of the day that’s the issue, right? The issue of all games, and the frontier – the permeable membrane – that we are all pushing up against is how do we get people to feel as much for our games as people do when they are part of a movie or when they are part of a more emotional experience in their life? 

And so we’re all exploring the art of how we get people to care, how we get people to feel while playing our games. And, y’know, fight games have traditionally just relied on adrenaline to get you to feel something. And I think it would be an interesting thing to have a sad character, and that character’s music is always sad no matter how they play. Or have an exciting character, or a happy go lucky, happy puppy character, y’know? I think the background-fill of the emotion for a character is where music’s power is, and where music can pull narrative to a new level. Cause it’s one thing to hear a story, or see a guy do something, it’s another thing to feel it.