MIT/CSAIL researchers add realistic sounds to silent videos, a step toward automating sound effects for movies?
MIT researchers have developed a computer system that independently adds realistic sounds to silent videos. Although the technology is nascent, it’s a step toward automating sound effects for movies.
“From the gentle blowing of the wind to the buzzing of laptops, at any given moment there are so many ambient sounds that aren’t related to what we’re actually looking at,” says MITPhD student Andrew Owens . “What would be really exciting is to somehow simulate sound that is less directly associated to the visuals.”
The notion of artificial sound generation has been around for sometime now, with concepts such as procedural audio, and in many ways its long overdue that the same amount of attention and computing power that is afforded to visual effects, be directed towards sound generation. CSAIL is directed by Tim Berners-Lee and is the largest research laboratory at MIT and one of the world’s most important centres of information technology research. I have found several articles which discuss this new development and have selected sections of them here :
“Researchers envision future versions of similar algorithms being used to automatically produce sound effects for movies and TV shows, as well as to help robots better understand objects’ properties.
“When you run your finger across a wine glass, the sound it makes reflects how much liquid is in it,” says CSAIL PhD student Andrew Owens, who was lead author on an upcoming paper describing the work. “An algorithm that simulates such sounds can reveal key information about objects’ shapes and material types, as well as the force and motion of their interactions with the world.”
The team used techniques from the field of “deep learning,” which involves teaching computers to sift through huge amounts of data to find patterns on their own. Deep learning approaches are especially useful because they free computer scientists from having to hand-design algorithms and supervise their progress.
The paper’s co-authors include recent PhD graduate Phillip Isola and MIT professors Edward Adelson, Bill Freeman, Josh McDermott, and Antonio Torralba. The paper will be presented later this month at the annual conference on Computer Vision and Pattern Recognition (CVPR) in Las Vegas.
In a series of videos of drumsticks striking things — including sidewalks, grass and metal surfaces — the computer learned to pair a fitting sound effect, such as the sound of a drumstick hitting a piece of wood or of rustling leaves.
The findings are an example of the power of deep learning, a type of artificial intelligence whose application is trendy in tech circles. With deep learning, a computer system learns to recognize patterns in huge piles of data and applies what it learns in useful ways.
In this case, the researchers at MIT’s Computer Science and Artificial Intelligence Lab recorded about 1,000 videos of a drumstick scraping and hitting real-world objects. These videos were fed to the computer system, which learns what sounds are associated with various actions and surfaces. The sound of the drumstick hitting a piece of wood is different than when it disrupts a pile of leaves.
Once the computer system had all these examples, the researchers gave it silent videos of the same drumstick hitting other surfaces, and they instructed the computer system to pair an appropriate sound with the video.
To do this, the computer selects a pitch and loudness that fits what it sees in the video, and it finds an appropriate sound clip in its database to play with the video.
To demonstrate their accomplishment, the researcher then played half-second video clips for test subjects, who struggled to tell apart whether the clips included an authentic sound or one that a computer system had added artificially.
But the technology is not perfect, as MIT PhD candidate Andrew Owens, the lead author on the research, acknowledged. When the team tried longer video clips, the computer system would sometimes misfire and play a sound when the drumstick was not striking anything. Test subjects immediately knew the audio was not real.
And the researchers were able to get the computer to produce fitting sounds only when they used videos with a drumstick. Creating a computer that automatically provides the best sound effect for any video — the kind of development that could disrupt the sound-effects industry — remains out of reach for now.
Although the technology world has seen significant strides of late in artificial intelligence, there are still big differences in how humans and machines learn. Owens wants to push computer systems to learn more similarly to the way an infant learns about the world, by physically poking and prodding its environment. He sees potential for other researchers to use sound recordings and interactions with materials such as sidewalk cement as a step toward machines’ better understanding our physical world.
The Computer Science and Artificial Intelligence Laboratory – known as CSAIL – is the largest research laboratory at MIT and one of the world’s most important centers of information technology research.
CSAIL and its members have played a key role in the computer revolution. The Lab’s researchers have been key movers in developments like time-sharing, massively parallel computers, public key encryption, the mass commercialization of robots, and much of the technology underlying the ARPANet, Internet and the World Wide Web.
CSAIL members (former and current) have launched more than 100 companies, including 3Com, Lotus Development Corporation, RSA Data Security, Akamai, iRobot, Meraki, ITA Software, and Vertica. The Lab is home to the World Wide Web Consortium (W3C), directed by Tim Berners-Lee, inventor of the Web and a CSAIL member.
Hi! My name is Barney, I’m currently an Audio Production Student at the LSFM, and I’ve spent the final year of my degree getting started in sound design and music for games.
I’ve learned quite a lot about the process and the industry along the way, so I’ve decided to write this post as a guide for those people that are looking to getting into game audio but have no idea where to start. This guide is specific to the Audio Production course at the LSFM, but I’m sure a lot of what I talk about will be helpful for anyone interested in getting into game audio. I will also clarify to begin with, that I’m not writing this claiming to be any kind of expert – I’m merely pulling together some pointers from the stuff I’ve learned myself along the way.
I actually got into game audio by accident, in many ways – I came to Lincoln to study Audio Production because I started making music in my teens, and studying it at university seemed like a good option. I unfortunately became quite disillusioned with music production and spent a lot of time worrying about what I was going to do after I left university, unsure if I would be able find to sustainable work in making music. I stumbled into sound design at the end of my second year, fell in love with it, and didn’t really look back. I spent a lot of time doing re-designs of films and games, as these were visually exciting and meant I could explore some interesting sound aesthetics. Long story short, I managed to talk my way into my first gig making sounds for an indie game, after showing them some of the work I’d done in my spare time – I’ll talk more about how I got my first work in games later in this post. I convinced myself that sound for games was the way forward for me, and began my third year with the intention of spending as much time as possible advancing my knowledge of game audio. Since then it has really snowballed and my time is now packed full of multiple exciting projects, which I hope will continue after I leave Lincoln.
I hope that what I’m going to share is helpful for both those that have a small interest in game audio, and those that would like to pursue it as a career. I will be writing more to address the latter, but hopefully you’ll get something from what I have to say no matter what interest level you are at. There are two equally important sides to working in game audio, and I’ll structure this guide to reflect that: half of this post will be about actually getting into the games industry, finding work, building professional relationships and a network of friends and acquaintances who will eventually be key to helping you to find work. The other half will be about the specifics of game audio, how to approach creating sound design and music for a game, the actual process of recording and creating sounds, arranging them in middleware and implementing into a game engine.
Understanding the Industry
One of the most important things that I spent a long time learning at the start, was how the games industry actually works and how I could fit into that. At the moment, the game development scene is divided into two main areas: indie games, and AAA games. This probably isn’t news to most people who have some interest in games, but for me, it was really key to understand. The huge AAA games have often ruled and stand as the ‘go-to’ example of what a ‘game’ is, but currently, indie games are really booming. The best thing about indies is – by definition – they are made by anyone, and as such, they are hugely accessible. They are can often be very different, very personal and extremely fun games, made by individuals or small teams, who are working out of passion more than any pursuit of fame or money. The indie scene is also not limited by location – it’s everywhere. Even in Lincoln. I have been lucky enough to work on two indie games with developers who live in Lincoln, and the experience has been fantastic. Being able to sit down and meet in person discuss ideas and actually create the game has been really great – it has taught me so much about collaborating and working in the indie scene.
In that respect, I initially disregarded games when I first began experimenting with sound design because I thought that game development was really limited to the big studios – and that the kind of games created by students probably weren’t very good. I was very wrong, and after having spent time with student developers, I can confidently say that they have huge creative possibility.
So this is where I’d recommend to start looking for work, initially – get in contact with some students that are making games. There’s a game development course at the university, as well as some computer science masters – these are the kind of people that will be making games that you can approach. There are also regular game jams at the university, which are really important for building relationships and learning – I’ll talk more about those later.
Obviously there are two sides to pursuing a career in game audio – you could follow the AAA route and use indie games as a way to fill your showreel with interesting content, or you could make indie games and stay with indie games. I don’t think either is a bad option, I really think it is down to the individual. I will say that, this guide is written from the point of view of looking to gain indie game experience whilst still at uni, and to cultivate a network and skillset favoured by indie development – that doesn’t have to mean you couldn’t pursue AAA jobs after that.
This one is absolutely huge, and it can seem really daunting when starting out. The games industry – and the game audio part of the games industry – is very small. It’s also very personal. This means that a cold-calling approach will almost never work. Obviously having impressive work to your name is important, but nothing will be better for finding indie game gigs that having an extensive network of friends and acquaintances who can both vouch for your skills and enthusiasm, and demonstrate you’ve engaged with the industry enough to make some meaningful connections. It’s really helpful to try and make those connections in person, too: thanks to the internet, we can now chat to pretty much anyone at any time. This is great, and I’ve loved the fact I can talk to people around the world about my work and their work. But those connections online won’t ever replace actually meeting people face-to-face, and being able to ask questions and learn from those who’ve done what you want to do. Below I will go into detail about how you can build you network and make connections with people that will be beneficial to you.
People realised, a long time ago, that making connections in person is really important. Because of that, meetups have emerged, and they now provide an opportunity for you to meet people with similar interests and aspirations to you – essentially a less formal networking event. There are two main game audio meetups in the UK, one in the north (Game Audio North) and one in London (primarily organised by Graham Gatheral). I’ve been a regular attendee of the northern game audio meetup since September of my third year, as soon as I knew I wanted to pursue games. I’ve been to about 5/6 of their events now, and it’s now become more a case of catching up with friends to talk about what we’re working on, than traditional networking. The first time I went along to Game Audio North I was really nervous, as I’d never really been to a networking event before – or met people in the flesh that were working in what I wanted to work in. My nerves were pointless, and even at the first event I met people who gave me tons of help and advice about my work, and have continued to encourage me and invest in what I’m doing. The meetups grow really organically too – and the northern meetup has become bigger and bigger, attracting more people from different areas of the north each time. I’d hugely recommend getting along to one of these meetups – it’s been massively helpful for me.
It wasn’t really until my third year that I realized the potential that lies in speaking to the guest lecturers that come in during the course. During my second year, before I had any interest in sound design, we’d had a guest lecture from a AAA game sound designer who’d worked on a number of really cool titles – games I’d played and loved. At the time I’d enjoyed his lecture, but hadn’t really though much of it then as it wasn’t an interest of mine. Once I decided that I wanted to get into game audio, he became my first port of call. Despite the fact I emailed him months after he’d given the lecture, he was still really willing to speak to me and share some knowledge. He looked over some of the work I had at the time, and game me lots of very useful criticisms on it that helped me improve my technique and approach. We’ve stayed in touch since then, and he continues to help me out with my work and my journey into the industry. My point here is just to exercise every potential connection you have – be persistent, be polite and don’t be afraid to ask for things – even if it’s just advice. We have a fantastic selection of lecturers come in, especially during the third year – I think not making the most of speaking to those people is a bit of a waste of time.
Which leads me onto my next point – email. Emailing people is actually one of the biggest tools at your disposal for building relationships and finding work. Previously I wasn’t really that invested in email, but I’ve really become a bit of an email warrior during the time I’ve been trying to get into the games industry. You can email pretty much anyone – and generally they’ll give you a reply. As I said before, it’s really key to be polite and confident when contact people out of the blue – often people play the desperation card when approaching people they want to work with/for, and while this is something we can all understand (it’s a really tough industry to get into), I’d suggest it’s better to just be friendly and to the point. Be professional, and you’ll find emailing people can often open up a lot of doors.
During the last year I’ve absolutely rinsed social media, for better and for worse. I do feel bad for my non-game audio friends, as my Facebook has become a stream of game audio related posts and I can imagine that might become annoying! That being said, social media has been massively powerful for me to build relationships and build some kind of a reputation in the game audio scene. I’d really recommend getting on Twitter, and getting really active about it. Use hashtags, particularly the #gameaudio hashtag, to reach out to people who might be interested in what you’re doing. Pretty much anytime I’m working on something or out recording I’ll tweet about it, and I’ve built it up to the point now where I’ll get a good number of people interacting with it. This is good, because information can spread really quickly on Twitter – you begin to build a reputation, and if a potential employer has already heard of you or your work, it’s a great sign.
Once you feel you’re at the point of being confident in your craft (which I’ll talk more about in the next section), you’ll want to begin looking for work. I really think that a ‘fake it til you make it’ attitude is good here – that old cliche of jumping in at the deep end is really relevant. I’m not saying go into it knowing nothing – have a base knowledge of what you’re doing before you starting talking to developers – but I think that you’ll find you learn infinitely more when you jump into a project and figure it out as you go along. Getting work is one of the toughest aspects to the whole thing – some people are really good at finding work, other people struggle with it. I’m quite lucky in the fact I haven’t struggled finding work thus far – it’s not been easy, and it’s taken some persistence – but even in a relatively small city I’ve managed to work on a good number of projects.
As I’ve said already, it’s a case of building relationships. I spoke before about building relationships with game audio people – this is great for helping to build a reputation, but to actually find work, you want to build relationships with game developers. This is the best way to find work – make friends with people, then they’ll come to you to work on their projects because they know you do sound and they know you’re enthusiastic about what you do.
The two best places to meet game developers is at game jams, and at indie meetups. Like game audio meetups, game developers also meet up to chat about their work and network. This is a great opportunity for you to go along, meet some people, and potentially find games to work on. Don’t go with just the intention of finding work – people can tell a mile away when you’re only motivation to talk to them is for work. This might sound like a weird paradox – but there’s a line and you’ve got to find it. As I’ve said before, desperation doesn’t work – I’ve gone to meetups previously feeling like I had to find a game to work on or I’d have nothing for my portfolio – and my desperation for work reflected negatively into the way I handled myself. Since finding work, I can now treat meetups as more of a social event – and I find that people are even more interested in working with me now that I’m not actively pursuing it. Just be friendly and enthusiastic, and people will like you. There’s a meetup in Lincoln (Lincoln Indies) that happens from time to time, as well as the East Midlands Indies meetup that happens every month or so.
The second best place to meet developers is at game jams; game jams are a bunch of game developers meeting together for a period of time (usually 24 or 48 hours) to make a game, from scratch. This is a lot of fun, and often some really interesting games emerge from the creative process. It’s also great as an audio person too, as you learn a lot in a very short amount of time. Unfortunately audio can sometimes be overlooked at game jams – as the majority of the time is spent actually building the game. However, it can be a great opportunity to sit and learn some software you’ve been meaning to look at, and involve that in the game you’re making – one of the first jams I went to I used the time to learn FMOD, and we used it in the game we built.
The story of the first game gig I got is a bit different. There’s a weird paradigm in games – particularly AAA games – that you are expected to have worked on a game already in order to work on a game. This obviously is a difficult barrier to jump – and I found a way to navigate it in my own way.
As I mentioned at the start, in the early days I only had a few sound re-designs to my name – and this made up the majority of my portfolio. I knew I wanted to work on a game, but I was also aware I didn’t have any experience at that point in work on sound for games. I was aware of a ex-student developer, who was a few years ahead of me, who was known around uni for creating games. I sent him a message, completely out of the blue, initially asking about the audio implementation process he used on his games. He was happy to talk to me, and also mentioned that he was looking for someone to finish up the sound design for one of his games. He asked that I send over some of my previous work, and I scraped together some of the sound re-designs I’d done for him to see. I emailed it over, and didn’t hear anything from him for a while. It was during summer, and I wasn’t doing a lot else at the time, so it felt like forever and I was starting to worry that what I’d sent wasn’t good enough. It was clips from re-designs that were nothing like the game – it was an 8-bit retro style game, and that wasn’t something I’d demonstrated any ability in creating. I decided to spend a bit of time showing the developer that I could make sounds in their style – the game had a bit of gameplay footage online, on their Steam Greenlight page, and I decided to download it and re-do the audio. The original video just had music in the background, so I pulled it off YouTube, muted it, and built all the sounds of the scene from scratch. It was a lot of fun – and I was really pleased with the sound of it. I finished it, to the best of my ability, and sent it over the developer. His reaction was fantastic – he was clearly impressed that I’d take the time to do it, and I think he liked the sounds I’d made. After that point they were really enthusiastic about having me on board to work on the game, and despite the fact I didn’t have any experience with games previously.
All that said and done, it’s important to begin learning the actual practice of game audio as soon as possible, so you’ll have a bit more of a better idea of what you’re doing when you land these gigs. We’re lucky to be in a stage of the games industry where a lot of extremely high quality tools (that have a lot of potential) are within easy access, and in many cases, free. I found that my third year was the ideal time to learn how to use these tools and programs, and I have now reached the end of the year with a pretty solid understanding and range of ability within most of the major tools that are expected of a entry level game sound designer.
The Process and the Tools
I think there are a number of stages that you should work through in order to best approach game audio. I’ll speak specifically for sound design here, as that’s my main focus – it’d be slightly different if you wanted to be a composer. I think it’s really important to have a solid grounding in sound design to start with, before starting the approach the interactive side of things; spend time re-doing sound for clips from games or films, and send the results to a few industry sound designers for some honest feedback. This will give you areas to work on and improve, and slowly develop your skillset in the sound design area. Being skilled at sound design is a huge task – and people spend their lives perfecting it, constantly learning and developing their aesthetic. Unfortunately, the sound creation aspect of game audio is actually a very small – and a lot of the work is actually done in the implementation processes and code. Below I will look through the steps I think you should best go through when approaching game audio.
Sound Design and Field Recording
Sound design and field recording are, as I’ve mentioned, really key in the sound creation area. Sound design is a skill that can be learned through trial and error, and by following in the footsteps of many of the masters of sound design like Walter Murch, Ben Burtt, Gary Rydstrom, Randy Thom, Charles Deenen, and many more. Find a sound designer who’s style you like, and attempt to re-create what they’ve done. It is through a process of replication with your own creative injection that results in the development of your own sound ‘style’, and general understanding of the art of sound design. It’s the same with field recording; the techniques aren’t especially difficult, and we have more than enough kit at LSFM that’s freely accessible for you to record an entire library for your own personal usage. It’s more of a case of dedication – I’ve spent many nights in fields and woods with a few mics and the portable recorder, freezing cold, sitting as still as possible in order to record an owl, or some birds. It’s incredibly rewarding to hear that sound in a game or a film now, because I know the effort and commitment that went into it. Be ambitious with what you want to record; don’t discount anything as being too inaccessible or unachievable – if you want to record something crazy like a jumbo jet, or a lion, or the ambience from the top of a mountain, do it. The more effort you put into recording your sounds, the more you’ll have at your finger-tips when creating for games – this will really pay off, sonically.
I’d recommend getting to grips with Pro Tools mainly – something most people on the Audio Production course already will have done. Other industry standard DAW’s are things like Nuendo, Logic or Reaper. At the end of the day, it’s really a question of what allows you to create what you want to create, at a high quality – I’ve stuck with Pro Tools, as it’s more of an industry standard – but use whatever you’re comfortable creating in.
Middleware is essentially exactly what is says on the tin – it’s the step between audio creation and the game engine. Historically (and still currently) audio programmers will build custom audio tools for sound designers to work with that is specific to the project or game engine. At big AAA studios, where the game engine is proprietary (the studio’s own), the toolset will be also. Many of the bigger AAA companies still use their own middleware tools, and these are not publicly available. In recent years however, the demand for commercial middleware and audio tools has grown, and a number of different software developers have stepped up to fill that need. As a student, and an entry level indie sound designer, the two main middleware tools to focus on are FMOD Studio and Wwise. These are programs that will allow you to import your own audio files, arrange and ‘implement’ your audio into different audio systems for playback in the engine, and then export neat packages that the developers can pop into the game with little hassle (most of the time). This is ideal, because it allows a sound designer to create these complex audio systems within a DAW-like environment without a extensive knowledge of code, and with the precision of high quality audio creation.
The implementation part is a little tricky, to start off with, because of the interactive element. Coming from a background of working to a linear, fixed timeline, this can be challenging to appreciate and understand. For example, if I give my character footsteps, I have to realise that I’ll need to consider footsteps for all of the (for example) 15 different surface types the game developers have put into the game: I’ll also need to realise each of those surfaces could be in areas with different amounts of reverb, and I’ll also need to consider the movements of the character, and create sounds that reflect that; perhaps they are crouching, creeping stealthily, sprinting, or limping – each one of these possible alternatives has to have a sound created for it. Not only that, they also need to have enough variation for the player to not notice repetition in the sound, and simultaneously maintain a level of coherence that keeps the sound uniform and recognizable for the character. Suddenly what seemed like a simple footstep becomes a huge task, with a lot of different variables that must be considered for it to work properly. This is an example of a complex audio system – and compared with some of the AAA audio systems – that’s not really that complex at all.
I’d recommend starting off with one of the two main middlewares – FMOD or Wwise – and getting to know it as much as you can independently. Technically, you can begin creating audio systems in these without needing to touch game engines. There’s a wealth of good stuff on youtube that’ll teach you the basics and the more advanced aspects of each tool. Personally, I’d recommend FMOD – it’s very similar to a DAW in the way it is laid out, and it’s integration into game engines (once you reach that stage) is a lot smoother than Wwise (from my experience, at least). Both of the tools do pretty much the same thing, so either way it doesn’t really matter which you decide to learn. It’s the same with learning DAW’s – once you’ve learnt the basics of one, you’ve learnt them all.
Game Engines and Code
Game Engines are the tools game developers use to bring together all of the important aspects of a game and make something meaningful from it; they can include code, animation tools, material tools, props, 3D and 2D models, a physics engine, AI tools, UI tools, audio and much, much more. Most developers will choose an engine functions that best fit they game they’re trying to make; for example, if they wanted to create a 2D game, they might choose an engine like GameMaker, or Unity; if they wanted to create a 3D game, they might choose Unreal Engine or CryEngine; the choices are extensive. Each engine has positives and negatives, and really it’s more down to the preference of the developer.
Code is also a tricky one, as it’s a lot to learn, especially as someone from an audio background. The visual scripting system in Unreal Engine is ideal for learning how code works, and it can enable you to get started creating stuff really quickly; other engines like Unity will pretty much just use code for its logic systems, so this is something well worth learning (or at least having a grasp of).
Knowing the full ins and outs of engines and code isn’t 100% necessary, but it will enable you to have much more control of the way your sounds work within the game world. I’ve been lucky enough to work on projects where I have been able to sit down with the developer and implement the sounds with them; I’ve also worked on projects where the engine session has been shared and I’ve implemented all of the sounds myself, directly into the engine, with middleware. This is extremely rewarding, especially when you figure something out for yourself and it works well to serve its purpose within the game.
Personally, I’d recommend starting with Unreal Engine (UE4) as that has a nice introduction into the logic of code through its visual scripting system, Blueprint. It is also very visually focused, and you can get some great visual content up and running easily for you to create sounds for. It’s very well documented, and has a large user-based who are generally more than happy to help you when you’re stuck. As I mentioned before, it also has fantastic integration with FMOD studio – which is great for getting started putting sounds into the game world. Its internal audio engine is also really powerful and well worth checking out; it works in more of a modular system, much like the Blueprint visual script, and it’s relatively easy to learn.
So you’ve got some solid skills creating decent sound design in a DAW, you’ve figured out a middleware (such as FMOD) and you’ve got your audio functioning as you want it to within the engine – where next? Now you’ve got your skills, it’s time to start telling people about it. If you’re serious about getting into either the indie games industry, or the AAA games industry, you’ll want some kind of demonstration of your skills and abilities, for you to use as a enabler to get work. This is a showreel, and it’s one of the most important things you will create as an entry level sound designer. I created my own website to feature my showreel, some aspects of my engine/middleware experience, my CV and skillset, and my portfolio of games. The showreel is the one to focus on, as this is what most employers will look at for an overview of your abilities. It needs to demonstrate two main aspects of your work: firstly, it needs to show you have a strong sound aesthetic in your sound design work. Pure sound design is important, so don’t be afraid to include some linear clips in there to show your abilities. Secondly is interactive work; you’re applying for a game job, so make it clear you understand interactive audio and the tools – maybe include some clips from a game you have worked on, or even just a demo of an audio system you have created in FMOD.
Above all, your showreel needs to be engaging and eye-catching. Keep it short and simple, start strong and keep it strong throughout. If you’re sending it to an audio director at a AAA company, they likely receive showreels all the time; give them something interesting within the first 20 seconds, don’t expect them to sit through the full reel if it isn’t that great – they probably won’t. A strong showreel will be an invaluable tool for building your portfolio of work, so spend some time putting one together and make sure it’s impressive.
Along the way, I’ve picked up a lot of resources that have been massively helpful in shaping my understanding and skillset as a game sound designer. Podcasts have been huge, and I really can’t recommend checking out some of the game audio podcasts enough; Beards, Cats and Indie Game Audio pretty much taught me everything I know, and I really recommend anyone with an interest in game audio downloads their full podcast history and listens to it on repeat. Tonebenders is also really fantastic, and regularly features guest slots from some of film’s biggest sound names. It’s more of a sound design podcast than game audio specifically, but it’s still really helpful and interesting. I’ll also highly recommend the Game Audio Podcast, they regularly have really interesting content and guests, and their back catalogue features many years’ worth of GDC (Game Developers Conference) roundtables. It’s also worth mentioning the Game Audio Hour and Bleeps & Bloops. An incredible wealth of information and knowledge there, well worth investing the time and listening to as much of it as you can.
During the year I’ve also made great use of the More Books service at the library, and we there is now a fantastic range of game audio books available for anyone to use. There’s a number of fantastic Karen Collins books in there (which I highly recommend), as well as Aaron Marks’ ‘The Complete Guide to Game Audio’, Richard Stevens and Dave Raybould’s ‘Game Audio Implementation’, ‘The Oxford Handbook of Interactive Audio’, Scott Looney’s ‘The Essential Guide to Game Audio’, Martin Wilde’s ‘Audio Programming for Interactive Games’, Greg deBeer’s ‘Pro Tools 10 for Game Audio’ and lots more invaluable resources. I really recommend getting stuck into some books, as they can a massive help and knowledge based for your work.
I’d also regularly check out Designing Sound – they have a fantastic collection of interviews and articles from over the years about film and game sound, and I often find really interesting stuff on there that informs my own work and approach. A Sound Effect is also well worth checking out, they’re an independent sound effects website but they have a fantastic blog section with some really cool articles on. Lastly, there’s a wealth of great talks available on the GDC Vault, a collection of some of the talks and roundtables from many years’ of GDC.
So that’s it. Thanks for reading this huge post, I really hope it is helpful for someone at some stage in their game audio journey. If you’ve read this and you have more questions, please do feel free to contact me on twitter or email me at email@example.com and I’ll point you in the direction of someone who knows more than me. 😉
This week we were fortunate enough to receive a guest Lecture from Mark Hills, a Lincoln University Media Production Graduate and sound engineer in a highly reputable audio post-production house in Soho, London called Soho Square Studios. Soho Square studios are a Dolby approved studio and focus on audio post production for Advertisement, animations, feature films, and voice recordings for games.
Mark began the lecture talking a bit about his background and how he got to where he is now. He explained how similarly to most sound designers he fell in love with electronic music at an early age. He took media at A-Levels and then became a Lincoln University Media Production Undergraduate in 2007.
I won’t go too much in to detail about his background as he has personally blogged about it, this can be found here.
Mark then talked about the challenges of getting into this industry and working in a studio like Soho Square Studios. He explained the importance of ‘running’ and how although we have (will have) a degree, we will still be expected to start from the bottom as runners. He explained how it’s better to go for a running job at a smaller studio with around 10-15 people, as the opportunity to grow, learn and promotion is higher. Whereas the larger companies, you could find yourself being a runner for many years and not get anywhere.
He also gave us some advice for our CVs. He said that the studio aren’t always that interested in how many student films you’ve worked on, what they want to know is, if you have a degree; and have some real work experience. What he meant by that is customer focused work experience e.g. working in a pub. The studio likes this as it shows you can deal with, and work well with clients.
The Lecture gave a great insight to what it’s like working in an industry we are all passionate about, but also gave us all a bit of a reality check in regards to the amount of work it takes to get to his position. This was a slightly different approach than most guest lectures give, however I felt like it was good for us to get a bit of a reality check to push us more to where we want to go.
The day after this lecture myself and another AP student, Matt Jones went to London and visited Soho Square Studios to interview Mark and his manager Tom Mackewn for research in our R&D case study module. We arrived at the Studio and saw Mark in action finishing up a session, where we were quite impressed by the speed of his Pro tools skills. He then gave us a tour of some of the studio rooms and showed us some of his previous work.
When we interviewed Mark, we asked questions about his role as sound engineer and what he does day-to-day etc. I was interested to find out how the working hours were 9am to 6pm Monday to Friday, as most creative industries involve working unsociable hours. He told us that they don’t have ‘official lunch breaks’ and work around their daily schedules to fit in lunch, and also about some of the perks of the job – he said how he loved being able to do a hobby for a job and that you get to work with some amazing high status people.
Today we were lucky enough to hear from Suddi Raval, a legendary acid house producer and a game sound designer/composer, working currently as the audio manager at the Warner Bros. game studio, TT games (responsible largely for the ever popular LEGO games).
Audio manager is essentially the highest audio position in the games industry – the audio manager works with the upper echelons of the game development studio to create a fully cohesive and well delivered soundtrack. The audio manager is involved with both creating audio for the project (Suddi talked about how he generally likes to work in VO, both recording voiceover for the project and using audio content from an existing IP, usually a film), as well as managing the sound designers that the studio employ. Suddi mentioned the studio employ 7 full time sound designers, as well as up to 2 more on contracts. This is a comparatively large number of designers – even considering TT Fusion is technically a AAA studio, with many AAA titles having 2-5 sound designers at most. Suddi did however mention that the number rises all the time.
I was so engaged with what Suddi was saying (and had a million questions popping into my head) I didn’t make any notes on the actual content of the lecture, so this post is mainly key points. However, I found myself nodding in agreement and hearing so much affirmation in what Suddi was saying in relation to my own work and journey into the industry. I’d contacted Suddi about a month before the lecture, and had asked him a few questions and sent him my showreel. He very generously called me out by name and commented on my work, offering me feedback and advice as well as answering a lot of questions that I posed to him after the talk.
There were a few things Suddi mentioned that I want to draw attention to – as they relate in many ways to my project.
He spoke about FMOD, Wwise, and some of the games engines i’d demonstrated ability in on my reel: he talked about how TT use their own proprietary sound engine for implementation, that they generally taught junior sound designers as they join the organisation. He said however, that learning middleware like FMOD and Wwise was important, both for understanding the implementation process and to show that you’ve taken the time to actually learn something for your work. This is helpful and important for me, as I plan to use FMOD to implement my sounds into the game for my project. As I already have some experience using it independently, spending time using it in a game engine would be hugely beneficial.
I also asked him about junior sound designer positions, and the fact they are rarely advertised. I also later asked about the best route into in-house work, to which his response covers both questions – essentially, through work experience. Suddi said that a number of the people working at TT now came in through work experience: they’d proved themselves to be committed and easy to work with during the few weeks that they’d work at the studio, and they’d been asked to join the team full time. I think that work experience is something I should really look into further.
I’ll probably remember a lot of other things that stood out to me from the lecture, but i’ll leave this post at that. A fantastic opportunity to speak personally with someone high up in the audio world of the games industry, I learned a lot and found much of my ideas and preparations confirmed as being on the right track.
Games Designer Paul Bennun explores the growing popularity and ambition of music composed for video games.
Video games now have the resources to match that of the big Hollywood orchestral film scores. But it’s not just commercially that video game soundtracks are taken seriously. Composers are becoming more interested in it artistically and BAFTA and the Ivor Novellos have recently recognised the form with their own award categories.
Bennun traces the development of this new genre and finds out how it is changing the way music is made and consumed. Has it now left behind bleeps and bloops and arrived at the brink of artistic respectability?
Over the water, the Vancouver Film School runs a Sound Design course. Join Head of Sound Design for Visual Media Shane Rees for an exclusive inside look at BioWare’s Edmonton campus, and interviews with the many VFS grads on the Dragon Age and Mass Effect sound teams.