Game Audio In The Cloud, Part 2

By Peter Drescher
April 10, 2010

Last month, at the Game Developer's Conference in San Francisco, I talked about Game Audio In The Cloud, and I wanted to expand and explore the topic in a bit more depth. But first, some definitions!

The Cloud: What Is It?

When I talk about The Cloud (Capital T, Capital C), I'm talking about a currently fictional technology. Despite advertising claims and vaporware demonstrations at trade shows, The Cloud (as I envision it) does not yet exist ... but when it does, it will dramatically change the way we do business, listen to music, and play games.

The Cloud is basically The Internet, redesigned for an ultra-wideband wireless world. The basic components are servers, lots and lots of servers, interconnected via high-speed trunk lines, quite like Internet servers are connected today. But instead of sending and receiving packets of data from personal computers, Cloud servers up-and-download information from wireless devices, i.e. cell phones.

That may not seem like a big difference, particularly since cell phones these days resemble tiny powerful computers. But it's actually a leap in both quality of service, and quantity of data, that heralds a new kind of technology, with interesting implications for wearable computing, social networking, audio/video production, and lots more.

Cloud Services: What Are They?

A Cloud Service is basically a wireless communications protocol that allows a Cloud-connected device to talk to the servers ... and it's a conversation, not a broadcast. Cloud Services will receive and store as many pictures uploaded from mobile cameras, as they will download JPEGs from web pages. It's a back and forth dance of data, even more so than with Internet-connnected computers.

Cloud Services will store ALL of the data your mobile device encounters or creates ... and that includes not only pictures you've taken, but emails you've written, contact info you've entered, music you've listened to, games you've played, social networks you've used, books you've read, items you've purchased, and on and on. Simultaneously, Cloud Services will access all the vast realms of data on the Internet, and deliver it to the palm of your hand at the touch of a button (or a voice command).

Most importantly, the Service is where your data lives, and any information stored on your Cloud Device is simply a copy for convenience. Your Cloud Service account will be the sum total of the digital trail produced by all your online personae, accessible at anytime, anywhere, by you, and by anyone you approve (and barb-wired, steel-gated, megabit-encrypted, firewall secure from everybody else!)

Cloud Devices: Where Can I Buy One?

terminal.jpgYou can't ... yet. A Cloud Device will work differently than today's fast-processor, massive onboard storage, cell phones. Instead, it will be a "thin client with a fast connection". Why? Because when you've got a fat enough wireless pipe, with totally reliable, completely global coverage (remember, we're speculating about future technology here), there will be fewer and fewer reasons to store ANY data in your pocket. A Cloud Device becomes more like a dumb terminal than a smart phone.

In fact, The Cloud, with its Services and Devices, will function more like an old-school 1970's IBM mainframe than a modern Internet network. Early in my checkered career, I worked as a programmer on one of those OS 360 behemoths: the thing was the size of 8 big filing cabinets, required a raised-floor, temperature-controlled, machine room to house it, used floppy disks the size of Frisbees, and yet did not contain even a single Megabyte of memory.

I wrote my programs in COBOL using a terminal, basically a keyboard and screen in a single unit, that connected to the mainframe down the hall. (Note: this was a huge step up from the previous input method: punch cards!) Terminals had no CPU or memory; the only electronics inside were for displaying monochrome text on the screen, and for adding lines of code to the file on the mainframe.

But overall, system functionality was totally analogous to how The Cloud will work. The Cloud is the mainframe, the Cloud Servers are the processors inside the filing cabinets, the Cloud Services are the multiple tasks running on the OS, and the Cloud Devices are the terminals ... get it?

A Mainframe Computer with Terminals vs. Servers in The Cloud with Wireless Devices

The only thing running along the wires from terminal to mainframe (or wireless from device to server) are commands up, and data down. No processing is done on the terminal/device at all. The work happens on the mainframe/Cloud servers, and the result is sent back down the wire/less in a format the terminal/device can display. It's actually a much simpler procedure than maintaining a network of computers.

The Sidekick Connection

Not coincidentally, the terminal model is exactly how the Danger Service works. It's a fast network of powerful servers running smart software that collects and formats data for wireless delivery to and from low powered mobile devices. Any data entered into the device (aka the T-Mobile Sidekick) is stored on the Danger servers, and transcoded web pages are sent down to the device quickly and efficiently, on demand.

sk2009.jpg Of course, nobody thought about it that way when the product launched in 2003, but people now recognize the Danger platform as a pioneering implementation of a Cloud Service, and the "thin client hiptop" as a prototype Cloud Device. Obviously, the Sidekick does more than just send up HTTP requests, and display websites: it makes phone calls, plays music, runs downloadable games, etc. Also granted, futuristic Cloud devices will be sending up more than just text commands and address book info. They'll be transmitting pictures, audio and video streams, GPS locations, and more.

But the general idea of "raw data up / processed data down" still holds true, and demonstrates the fundamental difference between The Cloud and The Interwebs.

The Google Connection

When Andy Rubin (one of Danger's founders) went to Google, he hired many of my Danger friends and colleagues to help build the Android platform, so it's not surprising that many of the features and functionality (even the code!) seem somehow familiar to me. Android is what the Danger OS might have become if a) we hadn't been so beholden to the carriers, and b) we'd had all the money there is.
SO if the Sidekick was a prototype Cloud Device, you can think of the Nexus One as another step in the evolution of cell phones towards a Cloud-based system.

However, the N1, far from being a dumb terminal, is actually quite smart. It sports a spiffy CPU, Gigabytes of storage, downloadable apps and music, and much more. And of course, in a departure from the Sidekick form factor (and like the touch-screen iPhone), it has no physical keyboard.

Now, readers of this blog might recall how much I hate using virtual keyboards, so I was quite happy to discover that the Android voice-to-text feature works well enough to practically eliminate the need for physical keys. The inclusion of a Sidekick-like trackball also facilitates fast, precise, cursor positioning in text fields, so editing mistakes, and adding stylistic embellishments is easy -- in contrast to the iPhone's "point and hold to reposition cursor" procedure, which is seriously inconvenient (particularly if you've got fat fingers).

A Working Cloud Service

But the thing I like the most about Android voice recognition? It's an excellent example of a working Cloud Service. You talk to the device, it records and sends the audio data up to the massively powerful Google servers, the audio file is translated into text using outrageously fast processors with access to vast farms of "sound-pattern => word" databases, and the result is transmitted back down to the phone. Pretty slick!

Notice that the CPU intensive work of audio-to-text translation is done by the server, not the device. Notice that voice recognition doesn't even work without a 3G or WiFi connection (like an unplugged terminal, a Cloud Device is kind of a brick without access to a high-speed network). Plus every time you talk to the device, the audio patterns are weighted into the database, increasing the accuracy of subsequent translations. It's pretty good already, though it can make amusing hash of slang expressions. Even more entertaining (or annoying, depending on your viewpoint) is that it recognizes curse words quite well ... and ##censors## them!

Also Cloud-like, Sidekick-style, most of your data is stored on Google servers linked to your GMail account, accessible and updatable from multiple platforms, and easily transfered from one Android device to another. Of course, downloadable apps and MP3s break the Cloud paradigm, but I believe even that will change moving forward.

SO, what about Game Audio In The Cloud!?

Uh, yeah, that's the topic I set out to discuss, isn't it!? And that's what Part 3 of this series will be about ... stay tuned!

   - pdx

You might also be interested in:

News Topics

Recommended for You

Got a Question?