Apple Spatial Audio

People are sleeping on Apple Spatial Audio.

The mistake is thinking that audio channel technology such as 5.1, 7.1 surround sound can be separated from the audio delivery hardware technology.

Olivia Tambini has a recent piece focused on audio technology products including Sony 360 Reality Audio and Dolby Atmos.

As Olivia mentions, a company like Sony is going to have to not only sell hardware that plays Reality Audio, the company will have to convince media producers to use the branded format and then distributors to offer support for it in their apps.

That plan seems unlikely to succeed. The better bet would be to try and be the premier partner of Samsung Android phones, which is a pretty weak play compared to the audience they’ll give up on iOS.

Speaking of iOS and Apple, Spatial Audio has to be delivered using CMHeadphoneMotionManager. And you’ll need Airpods Pro or Airpods Max and an iPhone or iPad running iOS 14+ to hear it.

Some of my most tech forward pals have yet to even experience Apple Spatial Audio even though they have the equipment needed to play it back. So, early days still.

Interestingly, Final Cut Pro X has literally no mention of Spatial Audio in its help documents. It is as though the technology doesn’t exist. I found an example of filmmaker Gary Yost, a serious OG of 3D everything who managed to map spatial audio into Final Cut Pro back in 2018. I love this quote in that blog entry:

“Audio is as important as video, people!”

Gary Yost
3D OG

Big film and game productions are already mixing audio for the tech though. Sound Particles seems to be the premier software solution, advertising some huge films that have used their tool to mix (program?) spherical audio.

Yet while there seems to be plenty of traditional media ready for this, traditional films can’t possibly offer the immersion expected from spatial audio because the viewer is never in the scene. We’re observers of scenes from constantly shifting vantage points.

In that way, Apple’s head tracking technology, which is undoubtedly part of a future AR platform almost points away from the way media was, or rather is:

There’s a recent interview with Joel Douek discussing “Spatial Audio and Immersive Content” on the 2nd Level Podcast. Notably, the podcast plays back in the same way as all others.

It isn’t that the production of audio that is not mixed for spatiality is wrong. Nor is it wrong to be lacking a device capable of playing back spatial audio with head tracking.

But I suspect we’ll see audio production from today the way we see 480i SD video or that square stuff of 80s and 90s TV.

Given there is a huge amount of amazing audio, I wonder what remasterings for spatial audio will look like. They can change the art dramatically.

David Simon’s blog entry on HBO’s “remastering” of The Wire speaks some to how important it is that content creators retain creative control over their work.

Can the remaining Beatles remaster Sgt. Peppers Lonely Hearts Club Band for spherical, 3d audio? Would Lennon have put himself on the left side? Does the French horn quartet over the bridge play in a group over to the right or are they surprisingly up on a balcony behind us?

Who has the authority to make decisions about how Purple Rain’s intro should be mapped into a 3d space? And how mad would Prince have been that anyone was even considering doing such a thing.

Edit: My brother Ryan pointed out that Sgt. Peppers has already been remixed for 3D audio, in part through the Cirque du Soleil “Love” production.

A trivial amount of searching reveals that the 50th anniversary editions of Sgt. Peppers and Abbey Road include 5.1 audio. The official page suggests that Dolby Atmos mixes were created by Giles Martin and Max Okell:

The anxiety from fans was perhaps that Martin could have gotten trigger-happy and experimented a little too much with the mix. There is a great balance struck on this master, with plenty of subtlety in the use of the Dolby Atmos 5.1 mix.

Giles Martin even spoke of some of the adjustments he had to make to stay relatively true to the original recordings; “Now, modern-day recordings, they have all drums in the center, because they sound stronger. And actually we did that with “Come Together,” and then we realized it that it didn’t sound as good. The drums need to be slightly to one side — they need to be slightly off-kilter, if you like — in order to make it sound cool.”

Abbey Road – 50th Anniversary 5.1 Mix – Giles Martin, By Ben Jacklin

Rock Show, an App for the Apple iPad

My software company, Neutrinos, has been hard at work on an application for the Apple iPad called Rock Show.  Rock Show is an interactive art gallery for concert posters that lets you view, share and buy original limited edition artwork.

We’re very excited about the potential of the iPad and hope that this application does the device justice enough to be included in the launch of the grand opening of the  iPad App Store.

For more information about Rock Show, visit the official website, RockShow.fm. You can enter your email address to be notified of availability. Be sure to follow the app on Twitter @r0ckshow.  (The first o is a zero)

Killer Application for Apple Tablet is Board Games

There’s a lot more speculation about the tablet since a New York Times blog entry that included a quote that people will be “surprised how you interact with the new tablet.”  A lot of the conjecture is based on applications for patents that Apple has filed in the past few years.  I have an angle that offers interesting interaction with the Apple Tablet, iSlate, iGuide or Apple Slate with a killer application is decidedly low-tech.

The idea is that the iSlate will communicate and be driven in part by nearby iPhones and iPod Touch devices over Wi-Fi or Bluetooth.  Authorized iPhone and iPod Touch devices nearby the iSlate will act as sophisticated remote controls.  This use case takes advantage of the user’s likely existing Apple hardware in the Touch platform and focuses on the fact that the iSlate will likely have a large, bright screen with a reasonable viewing angle. Let’s examine a potential killer application of this idea.

The tablet screen will be able to lie flat on a table between two or more people.  Through iTunes, the owner of the tablet will purchase an application that runs full screen on the tablet.  The publisher of the tablet application will also make available free applications that run on the iPhone and iPod Touch and communicate directly with the app running on the tablet.

Using the Apple Tablet as a Centerpiece to Electronic Board Games

A great example would be the classic board game, Scrabble.  In Scrabble, you have a board that requires physical placement of letters on a major center area.  It also requires players to keep the letters in their tray secret from other players.  In a Scrabble for the iSlate scenario, players need only to have an iPhone or iPod Touch to virtually hold their letters.

The iPod Touch or iPhone could vibrate or make a noise when it is a player’s turn and it would be used to send the results of the player’s decisions to the Apple tablet at the center of the table.

One important aspect of this is size, the traditional Scrabble board is a fair bit larger than the expected 10.1″ Pixel Qi powered display.  I believe that this type of thing could be overcome by the tablet giving intelligent focus to the most relevant portions of the board.  Additionally, the iPod Touch and iPhone could be used to manipulate the current viewing area on the tablet, or that area could be manipulated directly using standard multi-touch.

Many other board games would work in this scenario, including Monopoly, where you are handling all kinds of cash, property cards and frequent score calculations. The value you get out of the pairing of a Tablet and the iPhone is that there is a new bridge between physical social interaction and the convenience of an electronic presentation.

Using the Apple Tablet to Share Presentations in Small Groups

There are many other killer applications where the iPhone and iPod Touch serve as input devices to the iSlate or Apple tablet.  One I would use in my meetings for my iPhone application design company, Neutrinos, LLC is a business application is in small group presentations.  The tablet is set up to run a presentation application full screen.

The tablet is handed to one or more clients seated near each other.  The presenter uses an iPod Touch or iPhone to control playback of the presentation.  The presenter’s multi-touch device offers presentation notes, previous and next slide previews and allows them to trigger in-presentation events like animations or even jumping out to a web-view for the clients to explore.  This gives the presenter an opportunity to give a private and engaging presentation in the middle of any cafe, airport or public space.

Using the Apple Tablet for Illustration and as a Supplemental Display Area for Floating Windows

A final use case scenario for using the iPhone and iPod Touch to drive the Tablet is in illustration.  I’ve been working directly with Portland artist Carolyn Main who spends a lot of time with her Wacom tablet.  While the Wacom offers a great deal of pressure point precision that the Apple Touch Platform is unlikely to compete with any time soon, it is reasonable to think that app developers will try to deliver illustration and animation applications that allow creation on the go.

Having an illustration application like Adobe Illustrator running on the tablet, and then being able to use an iPhone or iPod touch for swatch, tool or layer management would leave more of the iSlate’s screen real estate for drawing.

In John Gruber’s recent post about the tablet he writes: “And so in answer to my central question, regarding why buy The Tablet if you already have an iPhone and a MacBook, my best guess is that ultimately, The Tablet is something you’ll buy instead of a MacBook.”  My suggestion is that people buy the tablet because they already have an iPhone or iPod Touch.  Having both a Tablet and a touch makes the Tablet more useful with some great use case scenarios the low-tech realm of board gaming to business.

follow rob on twitter