I wanted to make a 3D DAW, where spatial audio, physics, and virtual acoustics are all directly integrated into the engine.
This makes it easy to create music in 3D, and experiment with new techniques which aren’t possible in traditional DAWs and plugins.
I’d love to get any feedback on this software (Mac/Windows) to make it better.
You can download it for free through the website.
Thanks, Noah
Ambisonics, especially higher-order ambisonics, also is capable of spatial panning, though it encodes the 3D soundfield rather than individual sound sources. Again, there are a few physical formats.
Sorry if this sounds like some self-promotion - I specifically built this functionality[2] for my previous audio engineer so that he could quickly design and test sounds without the hassle of having to build/install an APK with every new audio file.
[0]: https://www.DodgeALL.com (only for Quest 2/3/3s/Pro headsets)
[1]: https://valvesoftware.github.io/steam-audio/
[2]: https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...
And then I recall there was this one old Kickstarter project that made headphones with that idea. It had an IR transmitter for head tracking.
I think it might be quite difficult to track head movement with just earbuds and not have the tracking drift, though.
It's definitely possible to have head tracking, I think it will just depend on what headphones and hardware offer it. It would be cool to give you that kind of VR listening experience but just on headphones
[1] Oddly the surround sound soundtracks that impress me the most are old VHS tapes encoded with Dolby Pro Logic
[2] let's see... Close to the Edge by Yes, Supernature by Goldfrapp, Maximum-Minimum by Kraftwerk
[3] https://en.wikipedia.org/wiki/Head-related_transfer_function
I've been developing a VR spatial sound and music app for a few years with the Unity game engine, bypassing the game engine's audio and instead remote controlling Ambisonic VSTs in REAPER. I can achieve low latency with that approach but it's a bit limited because all the tracks and routing need to be setup beforehand. There's probably a way to script it on REAPER but that sounds like an uphill battle. It would be a lot more natural to interface with an audio backend that is organized in terms of audio objects in space.
What I'd like is more flexibility to create and destroy objects on the fly. The VSTs I'm working with don't have any sort of occlusion either. That would be really nice to play with. Meta has released a baked audio raytracing solution for Quest, and that's fun for some situations but the latency is a bit too much for a satisfying virtual instrument.
Here's my project for context: https://musicality.computer/vr
What you’re working on sounds really cool, I’ll have a look at it!
It sounds like Audiocube offers the kind of features that you need, although it doesn’t have realtime audio input (yet, I’m working on it and have it partially working).
One downside of selling into the pro audio market is piracy unfortunately. I learnt that the hard way and ended up having to use iLok.
Yeah multichannel is a priority. I don't think it's too far away - maybe a month or so in the lab.
And true, although I'm not a fan of iLok.
Ableton, Reaper, Reason, and all associated VST and VSTi's from the incumbents like Arturia through to Korg and Roland, are all hardware-dongle free. Most use some sort of 30 day license server check-in or similar cloud platform intermittent auth.
N.B. I don't use Waves or UA software. Ever. Licensing models are akin to common assault.
I still remember walking into one of the highest profile million dollar mixing stages in LA, opening their Pro Tools plug-in menu and almost every single piece of software was pirated. Unfortunately even people who could pay for software (and had a strongly developed sense of IP rights for their own work product) seemed to have no issues ripping us off.
It seems like "bakes in" spatial audio to binaural stereo?
But who is the market for that?
I love spatial audio on my AirPods but a big feature is that it moves with my head and can even be customized for my ears.
And I certainly don't want it applied when downsizing to a mono Bluetooth speaker.
It seems like you'd need to export your final product to surround/Atmos for the way people want to, and currently do, consume spatial audio? I assume the target here is Apple Music, short films, etc.?
I mean I think the concept of the 3D DAW is great. I just want to make sure there's a product-market fit here, so you can succeed. Or is there a market I'm overlooking?
I think I need to narrow down the market a bit. Currently I'm aiming more at audio and sound design experimenters rather than the deep technical stuff - although that is a goal later.
I'd like to create a 'listener app' so people can have full immersion, control, and exploration of audio scenes.
The binaural process is more of a high powered, more flexible and controllable version of a binarual plugin, so this would be for making binaural mixes rather than slapping on everything.
I personally think the binauralizarion is one of the best sounding out there, it sounds great on headphones and has excellent sound localisation.
But you are completely right. I'm still working out product market fit, and I think it's why I'm finding a bit of friction reaching the right audience.
Thanks for your thoughts! Any feedback and input is very appreciated
Since you're going for a paid approach, it does seem like you need to locate the audio professionals mixing for spatial audio on Apple Music and TV shows, to see if you can show them why your 3D editor results in a better surround/Atmos final result.
I will say that I've often wondered what it would be like to "walk through" a band or orchestra as they were playing, with the binaural audio adjusting accordingly, which would require a full 3D approach like you've created, not just angles. I once attended an art project of a choir singing in 360° in a circle around you, where you were invited to move around to change the "mix" of voices, and it was magical. I think that's what you must mean by "exploration of audio scenes" and it sounds really cool. On the other hand, it's hard for me to imagine how you'd turn that into a business. Especially since I assume that's already relatively easy to do with video game/VR engines, even if perhaps the binaural effect isn't quite as good?
Good luck! I think the concept is very cool, and I hope there's a way to monetize it as an independent product.
Unfortunately that's no longer my hobby so can't test this for you but definitely scratches an itch for past me. Nice
Yeah, frustration with the existing solutions was one of the main reasons I started this project.
I was working in creating immersive audio, and I just found that none of the plugins allowed the level of placement that I wanted. It was all fiddly, and you couldn't easily move the listening position around, while also moving sound sources.
I came from an audio engineering background, not programming, so it took me a while to even learn how to make software! But now I made the tool I always needed.
How long have you been working on it out of interest?
[1]: https://www.afmg.eu/en/ease
[3]: https://10log.io/
I'd like to look more into the acoustic modeling capabilities later on - and give it some more advanced features
(asking because I listen to a lot of live jam music in stereo and noticed that they use a stereo mix with a virtual image)
For example, you could have an entire orchestra with each instrument close miked - and then arranged in audiocube in a vritual concert hall.
The user could even move around the space for full placement control - or you could simply render it to a binauralized stereo file for more immersive listening on headphones.
I'd like to develop a mobile "listening" app, which isn't for making projects, but listening to them and placing you in the virtual environment with control
I've had quad speakers before (with a "delay" unit for the rear speakers) but I think it would be hard to do 3D in stereo (I know about https://en.wikipedia.org/wiki/3D_audio_effect and HRTFs but I have never noticed any 3d-ness to the sound I'm listening to in games).
But does it support VST/AU in order to load instruments rather than "samples"?
It's just too much development work at this stage - not only the VST integration itself, but building a full MIDI sequencer timeline, editor, etc to support instruments.
Other DAWs do this perfectly well so I'm focusing on the 3D virtual environment moreso than tradtional DAW functions. Audiocube isn't a tool to compete with the likes of Ableton and Logic, but something to run on the side.
Although, if I had a big enough team I would love to give it all of these features. I'm just doing it solo at the moment, so I need to have a narrower focus.
(And of course, "and if you then want to consume the result in another DAW, you can". Chaining up multiple DAWs because they're all good at different things is one of the more fun things to do ;)
Currently it only exports/plays back in binaural stereo
So right now, is it possible to make something in Audiocube and export it to native spatial AirPod sound ?
At first glance I thought of: DEW
I sketched out an similar app, but never had super pressing need. I can think of many many uses for this from modeling performance spaces, minimizing resonances in industrial settings, crime scene reconstruction, art installations, speaker placement for large concerts and many more.
What research or similar tools did you look to for inspiration?
Some things that come to mind
Efficient Interactive Sound Propagation in Dynamic Environments https://cdr.lib.unc.edu/concern/dissertations/5425kb409
Precomputed Wave Simulation for Real-Time Sound Propagation of Dynamic Sources in Complex Scenes http://gamma.cs.unc.edu/PrecompWaveSim/
Immersed boundary methods in wave-based virtual acoustics https://www.pure.ed.ac.uk/ws/portalfiles/portal/257303782/Bi...
All of these uses are possible - my dream is for it to become much more advanced and powerful, almost like Blender but for audio and acoustics. It's still in early days and I'm a solo dev for now...
I'll look at those links. For me it was blender, and spatial audio plugins like the DearVR stuff, and game engines!
Or singing while walking through a tunnel?
Since it has capabilities that would be hard to replicate, rather than show the tool on the landing page, I would show the output. Remove the clutter and force people to listen to what the tool can produce.
The tool as it is now, is being marketed towards yourself, people that wanted to build that tool and know what it is. But everyone will know what it can do after listening to sample output.
You are right about the audio demos, I'll update the frontpage with this soon. Thanks!
Steam audio is pretty awesome in that regards because it supports HRTF and all the physical based goodies like occlusion/reflection and sound propagation. So you can get really really immersive spatial audio.
The only downside with this solution is that you can't do offline rendering. So my question is:
can Audiocube do offline rendering? seems like it would be one killer feature for my use case.
Offline rendering is at the top of my research list currently. It’s not in the app yet. It’s going to take a lot of work to build given that the physics engine calculations are tied to frame rate etc, so there is a lot of work to do - but I’m sure it’s possible.
Thanks!
I have a history experimenting with 3D audio - about 15 years ago I build myself a pair of ambisonic microphones, but until only recently I think the software support for ambisonic capture and mixing has been seriously lacking. Back when I built the mics I started working on a plugin suite for the processing, but I could never get it quite right. Nowadays, there are more 3rd party options I can use, and I will spend some time with this again :)
Yeah there is defiantly a lot of stuff missing or neglected in this field!
I used to use those plugins, but I found that there wasn't enough control.
Sure, you can automate the position of individual objects, but you cannot then easily move the listner's position in relation to your sources - which Audiocube can.
It gives you more control, flexibility, and realtime interaction when it comes to moving the listener around a moving scene. And you can control the distance of sound propogation, you can add in walls for reflections and occlusion etc - essentially building virtual acoustic environments.
