The cool thing is that after the song is analyzed on the server, the client can recompute and preview the results completely client-side through an implementation that uses Web Workers and WebAssembly. The audio previewing uses Tone.js. I am thinking of writing up some more details about the implementation in the future.
I'm still working on a way to explain this easily, but I like the idea of carrying over the concept of content-aware fill from images to audio.
Please let me know if you have any comments or questions!
But for today’s music, shortening the 2010s/2020 already shorter lengths would mean a song might not be more than a minute in length. On average, full unedited tracks today end up being a bit shorter than they used to be, solely due to the economics of streaming. Rather than paying for the content second by second, it is done by paying per track play. The result is a lot of 2 minute tracks, which were produced with the “verse” parts getting jammed together into the “chorus” with no break in vocals, which also uses pitch adjustments, “the “bridge” is an afterthought that is terrible, or more recently, nonexistent……… Instrumental solo? Anyone? Bueller?
Music is no longer anticipated, budgeted for, and purchased on launch day with great fanfare. We have grown accustomed to the idea that we should have everything available at our fingertips, and as a consequence of this we get exactly what we pay for.
A few days ago I was given the task of creating a corporate video - just a rolling slideshow for a shop-window display. Then suddenly it was going to go on YouTube as well - so needed some music! I found a suitable track but needed to edit it for length so its closing chord coincided with the credits card at the end of the video.
This tool might have saved me the bother of splicing the music in Audacity.
and that’s just the first random 2021 song I can come up with within a second.
Tonight I was at a Melvins show in Amsterdam. It is their 40th anniversary your, amazing! I’ve also seen Einstürzende Neubauten a few months back. Back in early 90s everybody was talking how the Stones were still playing for so long. But nowadays so many bands are still going.
I'm sorry, that's the best I could do.
I adore music, always have and follow what I hope are "real artists".
I haven't bought physical media for several decades and often buy albums weeks or months after release.
I think you've sliced this wrong.
But I mostly agree with you - I get very excited for artists that I really enjoy releasing new work, and "line up" so to speak by listening to it the day it releases on Spotify.
Identifying the good from the rest in such a place as Apple Music is a miserable experience. To Apple, they also make more revenue from their prioritization of marketing similar sounding two minute tracks in this “nouveau pop” format, backed up by a small amount of older superstar artist anniversary editions. Good original new music never makes it to the featured content sections.
And Genres and algorithms are a mess. It applies across the board to all music and is really a problem for the flavors of House, Techno, and other genres that are simply labeled “Electronic” or “Dance.” I’m getting Avalon Emerson one minute, Bicep the next, to be ruined by corporate mass marketed deadmau5, follow by a Bad Bunny remix….worse is the algorithm which thought I might like corporate edc-esque superstars and poorly autotuned remixes- though I have not added a single song to my collection in 20 years of digital music consumption…
Can we do better than “Dance” and “Electronic?” Of course they could, they haven’t. One must go elsewhere.
For House and Techno, this dearth of music discovery and search-ability methods by streaming companies makes room for independent music station alternatives, like Fault radio, or gives a reason for one to seek out artists via other means, like going to independent music festivals like Sunset Campout, Honcho campout, and other events highlighted on Resident Advisor, a poster in a nightclub, a text message listing the warehouse location.
(Apple Music did at least bring back Beats in Space.)
I think about the proliferation of streaming, and how it actually makes finding new content difficult for people who are not familiar with those other means of distribution.
- We have deprioritized the concept of the local radio station, now what was alternative rock is a rebroadcast of an AM Sports broadcast (i.e KFOG)
- the death of the sale of “singles,” made it a cheap entry point for people to experience a new artist. Releasing a track on Spotify doesn’t feel as substantial.
- and exclusively agreements contract provisions with corporate entities that engage in predatory practices that force up and coming artists to choose between performing for their fans at local venues, or extending their potential reach by putting their name on the bill for Coachella, sacrificing potential shows for a few hundred of miles away and several months on either side of the festival.
- Additionally entertainment conglomerates like AEM and LiveNation are increasingly becoming the owners, or managers of, the entertainment venues in cities around the world. Similar exclusivity agreements can have a significant negative impact on unaffiliated independent venues ability to compete.
I can’t speak for anyone in particular from the Gen Z or Alpha generation. I think for them, it’s all Apple or Spotify, music festivals if they can afford them, sharp discerning music choices on TikTok if they feel the need to branch out… and the question is - do they?
The norm is now a fully mass market formula that is almost impossible to break through… and the effect of this puts the chill on the ability to link good music with committed audiences.
I think I mostly explained how I feel there…
I'm just barely young enough to be a zoomer, so maybe my view will be interesting. From my perspective my ability to find "good" music is better than it ever would have been in the past.
I find music suggestions from forums, review sites, subreddits, friends, online people I follow, etc. Then I can immediately listen to it with no effort or expense. I found my favorite album of all time from a random comment someone left on an internet thread.
To me the idea of having to wait until a local radio station played a song, and then make a leap of faith on purchasing the album seems like such a worse experience.
Similarly, in terms of creating music, it has never been easier to learn, create and distribute your own music. The rise of the internet has made it so much easier to find a niche communities of people making incredibly diverse and experimental music together.
In my mind, streaming sites have two roles and do a very good job at both:
- make all music as accessible as possible
- suggest music for people who want to "passively" listen to music, which is how the average person has always listened to music (and that's a completely legitimate thing to want and enjoy)
I see a lot of "the death of cinema", "the death of music", "the death of video games" takes around, and I can't help but feel like these views come from people who have lost track of where the "niche" communities has moved on to and then feel despair when to them it looks like the "mainstream" is all that exists.
I hit upon the plan of taping stuff off the radio onto 1/4", and then I could splice a not-talked-over beginning onto a not-talked-over ending.
Later, I worked out that I could extend or shorten tracks, particularly if I could get a tape of the instrumental version, using the same trick.
No-one uses tape and razors these days, but it was good fun.
I think music suffers even more so because we're all so tuned into having the best at our fingertips that if a single moment in a song isn't to our liking we can skip and forget about it completely - i think this fuels the fast-short song market, easier to saturate with many short songs and get listens rather than to work/slave on a longer more intricate piece.
Back in the day, mixtapes with songs were slaved on and cherished, today slaving over something is seen as a negative quality.
I do sometimes worry about this. Some of my favorite songs growing up were ones that I didn't care for initially but which grew on me only after listening to it repeatedly. Now it's easy to dismiss anything that doesn't hook me right away. Thankfully albums are still being released, and you can force yourself to listen though them, or keep more of the "meh" songs in your playlists for longer periods of time, even adding them back into rotation after a while.
I don't think I'd give up the variety we have now though and go back to only having what songs are pushed at us through top 40 radio or the limited selection found in local stores. I get my music now from countries all over the globe. Finding old stuff all the time I'd never heard and new things just released.
We can still invest in the music we listen to and be rewarded. We just aren't forced to, so it needs to be deliberate.
I don't see any of the effects you describe on my feeds (tidal; previously apple music). Perhaps you need to switch to a service with a better recommendation algorithm, or nuke your personalization profile and start over.
Fast forward to something like Radiohead's Kid A and you have orders of magnitude more complexity going on the track. So many different sounds layered in very complicated manners. Its almost like a classical composition how there are motifs, movements, different emotions being evoked, but with any sound imaginable compared to what few are possible from the orchestra ensemble. A song from Kid A is just a part of the greater album itself. Nothing was made to stand on its own between radio advertisements.
It seems these days we are reverting to how commercial music was always made. Very commercial studio focused with the artists removed from production. Generic lyrics written by low paid songwriting staff and same old tried and true chord progressions we've heard forever. The artist is a brand and a sexy person meant to sell products versus someone particularly talented with an instrument or with songwriting skills.
It is easy to create a supercomlex composition, that’s what classical music have been doing way before 1970s. But to capture the minds and hearts of many people with simple things, now that’s the real mastery.
I think it was https://eternalbox.dev/ since that's all I can find on Google. But that site is down.
He created some excellent products from the Rdio API, and later Spotify … and I believe his analysis engine ended up being the foundation upon which Spotify's _play more tracks like these_ capability is based.
Looks like he's moved over to publish on Substack—there's a recent(ish) post reflecting on 10 years of Infinite Jukebox: https://musicmachinery.substack.com/p/the-infinite-jukebox-1...
However, that wasn’t the end of the Infinite Jukebox. An enterprising developer: Izzy Dahanela made her own hack on top of mine. To make it work without using uploaded content, she matches up the Echo Nest / Spotify music analysis with the corresponding song on YouTube. She hosts this at eternalbox.dev. It runs just as well as it ever did, 10 years later.Edit: I remember a multitrack file format from the past that allowed following both a composer-defined path as well as random/infinite ordering of sequences, it was called digimpro: https://www.kvraudio.com/forum/viewtopic.php?t=102403
Did I over-attribute the above to Paul? I didn't realize he wasn't a founder.
There is this version now, but it does not allow custom uploads anymore: https://eternalboxmirror.xyz/jukebox_index.html
Muchos Gratias to whoever keeps that spinning. I've been itching for infinite Jepsen for a while - but discovered the original site was very dead.
I don't remember exactly but back in the radio days, songs had to be a very specific length to be a pop single.
Where do you draw the line between acceptable "edits" vs unacceptable?
There are so many tools to customize and make your audio experience your own. Ultimately my ears are my own and I love living in a world where I can precisely control my audio experiences.
Interesting, as the primary use case for this seems to be something along the lines of editing music to fit into video projects.
https://www.youtube.com/watch?v=ZYT9vQLxt5c
For people who know every beat of Strobe, it sounds odd when things don't hit the same way, but at the same time it's kind of like a unique live mix.
I like to have multiple hour continuations of songs which I use to help me fall asleep. In the past I've made my own, but I'm terrible at audio editing. It is a ton of work and, for me, really hard to get right.
After a little bit of experimenting with Mofi, it seems to do a very good job and selecting when to repeat a sections.
[edit]
Here's my first attempt for a 30 minute song:
https://mofi.loud.red/edit/ebbf4b410181aa767152945cbb6a2d679...
You can use this example though if you just want to try: https://mofi.loud.red/edit/8bd3fdf780f8c3927e41029f3b957f8a7...
"Not Imported You haven't imported this file yet. Import it first to use it."
He just butchers an otherwise perfect song!
Will report back with results.
Edit: Results are great! https://voca.ro/13ar1g88LSKK
I'm guessing that's the timeout?
There are a few songs I love, but some of the best parts seem to be "buried" under other instruments.
For the backend, I found it helpful to implement monitoring proactively to be able to detect and debug issues without having to bug users which has been really helpful so far and allowed me to make some performance improvements too!
For the frontend it was interesting to optimize the result searching using the Web Worker + WebAssembly approach. I found some interesting ways to speed up things by parallelizing and, interestingly, manually collecting garbage in the WASM runtime.
And of course learning about the entire audio analysis portion and reading on how to find similar sections of audio!
I uploaded Wuthering Heights[1], selected just the trailing guitar solo (3:09 to just before the fade-out starts), and got a bunch of seemless sounding 3m tracks of just that guitar shredding away.
All the ones it generated were perfectly cut and pasted, with one exception in track 2 where I could tell where it was cut/pasted.
Can't wait to try it with the guitar at the end of Brothers in Arms (Dire Straits), or the flute solo in Locomotive Breath.
If anyone is looking for a good use of AI, I wouldn't mind a webpage that lets me say "complete this solo that was abruptly cut short" and get something great.
[1] This one is especially sad because the sound engineer/producer later lamented that he faded the solo out so quickly because the player was still improving like mad and what he had left after the fadeout was apparently better than what he had done while recording.
> On repeat: Make an extended version of your song's favorite part by choosing the catchy part and seamlessly repeating part of it!
John Mulaney's "Best Meal I Ever Had" story[1] had me wanting to do this for Tom Jones' What's New Pussycat so I could play that as an inside joke with some friends. This would definitely have made that easier. (Ultimately, I had a version of the song which started in the middle and "ended" right before it "started" so the effect could be mimicked if the file was played on loop. This had a short silence manifest as the media player "loaded a new song" from its perspective; didn't have that issue in Audacity.)
[0] https://www.pokemon.com/us/pokedex/loudred
[1] https://www.youtube.com/watch?v=Tv1l1eUhN-E (Usually called "The Salt and Pepper Diner")
And yes, I had this issue while testing also! A lot of players added a small gap when looping which was a bit annoying to test with...
Is the project open-source?
If you are interested in audio analysis in general, I would also recommend checking out librosa: https://librosa.org/doc/latest/index.html
You can use -F to get the available formats and find out an audio-only stream then -f FORMAT to download the specified stream.
1] A lot of the heavy lifting is done client side, which is excellent; I have a super fast processor! But, watching the CPU utilization, it seems to spike the CPU for a few milliseconds, then waits for a second or two and, spikes the CPU again, then repeats. I suspect there may be some significant performance improvements to be had on client recomputes.
2] While it is doing recomputes, it would be great to get a progress bar or, at least, a message saying, "This will take a long time, be patient." It took me multiple attempts to determine that the service was indeed working, just that my client was taking its sweet time recomputing.
With Eternal Jukebox I was able to emulate it by setting the jump percent very high manually at the right time (but had to be paying attention + had to get lucky with which branch it took), whereas here I'll see if I can get it going by calculating what the cut I want would set the song length to, and requesting that length and fewest cuts. (Edit: didn't work)
Sometimes the simplest features are the easiest to overlook but most useful
This is perfect for TikTok videos!
Will definitely be using your tool. Very cool!
> I am thinking of writing up some more details about the implementation in the future.
Please do :D
at 00:00 a 00:16 segment (00:00–00:16)
at 00:16 a 00:51 segment (02:05–02:57)
at 01:07 a 02:31 segment (01:34–04:05)
at 03:39 a 00:11 segment (04:03–04:14)
Though it's just anecdotal but I have seen that the quickest way to enter into the zone is to play repeating loops at 140 bpm.
Unfortunately I haven't really found many of these as most of the tracks I love are non looping. I think this can take care of it. Kudos on making this!
I set it to avoid the segment that the key changes and to favor the 2 solo segments.
It doesn't seem like its doing a good job. Only the first 5 seconds of the intro made it and then is just repeating over and over the big solo. In the end, a very small part of the short solo makes it in as an outro.
Maybe it works better at shortening songs or maybe changing duration closer to the original duration.
A less rosy scenario is like Spotify generating music to get around having to pay artists.
For example, ESR 102 does not have the feature and it breaks the site.
It needs to be generalized to video content, but the task is a bit easier.
Found a bunch of cool related links here [1].
1: https://en.wikipedia.org/wiki/Music_information_retrieval
I wouldn't mind giving money to services like this