The answer is, almost unfailingly, "this paper applies perfectly to our technique because we are just rehashing the same ideas on new modalities". If you believe it's unethical for GenAI models to train on people's music, isn't is also unethical to trick those people into posting their music online with a fake "defense" that won't actually protect them?
But 'protection' of any one song isn't the entire point. It only takes less than a fraction of a percent of corpus data to have persistent long term effects in the final model, or increase costs and review requirements to those stealing their content.
As most training is unsupervised, because the cost and limited access to quality, human labeled data, it wouldn't take much if even some obscure, limited market, older genres which still have active fan bases, like Noise rock to start filtering into recommendation engines and impact user satisfaction.
Most of the speech protections, just force attacks to be in the perceptible audio range, with lo-fi portions like those of TripHop, that would be non-detectable without the false positive rate going way up. With bands like Arab On Radar, Shellac, or The Oxes, it wouldn't be detectable.
But it is also like WAFs/AV software/IDS. The fact that it can't help with future threats today is immaterial. Any win of these leaches has some value.
Obviously any company intentionally applying even the methods in your linked paper to harvest protected images would be showing willful intent to circumvent copyright protections and I am guessing most companies will just toss any file that it thinks has active protections just because how sensitive training is.
Most musicians also know that copyright only protects the rich.
https://securitycryptographywhatever.com/2025/01/28/cryptana...
For early experiments people literally took Stable Diffusion and fine tuned it on labelled spectrograms of music snippets, then used the fine tuned model to generate new images of spectrograms guided by text, and then took those images and turned them back into audio via re-synthesis of that spectral image to a .wav.
Riffusion was one of the first to experiment with this, 2 years ago now: https://github.com/riffusion/riffusion-hobby
The more advanced music generators out now I believe have more of a 'stems' approach and a larger processing pipeline to increase fidelity and add tracking vocal capability but the underlying idea is the same.
Any adversarial attack to hide information in the spectrograph to fool the model into categorizing the track as something it is not isn't different than the image adversarial attacks which have been found to have ways to be mitigated.
Various forms of filtering for inaudible spectral information coupled with methods that destroy and re-synthesize/randomize phase information would likely break this poisoning attack.
More specifically, there are a few moving parts here - the GenAI model they're trying to defeat, the defense applied to data items, and the data cleaning process that a GenAI company may use to remove the defense. So we can look at each and see if there's any reason to expect things to turn out differently than they did in the image domain. The GenAI models follow the same type of training, and while they of course have slightly different architectures to ingest audio instead of images, they still use the same basic operations. The defenses are exactly the same - find small perturbations that are undetectable to humans but produce a large change in model behavior. The cleaning processes are not particularly image-specific, and translate very naturally to audio. It's stuff like "add some noise and then run denoising".
Given all of this, it would be very surprising if the dynamics turned out to be fundamentally different just because we moved from images to audio, and the onus should be on the defense developers to justify why we should expect that to be the case.
What artists don't realize by this they are just improving the models relative to human capabilities. The adversarial techniques like, for example making a stop sign look like something else, well likely be weeded out of the model by a convergence of model performance to average or above average human performance.
TLDR: Once these defenses are broken, all previously protected work is perpetually unprotected, so they are flawed at a foundational level.
Ignoring these arguments and pretending they don’t exist is pretty unethical.
In some of the earlier image protection articles the people involved seemed rather shady about the capabilities. Would have to do some HN searching for those articles.
But everything at the end of the day will be a scheme if the end result is for humans to listen to it. You cannot make a subset of music that can be heard by humans (and actually sounds good) that cannot be prefiltered to be learned by AI. I've said the same thing about images, the same thing will be true about audio, movies, actions in real leave, et al.
These schemes will likely work for a few of the existing models, then fall apart quickly the moment a new model arrives. What is worse for defense is audio quality for humans is remaining the same while GPU speeds and algorithms increase in speeds over time meaning the time until a model beats the new defense will trend to unity.
It will be really interesting as this knowledge percolates into more and more fields, what domain experts do with it. I see ML as more of a bag of tricks that can be applied to many fields.
It's his art and his livelihood too, so it's also personal. These people want to steal his art and create a world full of soulless cheap muzak, while simultaneously putting him out of work.
Get 'em, Benn! I should go buy one of his albums.
I am curious if anyone read Harry Potter in bootleg form from a LLM. I mean, LLMs are the worst tools for infringing - they are approximate, expensive and slow, while copying is instant, perfect and free. You can apply the same logic for other modalities.
Moreover, who's got the time to see someone else's AI shit when they can generate their own, perfectly customized to their liking? I personally generated a song about my cat and kid. It had zero commercial value but was fun for 2-3 people to listen.
I can steal an invention without wiping the inventor's memory.
These are other kinds of stealing, which deprives the creator not of the art itself, but the other benefits of having created it.
Any musician these days that thinks there is money in music by selling songs is delusional. Sad but true.
The Flashbulb - Parkways: https://youtu.be/C6pzg7I61FI
There's new models showing up regularly. Civitai recognizes 33 image models at this point, and audio will also see multiple developments. Any successful attack on a model isn't guaranteed to apply to another one, not even yet invented. There's also a multitude of possible pre-processing methods and their combinations for any piece of media.
There's also the difficulty of attacking a system that's not well documented. Not every model out there is open source and available for deep analysis.
And it's hard to attack something that doesn't yet exist, which means countermeasures will come up only after a model was already successfully created. This is I'm sure of some academic interest, but the practical benefits seem approximately none.
Since information is trivially stored, anyone having any trouble could just download the file today and sit on it for a year or two not doing anything at all, just waiting for a new model to show up.
Seems like an awful risk to deliberately strip such markings. It's a kind of DRM, and breaking DRM is illegal in many countries.
For instance, I've seen somebody experiment with Glaze (the image AI version of this). Glaze at high levels produces visible artifacts (see middle image: https://pbs.twimg.com/media/FrbJ9ZTacAAWQQn.jpg:large ).
It seems some models ignore it and produce mostly clean images on the output (looking like the last image), while others just interpret is as a texture, the character is just wearing a funny patterned shirt. This is while the intended result is fooling the model to generate something other than the intended character.
This sounds like you’re talking about img2img generation based on a glazed image instead of training, which isn’t the intended purpose.
Because of that reality, every artist who wants to make money must either participate in it, or completely isolate themselves from it.
These models have become an incredible opportunity for giant corporations to circumvent the law. By training a model on a copyrighted work, you can launder that work into your own new work, and make money from it without sharing that money with the original artists. Obviously, this is an incredibly immoral end to copyright as we know it.
So what are we going to do about this situation? Are we really going to keep pretending that copyright can work? It wasn't even working before all the AI hype! Ever heard the words "starving artist"? Of course you have!
We need a better system than copyright. I'm convinced that no system at all (anarchy) would be a superior option at this point. If not now, then when?
Not sure if "you" refers to model developers, hosting company or end users. But let's see each one of them in turn
- model development is a cost center, there is no profit yet
- model deployment brings little profit, they make cents per million tokens
- applying the model to your own needs - that is where the benefit goes.
So my theory is that benefits follow the problem, it is in the application layer. Have a need, you can benefit from AI, don't need it, no benefit. Like Linux. You got to use it for something. And that usage, that problem - is personal. You can't sell your problems, they remain yours. It is hard to quantify how people benefit from AI, it could be for fun, for learning, professional use, or for therapy.
Most gen-AI usage is seen by one person exactly once. Think about that. It's not commercial, it's more like augmented imagination. Who's gonna pay for AI generated stuff when it is so easy to make your own.
When someone creates art, copyright says that there is a countable result we can refer to as their "work". Copyright also says that that artist has a monopoly over the distribution and sale of that work. The implication is that the way for an artist to get paid for their labor is for them to leverage the monopoly they have been granted, and negotiate a distribution scheme that involves paying them.
When an artist chooses to work outside the copyright model, that means they must predetermine part of their distribution negotiation. That might be the libertarian option (gratis distribution with no demands), or it might be the copyleft option, where the price is demanded, but also set to 0. The artist may find payment for their labor by other means, but that's challenging to do in an economy where copyright participants dominate.
I do wish, though, that he would have introduced that perspective of the situation in this particular video. Leaving it out feels like making a video about learning to swim, set in the middle of the ocean.
the [transferability] rates just drop off significantly for audio (always felt it was a similar vibe to RNN ‘vanishing gradients’)
edit — specifically mention transferability
I hope that the adversarial attacks can be easily detected and circumvented, just like other IP protection measures have been subverted successfully.
And generative AI is not a person in the first place, so I don't think the appeal to learning makes much sense here.
Do they? Please cite your studies.
US Constitution, Article I, Section 8, Clause 8:
> [the United States Congress shall have power] To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.
Then you have guilds, trade marks, potter's marks, royal warrants, etc... all "IP" protections of their days.
Would open-source license violations be possible to penalize if not for intellectual property laws?
On the off chance that you are not: IP-rights does not cover "learning from" a source. What ML does is not in any way akin to human learning in methodology. When we call it learning that's an analogy. You can not argue a legal case from analogy alone.
Yes they can because the analogy directly applies. The technical details of how a computer learns, vs a human learns doesn't really matter here, and is an irrelevant difference.
The reason why the analogy directly applies is because in both cases it is about how IP cannot really control how someone uses the IP.
Just like how IP laws cannot prevent you from listening to music while standing on your head, it also cannot prevent you from training your models on it (while also standing on your head, lets say!).
Instead, IP laws only prevents the publishing of copies of the IP.
So the point and the analogy stands.
I thought this was the most common anti-IP sentiment.
An extreme example: I do not want my code to be used in weapons guidance systems. Am I not expressing my rights as an author?
Do you think a movie's creator can dictate that I'm not allowed to use a pirated version of a movie to display for 100% profit at my 10,000 AMC movie theaters? Or a book's creator can dictate that I'm not allowed to copy their book, put my name on it, and sell it on Amazon for half-price?
If you agree that a creator can do those things, then you're already in a gray area between "they can dictate everything" and "they have no rights." At that point, you're arguing over the precise location of the line.
You'll find many more people of the opinion that authors life+70 years is too long, than you will dismissing it entirely.
You think this will make the world better? For whom? Or worse, for whom? Or you place the highest importance on having a maximalist viewpoint that simply cannot be argued with, because being unassailably right in an abstract rhetorical framing is most important to you? Or you crave the elegance of such a position, reality and utility notwithstanding? Or you feel the need to rationalize your otherwise unfounded "belief" that piracy and/or training AI on protected IP should be allowed because you like it and are involved with it yourself? Or you think ASI is going to completely transform the world tomorrow, and whether we get Culture-style luxury gay space communism or something far darker, none of this will matter so we should eat, drink, and be merry today? Or some hybrid of that and a belief that we should actively strive toward and enable such a transition, and IP law stands in its way?
What is it? I've seen some version of all of these and frankly they are all childish nonsense (usually espoused by actual children). Are you a new species?
Of course it's different, but if we look closely, it is not copying. The model itself is smaller, sometimes 1000x smaller, than the training set. Being made of billions of examples, the impact of any one of them is very small (de minimis).
If you try to replicate something closely with AI it fails. If regurgitation was a huge problem we'd see lots of lawsuits on output, but we see most suits for input (training). That means authors can't identify cases of infringement in the outputs.
They don't, what's happening here is their music is being fed to a computer program in a for-profit venture.
This anthropomorphism of LLMs is concerning. What you're actually implying here is that you believe some computer programs should be awarded the same rights as humans. You can't just skip that like it's some kind of foregone conclusion. You have to defend it. And, it's not easy.
I believe that no artist has the right to tell anyone what to do with the art they have published. It does not matter what happens inside the algorithm with that art. Whether NNs (LLMs, by definition are about language) learn like humans or not is totally irrelevant to my point.
Most people support copyrighting to some extent, and this argument against anthropomorphism does work on them.
Copyright exists to enrich the interests of the publishers of a work, not the artists they funded. A long time ago, copyright was a sufficient legal tool to bring publishers to artists' heels, but no longer. Long copyright terms and the imbalance of power between different wealthy interests allowed publishers to usurp and alienate artists' ownership over their work. And the outsized amount of commercial interest in current generative AI tools comes down to the fact that publishers believe they can use them to strip what little ownership interest authors have left. What Benn is doing is looking for new tools to bring publishers to heel.
IP is fundamentally a social contract, subject to perpetual renegotiation through action and counter-action. If you told any game publisher in the early 2000s, during the height of the Napster Wars, that they'd be proudly allowing randos on the Internet to stream video of their games being played, they'd laugh in your face. But people did it anyway, and everyone in the games biz realized it's not worth fighting people who are adding to your game. Even Nintendo, notorious IP tightwads as they are, tried scraping micropennies off the top of streamers and realized it's a fool's errand.
The statement Benn is making is pretty clear. You can either...
- Negotiate royalties for, and purchase training data from, actual artists, who will then in exchange give you high-quality training data, or,
- Spend increasing amounts of time fighting to filter an increasingly polluted information ecosystem to have a model that only sorta kinda replicates the musical landscape of the late 2010s.
A lot of us are reflexively inclined to hate on anything "copyright-shaped" because of our experiences over the past few decades. Publishers wanted to go back to the days of copyright being a legal tool of arbitrary and capricious punishment. But that doesn't mean that everything that might fall afoul of copyright law is automatically good or that generative AI companies are trying to liberate creativity. They're trying to monopolize it, just like Web 2.0 "disintermediation" was turned into "here's five websites with screenshots of the other four". That's why so much money is being poured into these companies and why a surprisingly nonzero amount of copyright reformists also have deeply negative opinions of AI.
I believe that artists see the current IP laws critically, of course they do as it directly impacts how they finance themselves and how they bargain. But I do not care how good/bad the bargain for the artist is. IP laws should be abolished regardless of what artists want.
What you are mad about is not imaginary-ness, but centralization[0]. Any critique of law should be done on this yardstick - i.e. is that law protecting the interests of the many, or the few?
In prior decades, ordinary people did not have to care about copyright law, which is why the bargain was tolerable then: copyright took money out of the pockets of the few (publishers) and gave it to the many (authors). But now that publishers figured out how to buy up all the valuable assets and you can practically enforce copyright on individuals, the system has changed to one that takes money from the people at-large and puts it in the pockets of about four publishers. On that basis alone you can argue for abolishing copyright[1] because it no longer serves the interests of the public.
Now, let's look at the business model of AI companies. AI compute is hideously expensive, and so every company involved is already a highly centralized tech monopoly. And with only a few exceptions the largest models are all jealously protected trade secrets you're only allowed to access through heavily monitored APIs. Models with published weights do exist, but they are invariably either too big to run on most hardware or lag in capability and performance. The practical fact of the matter is that most people will use AI - even open models - through inferencing servers ultimately owned by tech monopolies. This is a deeply, incredibly centralizing force.
And, unlike copyright, trade secrecy is one of those few IP laws[2] that aren't completely imaginary. It's rooted in possession, just like physical property rights are. The "no imaginary rights" argument still gets us horrible tech monopolies that can censor or marginalize you.
[0] aka "wealth inequality"
[1] What you replace it with is a separate question, but again, Benn Jordan wants socialized ownership.
[2] "Laws that allow you to dictate the conduct of your competitors"
Artists rely on some form of IP to help secure payment for their creative works. They need this payment to be able to afford their own subsistence so that they can continue to live and create.
In an alternative system, maybe you could abolish all forms of IP outright, but how will you do that under capitalism while sustaining (already impoverished) artists?
If you are against the principle of IP, you are essentially saying that an entire segment of capital should be deactivated, and effectively the only jobs remaining would be those of active service/tangible goods. In the age of digital media, basically everything is instantly and infinitely replicable, so you are effectively asking for a world in which it becomes rapidly impossible to make money off of any kind of digital good (music, literature, film, software, etc.) This has an obvious material consequence of disincentivizing creation of these works simply because if the creators need to earn wages in tangible good/service markets they have strictly less time to devote to the creation of creative works.
I also pirate every single book I read. Sometimes I buy them though.
I also pirate every single show I watch. I never buy them.
Music is a bit difficult, but I pay for Spotify, but I wouldn't mind paying for the service if Spotify had no rights to the songs and wasn't compensating the artists.
How can you possibly justify this? Do you propose professional artists/authors/musicians just shouldn't exist?
Leaning from music doesn’t mean imitating that music. People can for example learn from mistakes without imitating those mistakes.
It's been struggling since the internet became a thing. People got more content than they can consume. For any topic there are 1000 alternative sites, most of them free. Any new work competes against decades of backlog. Under this attention scarcity mode, artists devolve into enshittification because they hunt ad money, while royalties are a joke.
On the other hand, people stopped being passive consumers, we like to interact now. Online games, social networks, open source, wikipedia and scientific publication - they all run in a permissive mode. How could we do anything together if we all insisted on copyright protection?
We like to make most of our content ourselves, we don't need the old top-down model of content creation. We attach "reddit" to our searches because we value comments more than official sources. It's an interactive world where LLMs fit right in, being interactive and contextually adaptive.
Nobody owes artists the ability to make a living just for being artists.
Unless you have a service industry job or construct material goods, by your own arguments, I see no reason I should pay you either if you work on anything remotely related to intangibles like software.
I'm getting the sense that your perspective is colored more by some personal bias on what art is, what art making entails, and what art is good, rather than any sound logical principles around labor and economics, which is what any reasonable approach to IP should actually be based on.
I also never said that artists should not be allowed to do this, just that I hoped their methods were defeated. The same way I hope copyright protections are defeated.
If you are against IP, you obviously are against artists policing which model is trained on their data and which isn't. As this very obviously assigns ownership to the creation of the artist, aka IP.
It doesn't really seem like an IP thing to me - it's like making music that's intentionally hard to classify in a particular genre on purpose or something. Maybe that defeats Spotifys recommendation algorithm but that's totally unrelated to any rights to stream it or so on.
Eventually encoders/classifiers will get an 'anthropomorphic' loop of human ear capabilities to use as a prefilter and identify when sounds differ from the expected human hearing and identify the noise.
I mean this is what adversarial reinforcement learning is, humans cannot beat AI on this in the end.
(This obviously doesn't cover every single person -- some are just legitimately pro-IP.)
There's a big corollary to FOSS and gpl, but a lot of people seem to miss it. This future isn't for our culture, it's for business. And since our culture requires making money to make things adjacent to art, less culture will be made.
A lot of generated music is good enough to wind up in playlists. Lots of live bands already fake it, and lots of music venues have already replaced live music. That doesn't mean music will die, it just means it's harder to discover new music and go see a show.
I think there's some European countries that have some kind of basic income/stipend for artists. If this is the direction of the future we need to ensure something like this exists.
This type of vague posting is very popular in HN.
> X thing was originally universally leftist (it wasn't), now it's right wing only (it isn't), something something the real counterculture or grey tribe.
What's particularly funny is when that extremist politician is too stupid to understand the meaning of the song they're playing. I think the most famous example would be a certain somebody who kept playing Fortunate Son at his rallies, both him and the crowd oblivious to the fact that nearly every verse's criticism directly applies to him. A rich kid who got out of serving in Vietnam, cheated on his taxes at every opportunity, and is screaming about how great the country is while vilifying a demographic that has always been one of the strengths of our society.
I'd say that "respecting the property rights of the creative class" is very much indeed a 'left wing' perspective, too. Not just on a moral issue of "stealing their work is wrong" but also because artists very often mock or satirize right-wing personalities and platforms - punching up. Right wing artists tend to just punch down at society's weakest.
