Built-in checks prevent processing of inappropriate content, ensuring legal and ethical use."
I see it claims to not process content with nudity, but all of the examples on the website demo impersonation of famous people, including at least one politician (JD Vance). I'm struggling to understand what the authors consider 'ethical' deepfaking? What is the intended 'ethical' use case here? Of all the things you can build with AI, why this?
ffmpeg -i porn.mp4 -vf "crop=crop_w:crop_h:coord_x:coord_y" "definitely not porn.mp4"
*faceswap "definitely not porn.mp4*
ffmpeg -i porn.mp4 -i "swapped definitely not porn.mp4" -filter_complex "[0][1]overlay=coord_x:coord_y" -c:a copy "deepfake porn.mp4"
FT had a fantastic podcast on the porn industry and the guy behind Mindgeek. Like many stories about multinational entities, you constantly hear the usual refrains - noone can regulate this, the entities keep changing their name and face, there is no accountability, etc. But when Visa and Mastercard threaten to pull their payments, the companies have to listen.
Visa and mastercard are the de facto regulators of porn today, and mostly do so to prevent nonconsentual and extreme fetish stuff from being displayed on mainstream platform.
From what I gathered from the podcast, they're not super keen on being the regulator - but it's a dirty job and somebody has to do it.
Cryptocurrency and the like may offer a way out of that problem by allowing direct purchases, but only for companies willing to deal with the support burden of making everything nonrefundable.
That can be asked of 90% of what's come out of the latest AI bubble so far.
Like a lot of technology, AI has so much potential for good. And we use it for things like games that simulate killing one another, or making fake news web sites, or pushing people to riot over lies, or making 12-year-olds addicted to apps, or eliminating the jobs of people who need those jobs the most, or, yes, pornography.
We can do better.
Like what?
# process image to videos
if modules.globals.nsfw == False:
from modules.predicter import predict_video
if predict_video(modules.globals.target_path):
destroy()If the technology is actually made widely available that just reveals that the Pandora box was actually already open
If you're unwilling to recognize the benefits of something, it becomes easier to dismiss your argument. Instead, the truth is balancing trade-offs and benefits. Certainly there is a clear and harmful downside to this tech. But there are benefits. It does save a lot of money for the entertainment industry when you need to edit or do retakes. The most famous example might be superman[0].
The issue is that when the downsides get easy to dismiss, it becomes easy to get lost in the upsides. It'll get worse because few people consider themselves unethical. We're all engineers and we all have fallen for this trap in some way or another. But we also need to remember that the road to hell isn't paved with malicious intent...
I too have ethical concerns. There are upsides though. It is a powerful tool for image and video editing (for swapping, you still need a generator on the backbone)[0]. It is a powerful tool for compression and upsampling (your generative model __is__ a compression of (a subset of) human faces, so you don't need to transmit the same data across the wire). It is easy to focus on the upsides and see the benefits. It is easy to not spend as much time and creative thinking directed at malicious usages (you're not intending to use or develop something for malicious acts, right?!). But there's two ways to determine malicious usages of a technology: 2) you emulate the thinking of a malicious actor, contemplating how they would use your tool, and 2) time.
But I also do think application matters. I think this can get hairy when you get nuanced. Are all deepfakes that are done without consent of the person being impersonated unethical? I think at face (pun intended) value, this looks like an unambiguous no. But what about parody like Sassy Justice?[1]. Intent here is not to deceive, and the deep fakes add to the absurdity of the characters, and thus the messages. Satire and parody itself doesn't work unless mimicry exists[2]. Certainly these comedic avenues are critical tools in democracy, challenging authority, and challenging mass logic failures[3] (which often happens specifically due to oversimplification and not thinking about the details or abuse).
I want to make these points because I think things are post hoc far easier to dismiss than a priori. We're all argumentative nerds, and I think despite the fact that we constantly make this mistake, we can all recognize that cornering someone doesn't typically yield in surrender, but them fighting back harder (why you never win an argument on the internet, despite having all the facts and being correct). And since we're mostly builders (of something) here, we all need to take much more care. *The simpler you rationalize something to be post hoc, the more difficult it will be to identify a priori.*
Even at the time, I had reservations when building what I made. But one thing I've found exceptionally difficult in ML research is that it is hard to convince the community that data is data. The structure of data may be different and that may mean we need more nuance in certain areas than others (which is exciting, as that's more research!), but at the end of it, data is data. But we get trapped in our common datasets to evaluate[4] and more and more, our research needs to be indistinguishable from a product (or at least a MVP). If we can make progress by moving away from Lena, I think we can make progress by moving away from faces AND by being more nuanced.
I don't regret building what I built, but I do wish there was equal weighting to the part of my voice that speaks about nuance and care (it is specifically that voice that led to my successful outcomes too). The world is messy and chaotic. We (almost) all want to clean it up and make it better. But because of how far we've advanced, we need to recognize that doing good (or more good than harm) is becoming harder and harder. Because as you advance in any topic, the details matter more and more. We are biased towards simplicity and biased towards thinking we are doing only good[5], and we need to fight this part of ourselves. I think it is important to remember that a lie can be infinitely simple (most conspiracies are indistinguishable from "wizards did it"), but accuracy of a truth is bounded by complexity (and real truth, if such a thing exists, has extreme or infinite complexity).
With that said, one of my greatest fears of AI, and what I think presents the largest danger, is that we outsource our thinking to these machines (especially doing so before they can actually think[6]). That is outsourcing one of the key ingredients into what defines us as humans. In the same way here, I think it is easy to get lost in the upsides and benefits. To build with the greatest intentions! But above all, we cannot outsource our humanity.
Ethics is a challenging subject and it often doesn't help that we only get formal education through gen ed classes. But if you're in STEM, it is essential that you are also a philosopher, studying your meta topic. Don't need to publish there, but do think about. Even just over beers with your friends. Remember, it's not about being right -- such a thing doesn't exist --, it is about being less wrong[7]
[0] https://www.youtube.com/watch?v=2nxanN85O84
[1] https://www.youtube.com/@SassyJustice
[2] https://www.supremecourt.gov/DocketPDF/22/22-293/242292/2022...
[3] https://www.gutenberg.org/files/1080/1080-h/1080-h.htm
[4] I do think face data can be helpful when evaluating models as our brains are quite adept at recognizing faces and even small imperfections. But this should make it all that much clearer that evaluation is __very__ hard.
[5] I think it is better to frame tech (and science) like a coin. It has value. The good or evil question is based on how the coin is spent. Even more so how the same type of coins are predominantly spent. Both matter and the topic is coupled, but we also need to distinguish the variables.
[6] Please don't nerdsplain to me how GPTs "reason". I've read the papers you're about to reply with. I recognize that others disagree, but I am a researcher in this field and my view isn't even an uncommon one. I'm happy to discuss, but telling me I'm wrong will go nowhere.
[7] https://hermiene.net/essays-trans/relativity_of_wrong.html
Upvoted.
On the flip side, the ability to deep-fake a face in real time on a video call is now accessible to pretty much every script kiddie out there.
In other words, you can no longer trust what your eyes see on video calls.
We live in interesting times.
The feel of counterfeit bills, the color someone choose to wear, the sound that doesn't quite fit.
I think deep-fakes are mostly a danger to people without a lot of source material for their minds to compare against. You could trick me into believing I was taking with Elon, but not my son.
Others here seem to be interpreting the statement as, “I could be tricked because I am an older person, while a younger person would not be so easily deceived.”
And yet there have been several recent studies that show the younger someone is, the more likely they are to be scammed online.
> In 2021, Gen Xers, Millennials, and Gen Z young adults (ages 18-59) were 34% more likely than older adults (ages 60 and over) to report losing money to fraud,[1] and some types of fraud stood out. Younger adults reported losses to online shopping fraud – which often started with an ad on social media – far more often than any other fraud type, and most said they simply did not get the items they ordered.[2] Younger adults were over four times more likely than older adults to report a loss on an investment scam.[3] Most of these were bogus cryptocurrency investment opportunities.
https://www.ftc.gov/news-events/data-visualizations/data-spo...
I think you are misreading the post. Pretty sure they meant
you could trick me into believing I was talking with Elon, but you could not trick me into believing I was talking with my son
To which I agree personally, though I don't know how universal this is.
This was my thought about people in general, until more and more stories came out about the phone scammers pretending to be a grandson/daughter/family-member in need of a wire transfer/money to get them out of trouble. I still find it difficult to believe those are real scams that seem to work. This will probably escalate those even more with more people going to video calls. The panic of a loved one/child will not create a calm enough mind for thinking "hey maybe this is a deepfake" in most parents, atleast from my observations.
You think there are no counterfeit bills that feel exactly the same as the real thing? Pretty sure you're wrong.
https://arstechnica.com/tech-policy/2024/07/us-security-firm...
Movies and TV:
- As an alternative to motion capture for animation
- As an alternative to existing de-aging CGI when you want to flash back to a younger version of a character (especially for cases where newer sequels are being made for much older movies)
- As an easy way to get some additional footage if an actor no longer looks the part
- Conduct job interviews were interviewees faces are mapped to the faces of a few pre-defined images, to reduce a major source of implicit bias in interviewing
- Get some footage of yourself when you're looking your best, with great lighting, and use that rather than being off-camera if you're joining a meeting when you don't look great
- Create virtual spokespeople to represent your company in marketing, and allow that person to be played by different actors
- An alternative to blurred or blocked out faces for people giving interviews or whistle blowing
- Allow people to testify in court (virtually) without revealing their identity and risking retaliationThis all ends extremely badly.
If not someone you know then just a random stranger?
Not that I can think of a better use case but it is telling if this is the best we can do.
It's even more amusing when one sees glitches like eyes appearing in front of a strand of hair...
- streamer goofing around.
- Perhaps something like this could be used to map your facial expressions onto video game characters in real-time.
- could take tictok style social media to the next level of absurdity. make me into a meme. Ghana says goodbye etc.
In the overwhelmingly large majority of cases it'll be used for porn, scams and maybe 5% of the time it'll crop up in a meme of Donald Trump and Joe Biden singing memey Chinese songs.
Can't wait to hear all the stories of grandma losing her life savings cause a scammer can use Timmy's hyper realistic deep faked face at a click. AI truly is the future
So, quite literally, a "password" in its original pre-internet meaning.
This won't work.
I've resorted to using OTP apps with family and coworkers.
That seems extremely clunky and impersonal and I couldn’t imagine anyone in my family willingly agreeing to do it.
If somebody deepfakes someone in your network, it will most likely be for scamming / monetary purposes. (I don't care much about being pranked)
So you don't need to verify identity for a casual chat with a friend or family member.
On the other hand, here is a real situation that happened 3 days ago.
I get an instant message from my COO Nirina: the bank website says she needs a new auth code to be able to enter new wire payments for me to sign and pay the team. She provides me with the link the bank has referred her to ; a URL the bank says I need to access (with my higher privileges) to generate a new auth code for her.
First of all,
- Whoever the source is, there is no way I'm clicking that link. If I'm going to do anything about bank access codes, I'll login by manually typing the URL and I'll navigate the website myself.
Second,
- I would trust Nirina with my life. But should I trust that the person sending me that link is Nirina? I mean, if you got hold of her laptop on a business day, it wouldn't take you more than a couple minutes of snooping around to figure out that she's messaging back and forth with the company's CEO and hereby lies opportunity.
So how do I check it's her?
- I could call her (and fall for a deepfake)
- I could ask for something "only she would know" (and risk the chance that the answer can be guessed, or can be found somewhere in our several-years-long messaging history or on our 10-year-old online Drive)
- or... I could say "what's the code that proves this is from Nirina?" and wait for her to send me an OTP code matching mine.
Obviously I am not doing that every time someone from my team sends me a message.
Oh and by the way, I put this is place the day after someone tried to get access to our Meta Business account by impersonating someone on a Zoom call with me... it's not paranoia if they're after you!
you can't plan on everybody never having something come up.
I wonder how politics can be transacted in such an environment. Old-timey first-past-the-post might be the optimal solution if you can't trust anything from out of earshot.
Codes and seals predate computers (by quite a bit.)
The process of politicians debating and getting elected is going to have to be much more local. Just look at how easy it is to spread misinformation now.
> The process of politicians debating and getting elected is going to have to be much more local.
I'm no expert on government but that seems like it would be a good thing. IMO the best but most expensive form of government is Quaker-style Consensus Decision Making:
I've notice I've steadily become more ashamed to be associated with tech. I'm still processing how to react to this and what to choose to work on in response
Am I in a bubble? Do you share similar feelings or are yours quite different? I am very curious
Are you actively contributing to these areas you feel ashamed about? If not, you shouldn't really feel ashamed about what other people chose to work on, even if both of you work "in tech".
I'm sure not all people working on medical research agrees with what all other researchers are working on, but you cannot really control what others are working on, so why feel ashamed over what others are working on?
Why not? Someone who builds boxes that hold bombs can be ashamed of being in the munitions industry, even if they don't make the actual bombs.
Let me separate my face, body and words and craft the experience.
I want a model which is made photoreal with my own image, so it can be given a voice in real time with my words, but a filtered version of my facial expression and pose.
So how I look and act is essentially scriptable.
https://www.technologyreview.com/2023/09/19/1079832/chinese-...
https://www.technologyreview.com/2024/05/07/1092116/deepfake...
It’s already a requirement sometimes to take a video of your face from multiple angles using your phone - some identity verification service forced me to do it. I imagine that stuff like this will evolve to check for hardware attestations more, or use info from depth/lidar sensors to verify video and other sensor data align.
Only if by "grandma" you mean "Millennial" and by "grandpa" you mean "Gen Z." Your ageism doesn't jibe with reality:
https://www.ftc.gov/news-events/data-visualizations/data-spo...
Grandparents absolutely fall for some scams at disproportionate rates. And are less likely to be able to recover. (A 19 year old who loses everything has many more productive years to recover than a 72 year old.)
Also, humorously, millennials are starting to become grandma and grandpa. Elder millennials are in their mid 40s. It's young, but not impossible for them to be grandparents now.
https://github.com/hacksider/Deep-Live-Cam
Took me multiple minutes to find too.
I wonder, is there a universe where maybe cameras are updated to add some sort of digital signature to videos/photos to indicate they are real and haven't been tampered with? Is that feasible/possible? I'm not skilled with cryptography stuff to know, but if we can digital sign documents with some amount of faith...
I've heard folks mention trying to tag AI photos/videos, but it seems like tagging non-AI photos/videos is more feasible?
Relevant source https://github.com/hacksider/Deep-Live-Cam/blob/main/modules...
Question is, is it stoppable? Doubt anyone thinks it can be stopped unless you get into fascistic/communistic/authoritarian tactics of arresting people for just using it at all.
Like when they were brainstorming this as a product, what was the persona/vertical they were targeting?
A software engineer says to himself, if only I could keep these guns from jumping off the table and shooting people.
However, this really nails that pretty dead itself. Wonder if I can:
- Sit at home in pajamas.
- Change my face to Sec. of Def. Lloyd Austin.
- Put myself in a nice suit from TV
- Call the White House with autotune voice pretending to be going in for surgery yet again because of life threatening complications
- Send the entire military into conniptions (maybe mention some dangerous news I need to warn them about before the emergency rush surgery starts)
Edit: This [4] might be an Animate / Outfit anyone image... It's difficult to tell. Even with huge amounts of experience, the quality has become too elevated, too quick to check 1000's of depressing murder images for fakes because it might be a BS heart string story. All stories on the WWW are now, "that might be fake, unless I can personally check." Al-arabiya upvoted casinos and lotteries for muslims recently. [5] "they all might be fake."
[1] https://www.microsoft.com/en-us/research/project/vasa-1/
[2] https://humanaigc.github.io/emote-portrait-alive/
[3] https://humanaigc.github.io/animate-anyone/
[4] https://www.reuters.com/resizer/v2/https%3A%2F%2Fcloudfront-...
[5] https://english.alarabiya.net/News/gulf/2024/07/29/uae-grant...
And I don’t say this with excitement.
And this is the worst quality it will ever be. In the future it will be impossible to know who we are talking with online.
It is already ez to run text troll AIs on normal workstations... so...
AI will kill the Internet we know today and the new one im guessing you will have to have a Internet license attached to your identity which is backed by your internet reputation which you always want to keep it high for veracity/validity! You can still post anonymously but it wont hold as much weight compared to you posting using your verified Internet identity. This idea of mine i posted good number of times here and it gets downvoted but with the IRS in bed with ID.Me (elon musk is involved with them in some capacity) you can see what i mention with ID.me and the IRS being a small step in this direction. Otherwise no one uses the Internet (zero trust of it) .. it dies and we go back to reading books and meeting in person (doesnt sound all that bad yet ive never read a book before).
But maybe no, it wouldn't. Maybe it'd be deeply disconcerting. We have very strong norms around honesty as a society, and maybe crossing them in video just for a joke is comparably crass to giving somebody a fake winning lottery ticket.
