The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target and spoof individuals - especially elderly people.
Tailored attacks impersonating text or voice messages from close contacts and family members should be fairly common, and yet they're not. Robo-calls that carry out a two way conversation convincingly impersonating bank or police officials should be everywhere. Yet the only spam-calls I ever receive are from Indian call centres or static messages using decades old synthesised voice tech.
They recently took my mom for $25k for what was a few hours of “work” over the span of two days. When I reviewed their communications and got the full story from my mom they’re in some ways laughably bad and in other ways very cunning.
Turns out it all started with a comically bad initial e-mail, pop-up, and then remote access. Then follow-up calls and text messages (Bitcoin QR code with her bank logo) and multiple people impersonating banks, various government agencies, etc. The end to end pipeline to replicate this via AI would be very complex and difficult (today).
I imagine the increased scale, additional opportunity, and reduced “payroll” that could be realized utilizing AI (given the initial level of development effort) just isn’t there. Yet.
My Mom got one of those calls where the guy said my son was jail and they needed to pay his cash bail to get him out. They never called me or him, just panicked and went to the bank to get the cash out and the bank refused to give them the money because they knew it was a scam. They started asking my parents questions and pointing out how the scam worked when they got agitated they weren't giving them the money.
They called me from the bank and I texted my son and he was completely confused since he was at work and was just fine. I thanked the people at the bank profusely for their intervention.
It was a great learning experience for my parents as well. They are way more leery about strangers calling and have already hung up on several scammers. They also filter out email messages and won't click on any links in emails or pop-ups.
The whole experience really put their radar up for this stuff now.
(1) cheaper methods work too well for now to even bother with sophisticated approaches
(2) it’s too expensive to be worth the effort
It would be interesting to plot out the cost of an integrated AI stack for this over time.
https://www.npr.org/2024/05/31/1197959218/fbi-phone-company-...
Pig butchering scams uses all forms of kidnapping, human trafficking, and slave labor
For perfect audio spoofing, lots of audio would be needed. Bear in mind there are literally millions of podcasts available [2], and billions of youtube videos. Should be trivial to grab biographic data and voice samples from a subset of them.
[1] https://github.com/neonbjb/tortoise-tts
That sounds like a great recipe for spear phishing public figures.
I'm being glib here but also if you're the type of person who gets texts from the IRS from a number you've never seen and take it at face value that you can pay off your overdue tax bill with gift cards... like, you are already the perfect victim for this sort of scam. They don't need to be good, they just need you to self identify and leap right into the trap.
At the same time, most people are more clever than one tends to expect.
(That is, on top of all the other "free drops/skins" impersonation that twitch can't seem to shut down)
Seems like a fairly simple thing to solve though, limit ages of accounts needed to be featured etc
Note that the admiration/detestation opinion might not be as socially mandatory. But probably it’s as optional as an agnostic position is tenable in a society full of theists looking for some heretics to burn on the one hand, and fanatical atheists eager to decapitate any devout on the other hand.
A friendly fast talking man in an extremely busy sounding (fake) call center asked me if I was $name_I_put_in_the_form. The voices in the background were people further down the sign up process.
I said yes, then asked how he got my number. He said I just filled out the form on the website. Then the form was replaced by a new page.
They did a good job confusing me, it was very impressive. I don't confuse easily.
That's necessary for realism.
If the deepfake were doing or saying something decent, that would be a dead giveaway.
Because you know how to do that, and it's so much easier than helping them when they get hacked.
Friend receives an email from ISP, asking her to contact them.
She searches, comes across a "customer service number" on a legit looking page, calls them up.
(Whoever she called) plays out a 30 minute charade about how she's been flagged by IRS for illegal activity and is about to have her business accounts frozen, including multiple phone transfers to "another party" (played by different people) to boost authenticity.
And during this whole time, they not once asked her for any "red flag" information (e.g. account #, SSN).
Instead, it seemed to be a shell game of extracting limited information (last 3 of your account #?), then having "unrelated" parties parrot that back as proof of their "working for the government."
I expect it would have eventually escalated into an actionable ask, but they were definitely playing the intermediate-term game.
If not for the utter moral black hole of the endeavor, I'd be kind of impressed.
Last time I did this, it took three days of texting my new friend before it was finally clear that what she really wanted more than anything was to teach me to trade cryptocurrency.
Once, I thought I had her, because she spelled D&D like: D&D, but she played it off real cool and just explained that her English isn't that great so she used translation software.
In retrospect I think that all of her probing questions about my Svirfneblin cleric were because she later intended call him up and teach him to trade cryptocurrency. I like to think he's in some scammer's database now, causing confusion. He'd like that too.
Once I understood what she was after, I explained that my problem with cryptocurrency was that it resembled money too closely and really what I'd like to do with blockchains is to do away with money in favor of something entirely different.
Her training dataset had not prepared her for this conversation, so it was quite clear when her human handler took over. They were very rude, unlike their AI pet, and tried to bully me into sharing other people's contact info, which is when I lost interest.
Even then, that won't help scams and fraud that just trick you into sending money, or direct you to install malware.
Talk to them about investment / romance scams as well. Unfortunately, most folks do these things "willingly" and get in deep.
I bought Mom a Yubikey and helped her set it up on her Google account. She has it on her keychain. She doesn't need to remember how to use it, though, since it's only needed when she buys a new computer.
For good measure, I also helped her print out backup codes (and I know where they are) and I registered my Yubikey, just in case.
Nowadays, an old backup phone might also work, but I think paper backups are better because an old, unused phone might not start.
* Bayesian poisoning https://en.wikipedia.org/wiki/Bayesian_poisoning
* Weeding out poor mark candidates https://josephsteinberg.com/why-scammers-make-spelling-and-g...
But true randomness is not useful for determining if you are who you say you are.
Always has been.
Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses.
Put them back!
It's worse. Research "Scamicry".
Big business now is so fake, such a grift, drenched in PR deception, and lacking integrity and trustworthiness, there isn't much space left between what is "legitimate" and what is a scam.
If businesses like Google or Facebook hide URLs and email addresses that's not a casual "mistake". It's because that's to their profitable advantage to do so. And they know it puts you in harms way. So yes, they're complicit in scams.
To make themselves a little more competitive businesses are always learning from scammers, meanwhile good scammers keenly learn from businesses to look more legit. Some ransomware "services" even have better customer support than billion dollar companies. And big business is certainly using the same AI tools as cybercriminals.
So a problem isn't how clever and scurrilous scammers have gotten, it's how far legitimate services have fallen so that ordinary folk struggle to know the difference.
How can we trust our own insticts for selecting what is good and wholesome from what is rotten, when there are few moral differences? The only difference resides in a digital identifier.
The last two companies I worked for insisted that customer account security was the highest priority, but as soon as I said we needed to stop hiding links to our own website behind Hubspot tracking URLs so we don't train our customers to click links that look like gobbledygook garbage, the marketing team melted down and it became clear where user account security actually fell on the priority list.
I don't think it's always malicious, though. I think most people in most companies just don't realize the risk. Like, I had to explain to my doctor's office why I'm never going to "confirm my identity" by rattling off my DOB and address at the beginning of a call when they called me. I even think of those specific data points as public information anyway and I'm not going to participate in that nonsense. It had never occurred to them that this was risky behavior.
It did make me appreciate my parish priest's method, though. Every quarter or so, he reminds people from the pulpit that he will never email them asking for gift cards or anything of the sort. If the parish needs money for something, he promises he'll ask for it right from the pulpit!
https://krebsonsecurity.com/2018/07/google-security-keys-neu...
I adore my Yubikey.
They get paid every time a scammer makes a phone call.
https://news.ycombinator.com/item?id=40942307
Imagine old people getting phone calls from frantic children. They won't know real from fake. Add tech like this to SIM forgery ..and we will devolve from a high trust society to a no trust society.
Probably the funniest thing here is that this call reached me despite the fact that I am French, living in France. And so I really wonder how they ended up calling me. I mean, chance I would understand some English speaker with an Indian accent (I like how it sounds, but it’s definitely an additional difficulty as a non-native).
I read here and there how extortion of old USA citizens by some organized Indian citizens is really a thing. To my mind the main issue at stake is that we have global level communication facilities, extremely high wealth disparities at world scale, and no compelling global social endeavor to reach an harmonization of human quality of life for everyone. I don’t mean the latter is on the official agenda of most countries out there either, but at global scale it’s obviously even worst.
With all that in mind, blaming a whole nation for the illegitimate actions of some minority in the country, all the more when the international geopolitical context itself is all but fair, is probably not going to solve any issue.
CBC the Canadian nations news service has been trying to track scammers in India
https://www.cbc.ca/news/world/tech-support-scam-india-market...
Even Jim Browning and Kit Boga on YouTube two guys who scam the scammers it seems to be 100% people based in India.
Can someone show me a modern OS that would install software by clicking a link?
a common heuristic to look out for is "badly"-written/spoken communication. the "AI vs Actual Indian" comment and nigerian prince emails stand out for most people, but they still ended up working well enough to become this wide-spread.
you just need to employ some critical thinking now for most external communication now. it is no different from some highly-motivated scammers doing it the old-fashioned way. at the end of the day, we are trying to replicate the success of some native-speaking teens (https://news.ycombinator.com/item?id=32959001).
