> We are sorry. We regret that this incident has caused worry for our partners and people. We have begun the process to identify and contact those impacted and are working closely with law enforcement and the relevant regulators. We are fully committed to maintaining your trust.
> Like, how many other deprecated third party systems were identified handling a significant portion of your customer data after this hack?
The problem with that is that you'll never know. Because you'd have to audit each and every service provider and I think only Ebay does that. And they're not exactly a paragon of virtue either.
> Who declined to allocate the necessary budget to keep systems updated?
See: prevention paradox. Until this sinks in it will happen over and over again.
> But mere words like these are absolutely meaningless in today's world. People are right to dismiss them.
Again, yes, but: they are at least attempting to use the right words. Now they need to follow them up with the right actions.
Right! But, wouldn't a more appropriate approach be to mitigate the damage from being hacked as much as possible in the first place? Perhaps this starts by simplifying bloated systems, reducing data collection to data that which is only absolutely legally necessary for KYC and financial transactions in whatever respective country(ies) the service operates in, hammer-testing databases for old tricks that seem to have been forgotten about in a landscape of hacks with ever-increasingly complexity, etc.
Maybe it's the dad in me, years of telling me son to not apologize, but to avoid the behavior that causes the problem in the first place. Bad things happen, and we all screw up from time to time, that is a fact of life, but a little forethought and consideration about the best or safest way to do a thing is a great way to shrink the blast area of any surprise bombs that go off.
But in the real world, you have words ie. commitment before actions and a conclusion.
Best of luck to them.
<rolls eyes>
I feel like most of these people will never be senior managers at a tech company because they will "go broke" trying to prevent every last mistake, instead of creating a beautiful product that customers are desperate to buy! My father once said to me as a young person: "Don't insure yourself 'to death' (bankruptcy)." To say: You need to take some risk in life as a person, especially in business. To be clear: I am not advocating that business people be lazy about computer security. Rather, there is a reasonable limit to their efforts.
You wrote:
> Everybody gets hacked, sooner or later.
- Amazon
- Meta
We also have to remember that we have collectively decided to use Windows and AD, QA tested software etc (some examples) over correct software, hardened by default settings etc.
Here, Checkout has been the victim of a crime, just as much as their impacted customers. It’s a loss for everyone involved except the perpetrators. Using words like “betrayed” as if Checkout wilfully mislead its customers, is a heavy accusation to level.
At a point, all you can do is apologise, offer compensation if possible, and plot out how you’re going to prevent it going forward.
I totally agree – You've covered the 3 most important things to do here: Apologize; make it right; sufficiently explain in detail to customers how you'll prevent recurrences.
After reading the post, I see the 1st of 3. To their credit, most companies don't get that far, so thanks, Checkout.com. Now keep going, 2 tasks left to do and be totally transparent about.
As AI tools accelerate hacking capabilities, at what point do we seriously start going after the attackers across borders and stop blaming the victimized businesses?
We solved this in the past. Let’s say you ran a brick-and-mortar business, and even though you secured your sensitive customer paperwork in a locked safe (which most probably didn’t), someone broke into the building and cracked the safe with industrial-grade drilling equipment.
You would rightly focus your ire and efforts on the perpetrators, and not say ”gahhh what an evil dumb business, you didn’t think to install a safe of at least 1 meter thick titanium to protect against industrial grade drilling!????”
If we want to have nice things going forward, the solution is going to have to involve much more aggressive cybercrime enforcement globally. If 100,000 North Koreans landed on the shores of Los Angeles and began looting en masse, the solution would not be to have everybody build medieval stone fortresses around their homes.
Indeed, an apology is bad and no apology is also bad. In fact, all things are bad. Haha! Absolutely prime.
Every additional nine of not getting hacked takes effort. Getting to 100% takes infinite effort i.e. is impossible. Trying to achieve the impossible will make you spin on the spot chasing ever more obscure solutions.
As soon as you understand a potential solution enough to implement it you also understand that it cannot achieve the impossible. If you keep insisting on achieving the impossible you have to abandon this potential solution and pin your hope on something you don't understand yet. And so the cycle repeats.
It is good to hold people accountable but only demand the impossible from those you want to go crazy.
In terms of "downplaying" it seems like they are pretty concrete in sharing the blast radius. If less than 25% of users were affected, how else should they phrase this? They do say that this was data used for onboarding merchants that was on a system that was used in the past and is no longer used.
I am as annoyed by companies sugar coating responses, but here the response sounds refreshingly concrete and more genuine than most.
In my country, this debate is being held WRT the atrocities my country committed in its (former) colonies, and towards enslaved humans¹. Our king and prime minister never truly "apologized". Because, I kid you not, the government fears that this opens up possibilities for financial reparation or compensation and the government doesn't want to pay this. They basically searched for the words that sound as close to apologies as possible, but aren't words that require one to act on the apologies.
¹ I'm talking about The Netherlands. Where such atrocities were committed as close as one and a half generations ago still (1949) (https://www.maastrichtuniversity.nl/blog/2022/10/how-do-dutc...) but mostly during what is still called "The Golden Age".
That preceding line makes it, to me, a real apology. They admit fault.
Because these things take time, while you need to disclose that something happened as fast as possible to your customers (in the EU, you are mandated by the GDPR, for instance).
We are fully committed to rebuilding your trust.
"We will pay $500,000 to anyone who can provide information leading to the arrest and conviction of the perpetrators. If the perpetrators can be clearly identified but are not in a country which extradites to or from the United States, we will pay $500,000 for their heads."
Your recourse within US law is to petition the government to do something about it. Negotiate extradition. Go to war. Etc.
Hey donnie, these guys are "Venezuelan drug trafficers"
One places the company at the center as the important point of reference, avoiding some responsibility. The other places the customer at the center, taking responsibility.
The problem can not be helped by research research against cybercrime. Proper practices for protections are well established and known, they just need to be implemented.
The amount donated should've rather be invested into better protections / hiring a person responsible in the company.
(Context: The hack happened on a not properly decomissioned legacy system.)
I see it more as a middle finger to the perps: “look, we can afford to pay, here, see us pay that amount elsewhere, but you aren't getting it”. It isn't signalling virtue as much as it is signalling “fuck you and your ransom demands” in the hope that this will mark them as not an easy target for that sort of thing in future.
For customers it signals sincerity and may help dampen outrage in their follow up dealings.
The point here is that this is an expensive virtue signal. Although, it would be more effective if we knew how expensive it was.
It's also a term you can use against political opponents because it's much easier to speak well than to actually do good.
Refusing to negociate with criminals and help fund security seems like the proper long-term reaction for everyone.
Making it illegal to pay ransom is likely a much easier to implement and more effective solution.
And this isn’t virtue signaling - they literally did the virtuous thing that is better for society at the expense of their bottom line. That is just virtue.
Yes there are negative externalities in funding ransomware operations, not paying is still much more likely to hurt your customers than paying.
Besides, if they were genuinely interested in positive externalities they would be spending the money lobbying for a ransomware payments ban and not donating to universities.
You send them the payment, they tell you they deleted the data, but they also sell the data to 10 other customers over the dark-web.
Why would you ever trust people who are inherently trustworthy and who are trying to screw you? While also encouraging further ransomware crimes in the future.
If you don’t pay, the odds they will publish your data are closer to 100%. If you do pay, the odds have historically been much closer to 0% than 100%
You aren’t paying to be sure, but to improve your chances.
I would argue that it is being used all over the media to complain about anyone showing any signs of not being purely individualistic, as if individualism is the only true thing people actually honestly feel. This is obviously incorrect, empathy, professionalism, a desire for a sense of purpose, are all things that people objectively feel in the real world, everyday, everywhere.
I would argue that the expression "virtue signaling" is used systematically in individualistic right wing media by the right about anyone who say, for example, that they care about minorities or less fortunate people or to take action to support them, as if it was false. I would argue that this is harmful.
People do care a good fraction of the time, and they should be recognized for their positive actions, and encouraged. I would argue that we should definitely strive for a culture where individualism is not seen as the only true emotion that people can feel.
So, knowing the negative political and philosophical baggage, I would not use that expression, especially if you don't have actual proof that they don't care about security, professionalism, etc.
Endpoint security is a well known open problem for what no sufficient practices and protections exist.
In french we call that a "pied de nez". "Turning the table" / "Poetic justice" / "Adding insult to injury" would all be more correct than "virtue signalling".
If there was no attacker and the company gave half a mil out of nowhere to a security company (or a charity) and boasted publicly about it, that would be virtue signalling.
But refusing to pay the ransom and giving the exact same amount to security researchers is just a big, giant, middle finger.
And a middle finger ain't no virtue signalling.
Or just properly follow best-practise, and their own procedures, internally.⁰
That was the failing here, which in an unusual act of honesty they are taking responsibility for in this matter.
--------
[0] That might be considered paying for security, indirectly, as it means having the resources available to make sure these things are done, and tracked so it can be proven they are done making slips difficult to happen and easy to track & hopefully rectify when they inevitably still do.
- timely response
- initial disclosure by company and not third party
- actual expression of shame and remorse
- a decent explanation of target/scope
i could imagine being cyclical about the statement, but look at other companies who have gotten breached in the past. very few of them do well on all points
For that level of breach their response seems about right to me, especially waving the money in ShinyHunters' face before giving it away to their enemies.
Timely in what way? Seems they didn't discover the hack themselves, didn't discover it until the hackers themselves reached out last week, and today we're seeing them acknowledging it. I'm not sure anything here could be described as "timely".
From customer perspective “in an effort to reduce the likelihood of this data becoming widely available, we’ve paid the ransom” is probably better, even if some people will not like it.
Also to really be transparent it’d be good to post a detailed postmortem along with audit results detailing other problems they (most likely) discovered.
It’s a sliding scale, where payment firmly pushes you in the more comfortable direction.
Also, the uncomfortable truth is that ransomware payments are very common. Not paying will make essentially no difference, the business would probably still be incredibly lucrative even if payment rates dropped to 5% of what they are now.
If there was global co-operation to outlaw ransom payments, that’d be great. Until then, individual companies refusing to pay is largely pointless.
The extortionist knows they cannot prove they destroyed the data, so they will eventually sell it anyway.
They will maybe hold off for a bit to prove their "reputation" or "legitimacy". Just don't pay.
The ransom payments tend to be so big anyway that selling the data and associated reputational damage is most likely not worth the hassle.
Basic game theory shows that the best course of action for any ransomware group with multiple victims is to act honestly. You can never be sure, but the incentives are there and they’re pretty obvious.
The big groups are making in the neighbourhood of $billions, earning extra millions by sabotaging their main source of revenue seems ridiculous.
Until there is legislation to stop these payments, there will be countless situations where paying is simply the best option.
And selling the data from companies like Checkout.com is generally still worth a decent amount, even if nowhere close to the bigger ransom payments.
It’s not great, but it’s the least shitty option.
The cost of an attack like this is in the thousands of dollars at most, the ransom payments tend to be in the millions. The economics of not paying just don’t add up in the current situation.
The level of persistence these guys went through to phish at scale is astounding—which is how they gained most of their access. They’d otherwise look up API endpoints on GitHub and see if there were any leaked keys (he wasn’t fond of GitHub's automated scanner).
https://www.justice.gov/usao-wdwa/pr/member-notorious-intern...
They themselves are likely to some extent the victims of social engineering as well. After all who benefits from creating exploits for online games and getting children to become script kiddies? Its easier (and probably safer) to make money off of cyber crime if your role isn't committing the crimes yourself. It isn't illegal to create premium software that could in theory be use for crime if you don't market it that way.
To gift to a 529 regardless of the financial institution, you go to some random ugift529.com site and put in a code plus all your financial info. This is considered the gold standard.
To get a payout from a class-action lawsuit that leaked your data, you must go to some other random site (usually some random domain name loosely related to the settlement recently registered by kroll) and enter basically more PII than was leaked in the first place.
To pay your fed taxes with a credit card, you must verify your identity with some 3rd party site, then go to yet another 3rd party site to enter your CC info.
This is insane and forces/trains people to perform actions that in many other scenarios lead to a phishing attack.
Yes, we've (the software industry) been training people to practice poor OpSec for a very long time, so it's not surprising at all that corporate cybersecurity training is largely ineffective. We violate our own rules all the time
But he shrugged it off.
I bet there are quite a few shops online that may sell gift cards that are used in money laundering schemes. Bonus points if they accept bitcoin.
But those are all quite implicitly used by cybercrime. I can imagine there are quite a few tools at their disposal that are much more explicit.
I was involved in probably 15 operations with them while I was there. They would usually get C&C within six hours, every single time it was phishing lol.
But if we're holding users accountable because 1 out of every 100 clicks a link in a phishing email like clockwork, we're bad at both statistics and security.
Who is making money off of selling premium software, that's not marketed as for cybercrime, to non-governmental attackers? Wouldn't the attackers just pirate it?
> Wouldn't the attackers just pirate it?
Sometimes the software is SaaS (yes, even crimeware is SaaS now). In other cases, it has heavy DRM. Besides that, attackers often want regular updates to avoid things like antivirus detections.
Do you mean they thought the scanner was effective and weren't fond of it because it disrupted their business? Or do you mean they had a low opinion of the scanner because it was ineffective?
did you have bulletproof hosting and they caught you through other means like going after your payment providers or you made opsec mistakes or how exactly?
was it a website like Sportsurge where it simply linked to streams or did it actually host the streams?
explain
I think the answer is ok but the "third-party" bit reads like trying to deflect part of the blame on the cloud storage provider.
Often times it would have been easier to rebuild the whole project over trying to upgrade 5-6 year old dependencies.
Ultimately the companies do not care about these kinda incidents. They say sorry, everyone laughs at them for a week and then after its business as usual, with that one thing fixed and still rolling legacy stuff for everything else.
All work created by a company decays, it's legacy code within months.
Sure buddy, sure
They failed so damn bad and it's hilariously bad and I feel awful for the somewhat competent coworker who was stuck on that team and dealt with how awful it was.
Then we fired most of that team like 3 times because of how value negative they have been.
Then my coworker and I rebuilt it in java in 2 months. It is 100x faster, has almost no bugs, accidentally avoided tons of data management bugs that plague the python version (because java can't have those problems the way we wrote it) and I built us tooling to achieve bug for bug compatibility (using trivial to patch out helpers), and it is trivially scalable but doesn't need to because it's so much faster and uses way less memory.
If the people in charge of a project are fucking incompetent yeah nothing good will ever happen, but if you have even semi-competent people under reasonable management (neither of us are even close to rockstars) and the system you are trying to rewrite has obvious known flaws, plenty of time you will build a better system.
I can imagine that in a team that might be harder, but these are glorified todo apps. I am well aware that complete rebuilds rarely work out.
To me it seems most likely that this is data collected during the KYC process during onboarding, meaning company documents, director passport or ID card scans, those kind of things. So the risk here for at least a few more years until all identity documents have expired is identity theft possibilities (e.g. fraudsters registering their company with another PSP using the stolen documents and then processing fraudulent payments until they get shut down, or signing up for bank accounts using their info and tax id).
Essentially nobody checks the validity of document numbers, there’s rarely any automated mechanism to do this. You could just photoshop the expiry dates on the documents and use them for years and years, even if document designs changed you could just transplant the info from the old document into a new template.
So no, documents expiring does mostly nothing to alleviate identity theft risks in most of the world.
And anyway, targeted phishing attacks are of much much higher severity than identity theft. From this data you can probably gather everything you’d need to perform rather high quality phishing attacks against the bank accounts of checkout.com clients, easily causing tens or hundreds of millions of losses that would never be recovered.
If you read between the lines of the verbiage here, it looks like a general archived dropbox of stuff like PDF documents which the onboarding team used.
Since GDPR etc, items like passports, driving license data etc, has been kept in far more secure areas that low-level staff (e.g. people doing merchant onboarding) won't have easy access to.
I could be wrong but I would be fairly surprised if JPGs of passports were kept alongside docx files of merchant onboarding questionnaires.
How do you qualify this statement? Did you mean “should never”? Even then, you’re likely overstating things. Nothing prevents co-locating KYC/KYB information. On the contrary, most businesses conducting KYB are required to conduct UBO and they’re trained to combine them both. Register as a director/officer with any FSI in North America and you’ll see.
Why would merchants fill out docx files? They would submit an online form with their business, director and UBO details, that data would be stored in the Checkout.com merchants database, and any supporting documents like passport scans would be stored in a cloud storage system, just like the one that got hacked.
If it was just some internal PDFs used by the onboarding team, probably they wouldn't make such a big announcement.
Every country you operate in has different rules and regulations and you have to integrate with many third party systems as well as governmental entities etc, and sometimes you have to do really really technically backwards things.
Some integrations I remember were stuff like cron jobs sending CSV files via FTP which were automatically picked up.
The sheer amount of effectively useless bingo sheets with highly detailed business (and process) information boggles the mind.
Some time ago I alluded to existence and proliferation of these questionnaires in another context: https://bostik.iki.fi/aivoituksia/random/crowdstrike-outage-...
(If not, why not?)
(Imho, it would make sense if only the state can pay ransoms)
Why not? Legislators haven’t caught up yet, and banning ransom payments would likely cause some very uncomfortable situations.
This of course raises some pretty uncomfortable questions, should ransom payments in kidnapping cases be banned too? That would presumably cost actual human lives.
A more pressing issue is that banning ransom payments might dissuade ransomware, but wouldn’t affect the main problem of financially motivated hacking. The costs of these attacks are so low that a ransomware payments ban would probably not have stopped checkout.com from being hacked and having their customer data stolen, the criminals will still do crime even if they have to do slightly different crime that pays less.
The group responsible in this case was just selling data stolen from their victims for a long time before they pivoted to much more profitable ransom operations.
Instead, you would pay (exhorbitant) consulting fees to a foreign-based "offensive security" entity, and most of the time get some sort of security report that says if you'd simply plug this and that holes, your systems would now be reasonably safe.
Yes, that's why cryptocurrencies are a gift from heaven for these hacker groups.
Therefore, even if paying ransom money (somehow) must be legal, maybe it should be illegal to use crypto for it. You don't want to make it too easy to run this type of criminal business.
You go on some Russian crime forum and find a plenty of people offering to process bank transfers like these for some percentage of the money. As these particular payments would be somewhat consensual, you wouldn’t even have to worry about the funds getting frozen on the way.
Lots of US based incident response companies handling ransomware payments, this isn’t the domain of some sketchy foreign offsec joints.
US indicts two rogue cybersecurity employees for ransomware attacks
IMO, these aren’t safe to use anymore.
Probably someone was phished and they still had access to an old shared drive which still had this data. Total guess but reading between the lines it could be something like this.
Reading between the lines reveals the severity they're obfuscating, with contradictions:
> This incident has not impacted our payment processing platform. The threat actors do not have, and never had, access to merchant funds or card numbers.
> The system was used for internal operational documents and merchant onboarding materials at that time.
> We have begun the process to identify and contact those impacted and are working closely with law enforcement and the relevant regulators
They stress that "merchant funds or card numbers" weren't accessed, yet acknowledge contacting "impacted" users, this begs the question: how can users be meaningfully "impacted" by mere onboarding paperwork?
I can't quite work out who they donated to - it seems there are a number of Oxford Uni cybersec/infosec units. Any idea which one?
"Cyber Security Oxford is a community of researchers and experts working under the umbrella of the University of Oxford’s Academic Centre of Excellence in Cyber Security Research (ACE-CSR)."
I don't think it's https://www.infosec.ox.ac.uk/
There's also this AI security research lab, https://lasr.plexal.com/
It looks like Oxford are quite busy in this space.
This sort of data is generally treated very differently to the actual PANs and payment information (which are highly encrypted using HSMs).
So it's obviously shitty to get hacked, but if it was just KYB (or KYC) type information, it's not harming any individuals. A lot of KYB information is public (depending on country).
Fair play on them for being open about this.
> The system was used for internal operational documents and merchant onboarding materials at that time.
Ah so just all of your KYC for founders, key personnel, and the corporation to impersonate business accounts
> We estimate that this would affect less than 25% of our current merchant base.
Yikes, this affects 25% of their current merchant base.
This submission's edited title reads like the "target headline" from The Office (US):
> Scranton Area Paper Company - Dunder Mifflin - Apologizes - to Valued Client - Some Companies - Still Know - How - Business - is - Done
> Jimmy, where did the cookies go?
> Something that was on the counter is gone! I don't know how! It might not even be my fault! But I'm sorry!
What kind of an apology is that? It's not. It's marketing for the public while they contact the "less than 25% of [their] current merchant base" whose (presumably sensitive) information was somehow in "internal operational documents".
Oh but also took some of what they charge their customers and gave that (undisclosed?) sum away to a university. They must be really sorry.
In most cases they can get away with "We are sorry" and "Trust me, bro" attitude.