User signs up to try circleci for a private project of theirs. Grants read access to their private repos via github oauth
User also has many other private repos (company they work for, open source projects, forks, etc)
Could they have used the stored github credentials from circleci to clone every private repo in full the user had access to?
https://help.github.com/articles/managing-deploy-keys#deploy...
If the attacker has a bunch of tokens, could they have bulk downloaded source code before the oAuth stuff was revoked by Circle?
https://github.com/blog/1270-easier-builds-and-deployments-u...
Info have a Circle-CI deploy key per private repository (which I will revoke).
Identified - We have notified all users and recommended appropriate action:
"We are contacting you to inform you of an ongoing security incident affecting CircleCI customers, as a result of the compromise of our database (http://security.mongohq.com/notice).
We are taking aggressive action to protect your data and systems. At this time, we have suspended all CircleCI account access, and all builds & workers have been suspended. In addition we have revoked all access to Heroku and GitHub OAuth tokens and API keys uploaded to CircleCI.
We do not yet know the scope and impact of the intrusion and are therefore treating this event as if all data has been compromised. While we have no evidence that these credentials have been compromised, we urge you to revoke the following:
SSH keys that were uploaded to CircleCI API tokens added to CircleCI as environment variables secrets stored in GitHub repositories We will be keeping you informed at http://status.circleci.com and will update you at regular intervals as the situation progresses.
We deeply regret that this has happened and are working around the clock to resolve this incident and protect your data and systems." 22:25 PDT Update - We are still investigating the issue. The full team is engaged and we are working with upstream providers to diagnose and respond to the issue, and protect all of our users. We will keep you informed. 21:17 PDT Update - We are currently investigating an ongoing issue with our database service. At this time, we have suspended all account access to our service. All builds & workers have been suspended. We will have another update in the next 30 minutes. 20:20 PDT Investigating - "CircleCI is experiencing technical problems. We're investigating and should have an update within 30 minutes." 19:30 PDT
It's separate from your actual codebase having any private keys, which I agree would be a Really Bad Thing.
Does this mean our code is compromised too?
If they have your keys then they can get at your code.
Talk about domino effect.
Guess: this + mongohq was a targeted attack, aimed at a single customer of theirs?
I know that's not exactly proper but... at this point, the hackers are way ahead of the users of sites that happen to depend on MongoHQ.
Though, the assumption I'm making right now is that the hackers know exactly what sites they've compromised. The longer they have this information in advance of MongoHQ's customers' customers, they more damage they can do. That sucks terribly for those people (potentially)
All this said, a serious lesson about PaaS/SaaS/DBaaS--whatever you want to call it--has been learned today.
[edit] - I'm reminded here of the Epsilon breach. Epsilon is magnitudes larger than MongoHQ and they apparently did decide to announce the names of affected companies. I am not sure how they handled that internally. [1]
The end users of sites/apps/services using MongoHQ are probably unaffected in many cases since "a database" doesn't imply credit cards or even credentials or emails. The average database is probably just holding content for a site or app.
In probably close to all cases the end users would have no clue how they may be affected since they won't know what's in the databases, or what a database is.
CircleCI is certainly the more interesting target as they have access to numerous companies' source code.
How much do you think that is worth on the black market?
At this point it doesn't seem like they were 'hacked'[0], but simply involved in a larger security incident.
[EDIT] seems the linked page doesn't mention 'hacked' anywhere (literally - I searched for it!) Seems like the title[0] is a classic case of editorialising
[EDIT 2] As per below, would be very interested to see if their database being 'compromised' means that data was actually accessed by the attackers, or if they are just fall out from the bigger attack
[0] original submitted title was "CircleCI has been hacked"
Seems like a 'security incident' is code for 'hacked'.
From that line it does seem more likely that they might be one of MongoHQ's clients who MongoHQ noticed had had their data accessed as part of the bigger breach.
In that context, saying they have been hacked is probably fair. Would love more details.
Lots more discussion over at [1].
The main takeaway is to take security seriously, and employ multiple levels of security. The MongoHQ team are doing things like 2 factor auth, and restricting customer service tools to a vpn. As far as I can see, no framework or coding bugs.
As such, I always very carefully weigh the benefits such a service can provide against how much work it would be to do it myself (which will likely produce a shittier result, but which will be fully under my control).
It's not even that I think I can do security better than $SERVICE, but I'm a much smaller target. I'm not as afraid of a targeted attack than I'm afraid of somebody compromising one of these focal points and then just lifting everything.
CircleCI's response to the incident appear adequate. In the best case, their system was shutdown before any keys were compromised. If so, that level of monitoring is certainly better than what the average business would be able to roll on their own.
nightmares will be had.
was the buffer spamming incident just a way to deflect attention of something more sophisticated?
I wonder if anyone used the keys stored in circleci to get to the code of some service and if we'll see even bigger hacking news in the near future
</conspiracy>
