For any mildly complex modern app, you need integration tests to show context, and behavioral/functional tests (Cucumber) to show the code actually does what it's supposed to do according to the customer.
If you’re testing a REST interface, then how much additional context you need? They’re supposed to be stateless, so they can be tested in isolation just fine, because that’s how they’re supposed to operate. Or does it stop being a unit test once your test includes DB interaction?
For your typical web app, front end testing is the primary place you need anything more complicated, and even then you could debate how much of that should be manual.
I agree that much of the benefit doesn't apply, but even having the traceability makes some auditors happy.
1. Version control 2. 'Ticket' system 3. Automated builds 4. Automated deployments 5. Automated tests 6. Code review 7. ...
It starts to pay off as soon as you have more than 1 developer in the project. Though even with one person building something and setting up automated build and deploy to something there is a lot of value when someone else has to do the changes to a project. We have bunch of small projects with Jenkins and Team city set up. I can jump into a project that I never touched before, make a fix and I don't have to figure out how the hell I am supposed to make production version out of it or build it.
This is the world I live in - building pipelines for financial and health care giants. I agree with this article 100%. Not only can continuous delivery provide regulatory compliance consistency, but also more traditional human-bureaucracy approaches cannot provide it. There are too many potential leaky spots, and far too much potential for human error - in particular, people signing off on things they don't understand. And this includes compliance auditors!
Ehhh... I work in a regulated environment, and think I disagree. The reason I don't know if I do is because it's really a question of definitions.
By this article, developing with our super-manual bureaucratic compliance process of 15 years ago was CD (and probably further back, I've never had to dig that far in the archives). The things we did in the name of disaster recover, control-ability and traceability fit the provided definition of "deployment pipeline."
At the very same time, I think this article is trying to say our highly automated tool-assisted compliance processes of today are not "continuous compliance" because we have manual stakeholder synchronization gates and quality-enforcing review gates on the way to production.
No and no. The goal of regulatory compliance is to avoid liability. You maintain professional standards because that helps keep everyone safe, but you obey the rules to avoid the punishment associated with not obeying them.
This really matters when regulatory compliance conflicts with good judgement. Sometimes you do the bad thing because the bad thing is mandated in the rules. If a regulation says that you have to have "antivirus software" on the machine then you have antivirus software on the machine.... even if no antivirus software exists for that machine. You shoehorn something because the rules say you need it. You don't do this to increase security. You do it because the lawyers tell you that not doing it will get you sued.
But the way you avoid liability in this scenario is by implementing controls that reduce risk. You can fudge the audit if you want to. But for most regulated industries, if you have a major incident, you’ll have an investigation. If that finds your controls actually weren’t in place effectively, you can lose that protection, regardless of the outcome of the audit. The PCI SSC will tell you that a merchant that has been breached has never been found to be compliant at the time of the breach, and that’s probably true.
Part 11 of the FDA regulations that oversee software compliance enforce expensive re-validation processes for every major release of software. It's not something your customers are going to want to do very often. So set up a CD for each customer on version X.y.z where X is static and make sure that you don't accidentally ship a backwards-incompatible major-version change on that pipeline.
It's an interesting challenge for operations-focused teams but I agree that CD is a valuable tool to have.
For instance, you can check a folder remains encrypted or certain ports remain closed in dev and test before promoting your config management to production.
Quite involved to setup but a big tick for auditors.
This was for a firm that handled over 80% of all online loan applications, from all the major banks, insurance companies, credit unions, etc.... From the moment you typed in the first character of your name to when you hit the final submit button, everything was handled on their site, and proxied through to the customer systems via an iframe or other technology.
I tried to convince them that they should have automated compliance testing and confirmation on a regular and frequent basis (like every thirty minutes), but they refused. They said it wasn't necessary according to their auditor, and they weren't going to open themselves up to additional risk by having the tool run too frequently.
They had quarterly audits, and therefore they wanted the tool to only be run quarterly, and to only be kicked off manually at that.
While us normal human beings might be convinced of the need and desire to always be protected, but banks and financial institutions don't think like that. For them, they only ever do the absolute minimum required by law, because anything more just creates more risk.
I have seen multiple pipelines that don’t tell you who manually tested this change - or at least, don’t always tell you. The author here is assuming the existence of a lot of practices already, which makes his pointing to CD as magic look a lot weaker.
Resource Limit Is Reached The website is temporarily unable to service your request as it exceeded resource limit. Please try again later.
