DevOps & The Segregation of Duties

Posted on May 20, 2015

Many people believe that traditional IT Service Management (ITSM) and DevOps are incompatible.  Nothing could be further from the truth.  Most frameworks and standards for governance and security such as COBIT5, ITILv3, CMMI-SVC, ISO27K, etc, are treasure troves of good ideas.  The reason there is such tension between these two camps lies not in the words but rather in the perspectives of the reader.  Our perspectives are rooted in our core beliefs and principles.  In Systems Thinking (or LEAN if you prefer) we call these perspectives “mental models”.  Therefore, in order to change from a Traditional IT mental model to a DevOps/LEAN mental model we must challenge and change our beliefs and principles.

When an auditor comes knocking on the door of an organization which embraces the DevOps philosophy a number of long discussions will unfold.  There will be a lot of explanations and confusion, but ultimately they will be resolved one-by-one.  There is only one question which tends to scare people:  “Tell me about how you implement segregation of duties.”

Gulp.  “Um….  we’re DevOps!  We don’t believe in the segregation of duties!  Everyone is involved!”

Doh!  Wrong answer.

Fear not, there is a solution to this inpass.  Lets look at both sides of the equation here, our beliefs and the segregation of duties control.

“Segregation of duties” is a control (think “operational principle”) which helps us manage risk.  The classical implementation of this control is to create a policy and procedures which ensure that the a person who works with money isn’t the person who counts the money at the end of the day.  Why is that important?  Because if the person working a register stole $100, they could simply falsify the counting at the end of the day.  It breaks the check-and-balance which prevents theft.

When applied to computers the control remains unchanged but the implementation looks different.  In the context of a server segregation of duties means that a person that can change a system shouldn’t be able to remove evidence that they did so.  I think we’d all agree that this is a sensible idea, especially given the possibility that an account might be compromised.  But how do we implement this when so many DevOps proponents want to insist everyone should have root?

The first part of the answer is Role-Based Access Control (RBAC).  Solaris/Illumos has an excellent RBAC system.  Linux has SE Linux.  While not ideal, sudo can, and most frequently does, also do the trick.  

The second part of the answer is remote logging.  Logs should be immediately transported off to a secure remote server and the ability to modify logging parameters should be protected on each host.

Perhaps the most important word associated with any segregation of duties control is the word “reasonable”.  It is accepted that anyone fully bent on malicious activity will find a way to do so.  Your responsibility is to do a reasonable job in preventing it.

Just ask yourself, if someone connected to my billing database and modified a customers record, would I have a reasonable chance of detecting the change and who preformed it?  If the answer is “No” then you have work to do, whether your doing the DevOps or not.