Monday, October 22, 2012

Visible Ops and ITIL: What Not To Do

Today I looked to procure an additional copy of Visible Ops, “The Yellow Book”, for a coworker.  I stumbled upon a rather recent article at ZDNet that can be found here:

The summary is stated as, “With ITIL and change management, the Visible Ops framework promises a new approach to creating an efficient IT operational model. But its chance of success is open to doubt without changes to the way IT is procured, designed, configured, validated and implemented.”
I was rather surprised by the author’s perceived shortcomings in regard to the Visible Ops / ITIL Framework.  Some of the reader comments were not much better.  Let’s take a look.
The author states, “That distinction was never more apparent than in the almost pointless weekly change advisory board meetings. While the processes themselves were painfully bureaucratic and often a diversion from operational work, the meetings themselves were just strange.
With barely anyone in attendance, the board would ask for a justification for each change, with a response of "approve" or "rejected" even though it was clear they had little or no grasp of the technical explanation or implication provided.”
Why the hell would you allow your change advisory board to be scarcely attended and lacking the participation of the folks capable of properly evaluating the change requests raised?  Without a true CMDB, which few organizations actually possess, the only folks capable of properly evaluating these changes have all the organizational knowledge within their collective skulls.  Don’t blame ITIL for your organization’s inability to properly manage the attendance and content of these meetings.
Or let’s try this one.  The author goes on to state, “Then there was the security and risk-compliance chap who'd lock himself in his room, glued to his Tripwire dashboard spying on any unapproved changes. So imagine his amazement when I introduced him to VMware and vMotion.”
So, ummm, right.  Technology progressed.  Your organization failed to progress with it by eliminating virtualization workload shifts from the activities requiring change control/change documentation.  What does this have to do with ITIL?  So the answer is no, you really shouldn’t have to raise a change to allow vMotion to do its job – your process was simply hosed and you almost sound amused to have been part of the problem rather than a contributor to the solution.
Last but not least there is this gem.  Several years later, as a technical consultant my impression of the effectiveness of the change advisory board failed to improve. Late one night at a customer's datacentre, I'd pointed out that they had cabled up the wrong ports and that this would require a change to be raised.
"No need for that" replied the SAN architect, "I'm one of the board members". He then proceeded to pull out and swap the FC cables to his production hosts with a big grin on his face. Several minutes later his phone rang, to which he replied, "It's OK, I've resolved it. There was a power failure on some servers." Then he turned to me with a wink and said, "There you go. Sorted. Lubbly jubbly."”
This person should be coached immediately.  If this was not a first time offense a termination would be the appropriate response.  This is reckless behavior and should not be tolerated.
Well let’s not leave the readers off the hook here.  Someone named ammohunt responded with the following (edited four spelling errors), “Where the change control piece of ITIL breaks down is determine what is a maintenance task and what constitutes a change in our case we felt the better judgment should have been left up to us; the technical folks. Management disagree and decided that they would decide this...It got to the point where my manager was insisting that i create a change control for editing a host file. At first i started implementing tier 0; as in Null change controls..i.e. what management doesn't know won't hurt them which quickly earned me the cowboy label. Eventually the best solution was to give up on the company and seek gainful employment at a company with sane process; many others did the same.”
Well I am sorry, ammohunt, while your technical guidance should be considered when vetting which changes require proper approval/documentation you are not steering the ship here.  I recently witnessed a significant downtime to an EHR system as the result of, well, editing a hosts file.  Glad to hear you found gainful employment elsewhere - your previous employer likely gets some additional sleep knowing that current employees to do not share the cowboy mentality.

No comments: