A major topic on Twitter for me this week was that of problem prevention.
We all know error handling, error solving, creating workarounds or solutions for them, but "A stitch in time saves nine" is a debatable issue in ICT
Or is it?
It all started with a tweet from Jamie Oswald:
IT's biggest problem is its realization that there is more money in managing one's problems than in fixing them. #grrrrr
to which I responded:
@oswaldxxl also, hard to get budget for preventing problems from occurring
Thorsten Franz pitched in too:
I believe Thorsten, but his boss might be one out of many thousands. I believe there is not much of a business case in IT for problem prevention. Why? Because most of those problems exist mostly in people's heads, and not on paper. Even when on paper, they still have to gain support from those that have the money to fund them
This isn't true for all problems that get noticed: there are plenty that are out there and obvious for most, if not all. I'm talking about problems that don't exist in the perception of the users, the business, and management, but are raised by IT personnel "on the floor".
I once witnessed a discussion that first made me wonder, then wrily smile. It was about a small server room where servers and applications were placed. The room was managed by local IT, and one of those places where "temporary applications" were put
In IT, there's nothing more permanent than a temporary solution
As a consequence, there were little to no rules to enter an application into the room. Everything was fully virtualised, so that threshold was low too. Most application only had an owner, and nothing else. The IT people brought the following to the attention of the management:
- Only known contacts for applications are the business owners - no IT people are on any list
- Application behaviour and usage is unknown
- CPU, memory and disk space utilisation of the applications are unknown
- There are no criteria to determine who can put their application in the room, so in theory we have to allow everyone who requests so
- Applications aren't monitored, but there is no money involved either
- There are no requirements for max or average CPU, memory or disk space usage - in theory one app could take it all
- Between now and eternity, all physical hardware will be filled
They suggested to do an inventory of CPU, memory and hardware usage. A few measurements had indicated that they averaged out on 40% of all available resources. They also suggested to make a set of criteria that would maximise usage for new (and possibly existing) applications. And they suggested to have the contact person list extended with people that built or designed the applications, and get an operational manual for each one so applications could be stopped (and restarted) if that ever needed to be the case
The management listened. Or heard. Pretty soon during the talk, they intervened, asking how and why questions. The word should was used a lot, as was shouldn't. I watched. I saw the IT people get frustrated, unable to answer questions the management asked (which really were questions they should answer themselves). I saw them get lost, and demotivated in the end, when the management said there were more pressing problems at the moment, and that they would get back at this "at a later point in time"
I couldn't make sense of it. There was a clear problem ahead at the horizon, and an issue already now. Still, management didn't act.
Then I got it. There wasn't a problem. Yet. Management had to take this story and go to their managers and do a lot of would-be's and if-then's themselves, in order to make this non-existing problem visible. Chances were pretty great that they wouldn't succeed at doing so, and lose face
Losing face is the greatest sin for most managers - their face is pretty much all they have. For the management in this example, it would be far better to ignore the future problem and wait till it became a real present problem. Then they could go to their management, and even tell them what solution they had. In stead of troublemakers, they would be troubleshooters! And guaranteed to "win face", in stead of lose some
Solving problems? There's a lot more to gain by managing them until they become too great