While consulting on a very large deal in 2001, it became apparent that suppliers generally don’t have a good understanding of their operational costs. This led to a focus on cost estimation and a journey to understand IT cost estimation and the tools that are available.
Over the past 20 years, we have looked at a number of organisation’s costing tools. The first thing that became apparent is that they were all MS-Excel based and were “tweaked” based on the project at hand – a very dangerous practice. The second thing was that different parts of the same organisation will use different cost modelling tools for the same scope.
There is no formal training or accreditation for IT cost modelling. What seems to happen is that someone will build an Excel model and become the de facto “expert” in the area.
Galorath Incorporated defines three tribes of cost estimators:
The Napkins: Ad-hoc, hero driven group of estimators whose past successes are legendary and whose napkin scribbling is taken as gospel.
The Guts: Feeling and experience oriented group of project managers and workers whose years of experience,good and bad, makes them trusted prognosticators of what will happen regardless of the actual results (tenure benefits of being a survivor).
The Spreadsheets: Former and current Napkins and Guts members who believe translating their tribal knowledge onto MS-Excel spreadsheets bestows mathematical accuracy and empirical integrity on home grown algorithms.
So why is cost estimation so difficult?
IT projects cover a broad spectrum of technologies and activities:
- Regulatory changes and compliance
- Replacing obsolete hardware and software
- Business changes
- Process changes
- Different user interfaces
- APIs
- Software licensing
“The bigger the project, the more risk for potential problems. “It’s usually a misalignment of expectations on the customer side and on the capability of the service provider.”
Damien Bailey
Unfortunately, the estimation process is often subjective and not repeatable. The person creating estimates is usually an IT generalist and does not have a good understanding of the complexities and risks associated with the various cost model elements.
Over time the models become more complex and job security is ensured as none of the models are documented.
The models generally focus on the “run” or operational costs associated with IT infrastructure and typically ignore things like documentation, training and Service Desk skills required for the new project.
Software development
Software development estimates are generally based on “expert judgement”. For anyone interested in this, we would highly recommend the book “Software Estimation – Demystifying the Black Art” by Steve McConnell. One of the more interesting items in the book is the fact that an accurate cost estimate cannot be delivered until sufficient data is available – the Cone of Uncertainty.
Tools
In 2010, a supplier asked us to do a post implementation review of a major bid. We quickly came to the conclusion that they struggled with detailed costing. A worldwide search was then undertaken for cost estimation tools and we became the Australian resellers of the SEER suite of products from Galorath Inc. Given our background, we focus primarily on SEER for IT and SEER for Software.
With the challenges of analysing workloads that can be moved to the cloud, we once again went looking for tools and found CAST Highlight, an amazing tool that takes all the human factor out of software analysis. When coupled with SEER for Software, very accurate estimates can be produced to reflect the time and effort to remediate code in preparation for cloud migration.
While the tools are very good, the process is critical to improved cost estimation efficiency.
One of the other challenges that faces IT projects is the quality and age of the underlying applications and infrastructure. There comes a point where it is less expensive to retire and application than to fix it or add required additional functionality.
Galorath Incorporated
CAST Highlight
Cost estimation maturity
As with software development, organisations can be at different levels of cost estimation proficiency.
Very few organisations have implemented a dedicated cost estimation centre (CEC) with the sole purpose of providing IT costings. The CEC supports the CIO and IT to estimate costs for IT projects but can also support Procurement to do an internal costing for an IT project before going to market. This ensures that the stakeholder community are aware of the magnitude of the problem before engaging with the market. Many times we have seen customers having an expectation that a project will cost say $3 million and when they go to market it comes back with a price of $5 million. Unfortunately, what usually happens at that point is to see what can be achieved for $3 million…
The CEC can identify high risk projects as well as projects that have a limited return on investment. Thresholds can be set and these projects can be further reviewed for their viability and benefit to the organisation. A large US government department has implemented this approach and claims to be saving approximately $1 billion per annum by not initiating projects that have a low probability of success.
