There are number of different approaches to cost estimation. The challenge is to find a blend of techniques that provides you with a “rough order of magnitude” very quickly and as more data becomes available, the estimate can be adjusted accordingly.
But before we go there, what exactly is an estimate when we are talking about IT projects? The Oxford dictionary defines it as
“a judgement that you make without having the exact details or figures about the size, amount, cost, etc. of something”
When IT projects are funded, the Cost Estimates quickly become Cost Commitments.
The following is a list of cost estimation methods:
- Expert judgement
“Expert judgement” has many challenges related to bias but there are techniques available to make it a very quick and fairly accurate method of doing “rough order of magnitude” cost estimates. This assumes that the “experts” have been with the organisation for some time with domain knowledge in the area of the estimate.
- Top down
“Top down” is similar to expert judgement based on a high level view of the project requirements. It is useful to try and define the range that the project is likely to be within i.e. probably no bigger than “X” and no smaller than “Y” and with a duration of approximately “Z” months.
- Analogy based
“Analogy based” estimates can be very good if the organisation has good data from previous projects. We often find that data from previous projects is kept but not neccesarily in a format that can be used for comparisons. If there is good project data available then the current estimate can be compared to similar previous projects.
- Bottom up
“Bottom Up” estimates are the most accurate but require that all aspects of the detailed requirements have been completed. This means that a lot of design and analysis work has been completed without knowing if the project will be approved.
- Knowledge based
“Knowledge based” estimates are used where there is domain knowledge data that can be used to understand role, effort ad duration for the elements. Different element volumes can be applied to get the estimate.
- Parametric
“Parametric based” estimates use relationships between elements to develop the estimate. The SEER suite of tools uses a combination of knowledge bases and parameters to quickly develop comprehensive cost models.
The journey
Most organisations use home grown MS-Excel spreadsheets to calculate costs. Unfortunately, in many instances, each cost estimator has their own version and they don’t necessarily share key metrics.Activity based costing in IT has been challenging. Tools like Apptio help but when it comes down to the cost of managing a router, the detail gets a little fuzzy. Most CIOs can tell you that the Server Department can run 50 servers per FTE – but they also do a lot of other things besides just server monitoring and maintenance.
While consulting on a very large deal in 2001, it became apparent that suppliers generally don’t have a good understanding of their operational costs. This led to a focus on cost estimation and a journey to understand IT cost estimation and the tools that are available.
Over the past 20 years, we have looked at a number of organisation’s costing tools. The first thing that became apparent is that they were all MS-Excel based and were “tweaked” based on the project at hand – a very dangerous practice. The second thing was that different parts of the same organisation will use different cost modelling tools for the same scope.
There is no formal training or accreditation for IT cost modelling. What seems to happen is that someone will build an Excel model and become the de facto “expert” in the area.
So why is cost estimation so difficult?
IT projects cover a broad spectrum of technologies and activities:
- Regulatory changes and compliance
- Replacing obsolete hardware and software
- Business changes
- Process changes
- Different user interfaces
- APIs
- Software licensing
While some items are relatively easy to cost, things like data cleansing are very difficult.
“The bigger the project, the more risk for potential problems. “It’s usually a misalignment of expectations on the customer side and on the capability of the service provider.”
Damien Bailey
Unfortunately, the estimation process is often subjective and not repeatable. The person creating estimates is usually an IT generalist and does not have a good understanding of the complexities and risks associated with the various cost model elements.
Over time the models become more complex and job security is ensured as none of the models are documented.
The models generally focus on the “run” or operational costs associated with IT infrastructure and typically ignore things like documentation, training and Service Desk skills required for the new project.
Galorath Incorporated defines three tribes of cost estimators:
The Napkins: Ad-hoc, hero driven group of estimators whose past successes are legendary and whose napkin scribbling is taken as gospel.
The Guts: Feeling and experience oriented group of project managers and workers whose years of experience,good and bad, makes them trusted prognosticators of what will happen regardless of the actual results (tenure benefits of being a survivor).
The Spreadsheets: Former and current Napkins and Guts members who believe translating their tribal knowledge onto MS-Excel spreadsheets bestows mathematical accuracy and empirical integrity on home grown algorithms.
Software development
Software development estimates are generally based on “expert judgement”. For anyone interested in this, we would highly recommend the book “Software Estimation – Demystifying the Black Art” by Steve McConnell. One of the more interesting items in the book is the fact that an accurate cost estimate cannot be delivered until sufficient data is available – the Cone of Uncertainty.
CAST Imaging allows your software teams to:
- Automatically discover and see the application architecture
- Quickly navigate all components and their dependencies
- Respond faster to business needs – from small changes to modernization
- Increase cross-team collaboration and efficiency
- Speed up knowledge transfer to newcomers
- Keep a living knowledgebase of the application’s inner workings
Tools
In 2010, a supplier asked us to do a post implementation review of a major bid. We quickly came to the conclusion that they struggled with detailed costing. A worldwide search was then undertaken for cost estimation tools and we became the Australian resellers of the SEER suite of products from Galorath Inc. Given our background, we focus primarily on SEER for IT and SEER for Software.
With the challenges of analysing workloads that can be moved to the cloud, we once again went looking for tools and found CAST Highlight, an amazing tool that takes all the human factor out of software analysis. When coupled with SEER for Software, very accurate estimates can be produced to reflect the time and effort to remediate code in preparation for cloud migration.
While the tools are very good, the process is critical to improved cost estimation efficiency
understanding existing code and architecture. CAST Imaging allows application teams to:
- Learn application make-up in days vs weeks
- Understand end-to-end flows in minutes vs hours
- Comprehend app-to-app dependencies in hours vs days
Cost estimation maturity
As with software development, organisations can be at different levels of cost estimation proficiency.
Very few organisations have implemented a dedicated cost estimation centre (CEC) with the sole purpose of providing IT costings. The CEC supports the CIO and IT to estimate costs for IT projects but can also support Procurement to do an internal costing for an IT project before going to market. This ensures that the stakeholder community are aware of the magnitude of the problem before engaging with the market. Many times we have seen customers having an expectation that a project will cost say $3 million and when they go to market it comes back with a price of $5 million. Unfortunately, what usually happens at that point is to see what can be achieved for $3 million…
The CEC can identify high risk projects as well as projects that have a limited return on investment. Thresholds can be set and these projects can be further reviewed for their viability and benefit to the organisation. A large US government department has implemented this approach and claims to be saving approximately $1 billion per annum by not initiating projects that have a low probability of success
Benefits of a Cost Estimation Centre
While the focus of this section relates to IT costing, the Cost Estimation Centre can provide support to the CIO and CFO, business units and procurement.
If services are outsourced, the CEC can provide support for these roles to ensure that the supplier is providing fair prices to assist in negotiation. Importantly, if the project has been costed internally, it will show if the supplier(s) have missed something in their proposal.
The CEC can support the organisation by providing accurate costs for budget purposes and will provide insight into risks associated with a project. The ability to quickly do “what if” analysis can show the impact on Cost, Effort, Quality, Duration and Risk.
The CEC also ensures that cost estimation does not become single point exposed to a costing subject matter expert, and that robust processes for initiating, tracking and adjusting estimates are in place.
CEC implementation process
The start of the CEC implementation process is a workshop with the cost estimators and a review of the tools and processes currently being used. There will also be a focus on the time it takes to do different types/sizes of estimates and identify issues that may exist.
If there is agreement that improvements can be made, a business case will be developed for executive consideration and approval.
Once approved, mapping of the current state and future state tools and processes will be undertaken and a plan developed to implement.
Good data will be loaded into tools and knowledge bases and, if new tools are used, training the teams in their use.
Establish processes for version control and archiving of cost estimates.
Inform stakeholders of CEC engagement model and go live.
Review after six months to identify any areas that may need adjusting.
Work with the organisation and ensure cost estimates:
- are a range that becomes smaller as more data becomes available
- are used to validate suppliers bids/change requests
- are used in a standard manner for business unit IT budgets
- compare risks at different cost points
- assess the least time a project can be completed in
