Evaluation is rarely as simple as asking, “Did it work?” because the types of problems that you are working on may be complex, with many contributing factors, stakeholders and success criteria.
The more thoroughly you define your problem, the easier it is to evaluate, because your definition serves as an initial benchmark with the final evaluation being one last step in an on-going process of performance measurement.
Earlier, I said that there are broadly two types of information that you can gather; quantitative and qualitative. Both types are important when you are learning about the problem, and this same information is important again when you come to evaluate your solution, because the information that you gathered at the start of your project becomes a benchmark for evaluation.
Quantitative information is about objective facts, numbers and measurements, for example the cost of a project, the amount of system downtime in a month or the number of people using a system.
Qualitative information focuses on attitudes, behaviours and experiences, for example customer experiences, managers’ opinions or the changes in staff behaviour.
While you can gather ‘snapshot’ data that describes the present situation with the solution in place, if you don’t have an earlier point of reference then it is difficult to evaluate the impact of your project.
Therefore, you should prepare for evaluation as part of your original problem analysis and certainly as part of your implementation plan.
If your project goes according to plan, you will generate evaluation data in three stages.
In the short term, you can evaluate the project itself:
- Did the project deliver what was intended?
- Did the project run within its original timescale?
- Did the project run within its allocated budget?
- What did you learn from the project?
Then, in the medium term, you can evaluate the project’s impact:
- Did the project solve the problem?
- Did the project meet all of the solution criteria?
Finally, there are long term benefits that you can also evaluate:
- Did the project deliver a return on investment?
- Did the solution prevent the problem recurring?
Evaluation isn’t about proving that your solution and your project were a complete success, it is about assessing what you did and learning how it can be done better, faster and cheaper next time.