Embedded Files
Evaluation
Evaluation
During the design stage you create the evaluation criteria. This helps to guide the design and development of the solution. Once the solution has been finalised and installed, the formal evaluation process can be conducted. The evaluation might take place 3–6 months after the solution is implemented so that people have time to learn the new system and to give time for any minor bugs to be dealt with.
Evaluation methods
Evaluation methods
For each evaluation criterion, there must be a corresponding evaluation method that can measure the degree to which the criterion has been achieved.
• Objective (fact-based, measurable) results are solid facts that are hard to argue with. Measure whenever you can.
• Subjective results (emotions, opinions, personal judgements) can be gained from interviews, questionnaires and surveys. These should only be used when objective measurement is not possible or practical.
Remember: evaluation assesses your solution’s performance over time. It is not instantaneous in the same way that testing is. Any emotional or judgemental feedback is only gathered on appropriate criteria. For example, it is pointless to ask interviewees questions such as, ‘Is the new system faster than the old one?’ Even if you received an answer to this question, you would not be able to trust its accuracy.
When to evaluate
When to evaluate
Evaluation occurs after the solution has been in regular use for some time, so the solution is well ‘bedded in’ and its users are familiar and comfortable with it. A few months of regular, daily use is typical.
Evaluating a solution too soon can lead to negative feedback if users are not yet used to it and are slow and prone to making errors. Later, when they are comfortable and skilled with the solution, their feedback may be much more positive.
In cases when a system is used infrequently, but its success is critical to the organisation (such as creating school reports), evaluation may be done immediately after the system is used.
Efficiency can be measured in terms of speed or productivity (work produced in a given time), profitability (income generated versus running costs) and labour requirements (how much labour is required to achieve its productivity levels).
Effectiveness includes completeness, readability, attractiveness, clarity, accuracy, accessibility, timeliness, communication of message, relevance and usability.
Typical Evaluation Methods
Typical Evaluation Methods
Page updated
Report abuse