Continuing with the series, this week I’d like to introduce you to a fantastic method for writing up your interview notes and ensuring that you gather all the evidence you need for determining a fair and consistent interview process.
Introducing ORCE Methodology
ORCE stands for Observe, Record, Classify and Evaluate. It’s a way of asking questions which will help you to gather all the necessary data you need, and ensuring you make objective and fair judgements on the person’s suitability and capability for your job vacancy.
During the interview itself you are focusing on Observe and Record part of the ORCE methodology.
- Observe/Record - When observing, you should make extensive notes of what the candidate actually says. This forms a record of their likely behaviours and is the basis upon which subsequent evaluations will be made. At this stage you should try to achieve as detailed a record as possible for the candidate you are assessing, avoiding evaluating whether something is relevant and deciding which information can be left out.
- Classify - Having established a good record of what the candidate has actually said, the next stage is to classify them appropriately against the competencies you’ve already determined. To ensure that you do this effectively, you must be familiar with the competencies and the behavioural indicators.
- Evaluate - The final stage of the process is to evaluate the classified behaviours. We recommend using a numerically defined rating scale. This allows you to come to a clear and fair conclusion of performance against each individual competency and indeed to evaluate a person’s suitability for the job in question.
Things to watch out for when Evaluating your Evidence.
Recruiters need to be aware of the following potential sources of error when rating performance:
- Halo Effect - The tendency to generalise one aspect of good performance to all performance.
- Horns Effect - The tendency to generalise one aspect of poor performance to all performance.
- Central Tendency - The tendency to give “middle of the road” evaluation often so that one can avoid having to justify high or low scores.
- Leniency - The tendency to generally view people’s performance favourably.
- Harshness - The tendency to generally view people’s performance unfavourably.
- First Impressions - The tendency to be most influenced by initial impressions and to disregard subsequent contrary evidence.
- Recency - The tendency to be most influenced by the last observations made and to disregard earlier contrary evidence.
- Stereotyping - Allowing personal biases and prejudices arising from stereotyped views of groups of people to distort evaluation of actual performance.
Using a Rating Scale
We recommend the following rating scale when assessing your candidates:
5 = Significantly above the acceptable standard - The candidate has demonstrated most of the positive indicators related to the competency throughout the exercise, with no evidence of any negative behaviour. Very strong examples/evidence provided – clearly an area of strength. No need for personal development in this area for the role applied for.
4 = Above the acceptable standard - The candidate demonstrated most of the positive indicators related to the competency in the exercise with limited evidence of negative behaviours. Strong examples/evidence provided – potentially an area of strength. Limited need for personal development in this area for the role applied for.
3 = Acceptable standard - The candidate demonstrated more positive indicators related to the competency than negative indicators in the exercise. Any negative indicators would not raise concerns regarding ability to perform in the role. Development is desirable but not essential for this role.
2 = Below the acceptable standard - The candidate demonstrated more negative indicators of the competency than positive indicators in the exercise. Needs some personal development concentrating on strengthening desired behaviours for this role.
1 = Significantly below the acceptable standard - The candidate demonstrated significant negative indicators with little, if any positive behaviour related to the competency. Limited contribution to the exercise. Strong development needed concentrating on addressing negative behaviours and broadening positive behaviours.
N/E = No evidence - No evidence of this competency was demonstrated in this exercise.
When allocating a score on the rating scale you should be conscious that this is not a simple process of comparing the number of positive indicators against the number of negative indicators. The rating should not be a numerically based decision, but a proper evaluation of the evidence you have gathered and a determination of whether the individual is deemed to have demonstrated sufficient evidence for you to be comfortable that they will be effective in the role.
Before you write anything…
- Read your notes fully
- Read and annotate the competency for what evidence you have seen. Clearly link the evidence to the competency by assigning a positive or negative indicator to each piece of evidence, to do this tick the box, which represents the evidence you have gathered
- Link the evidence to the behavioural indicator, you must be able to back up your statement by evidence you have. Use the phrase ‘demonstrated by’ or ‘evidenced by’ to help you to do this
- Classify one piece of evidence to one competency
- A bullet point style is recommended as this is a summary of the evidence not a re-write of your notes
- Use an overall competency definition throughout this process to ensure you are clear in what you are looking for
- Avoid using names or 3rd parties for confidentiality
- Ensure that your write-up is comprehensive and can be understood by others who may not have been in attendance at the interview itself
- Be consistent
Remember that under the Data Protection Act candidates have the right to see the notes and any other information that you have gathered on them. It is your responsibility to ensure that your notes reflect only what you have observed.
For an informal chat about how we can help with your recruitment, please get in touch with Michelle on 01202 853647 or email us at firstname.lastname@example.org Watch out for the next step – After the Interview.