Competency Assessment Methods: Matching Evidence to Skill

Search for competency assessment methods and you get lists. Observation, questioning, tests, 360 feedback, a dozen more. What the lists rarely do is answer the questions HR teams actually ask once a framework exists. Is a knowledge test enough? Does self-assessment count? Who should do the assessing, and how often? Can one manager's sign-off stand on its own?

This post takes those questions one at a time, in roughly the order they come up.

Competency assessment methods are the ways you gather evidence that someone performs to a defined standard: direct observation, questioning, professional discussion, work product review, simulation, knowledge tests, witness testimony, 360 input and self-assessment. No single method suits every competency. Choose by what the competency is, how costly failure would be and how often the task happens, then corroborate with a second source.

What are the main competency assessment methods?

There are around nine methods in common use, and they split into two families. Direct methods let the assessor see the performance or its output first hand. Indirect methods rely on someone else's account, a conversation, or a staged version of the work. The table sets out what each one can genuinely tell you.

Method	What it shows	Best for	Weak spot
Direct observation	Performance in real working conditions	Practical skills and visible behaviours	Rare events, and one occasion may not be typical
Questioning	The understanding behind what was seen	Why a step was taken; "what if" variations	Knowledge alone does not prove performance
Professional discussion	Depth of reasoning across several real examples	Judgement, leadership, complex or senior roles	Depends on the assessor; can reward fluent talkers
Work product review	The quality of what someone produces	Reports, plans, designs, records, code	Authorship, and it hides how the work was done
Simulation or scenario	Performance in events you cannot wait for	Emergencies, faults, difficult conversations	Only as good as its realism
Knowledge test	Recall and understanding of rules and facts	Regulations, procedures, product knowledge	Says nothing about doing
Witness testimony	Performance seen by someone else	Work the assessor cannot attend	Drifts into opinion unless it describes what was seen
360 or multi-source feedback	How behaviour lands with colleagues	Interpersonal and leadership behaviours	Perception, not measurement against a standard
Self-assessment	The person's own view of their level	Opening the conversation; development planning	Weakly related to observed performance

These are close to the methods named in the Scottish Qualifications Authority's workplace assessor standard, Unit L&D9DI, which lists observation, examining work products and questioning as direct methods, and discussion with candidates, witness testimony, candidate statements, simulated environments and recognition of prior learning as indirect ones. Recognition of prior learning is less a method than a decision to accept evidence that already exists, so it is left out of the table.

Why can't a knowledge test prove someone is competent?

Because competence is something you do, and a test measures what you know. A person can score full marks on a lifting operations quiz and still set up a lift badly. The test is useful evidence of understanding. It is not evidence of performance.

The rail regulator puts this more bluntly than most HR guidance. The Office of Rail and Road's Developing and maintaining staff competence (Railway Safety Publication 1, November 2016) says questioning "should only be used to supplement other assessment methods, as knowledge alone cannot be assumed to infer competence." It is written for safety-critical rail work, but the logic holds anywhere a competency involves doing something.

The same idea sits inside the assessor standards as the definition of validity. SQA defines a valid method as "selecting and using a method of assessment appropriate to the skills and knowledge being assessed." A test is valid for knowledge. It is not valid, on its own, for skill or behaviour.

This is the step most method guides skip. They describe each method well, then leave you to choose freely, as if they were interchangeable. They are not. Every competency in your framework has a kind of evidence it needs, and some methods simply cannot supply it. If you have written good behavioural indicators, the indicators themselves tell you which: an indicator you can only see in a team meeting needs observation or witness testimony, not a quiz.

How do you choose the right method for each competency?

Work through each competency in the framework with the same six questions. The answers give you a primary method, a corroborating method and a reassessment rule, which is everything an assessor needs.

1. Classify what the competency is

Decide whether it is mainly knowledge (knowing the rules for handling personal data), skill (completing a fault diagnosis) or behaviour (keeping colleagues informed). Most competencies mix the three, so note the dominant one. That decides which column of the matrix you are working in.

2. Weigh the consequence of getting it wrong

ORR's guidance says methods should consider "the nature, complexity and risk associated with the task" and "the immediacy of the effect of non-competent performance." A competency where a mistake injures someone deserves a more rigorous method than one where a mistake costs an afternoon of rework.

3. Check how often the task really happens

If the task happens daily, observe it in normal work. If it happens once a year, or only in an emergency, you cannot wait for it. That is what simulation, scenarios and tabletop exercises are for, and ORR adds that any simulation should reproduce the real pressures, such as time constraints and communication protocols, as closely as possible.

4. Pick a primary method that sees the performance

For skills, that usually means observation or work product review. For behaviours, observation over time, witness testimony or structured 360 input. For knowledge, a test or questioning. The primary method should be the one that produces direct evidence of the dominant element from step 1.

5. Add a second method to corroborate

One method on one occasion is a thin basis for a judgement. Pair observation with questioning so you know the person understood what they did, or pair a work product with a short discussion so you know it was theirs. ORR's list of acceptable approaches is explicitly "any combination" of methods.

6. Set the reassessment rule and the record

Decide when the assessment expires and what triggers an earlier one, such as an incident, a change in procedure or a long absence. ORR's position is that "competence assessments should all have expiry dates," and that organisations should record why they chose each reassessment frequency. Then decide what the record must hold: the method used, the evidence, the assessor and the expiry.

What makes assessment evidence good enough?

UK assessor standards use five tests, usually shortened to VARCS: valid, authentic, reliable, current and sufficient. If a piece of evidence fails any of them, it should not carry a competence decision on its own.

• Valid: the method suits what is being assessed. A test for knowledge, observation for skill.
• Authentic: SQA's definition is simply "being the candidate's own work." This is the weak point of work product review and remote evidence.
• Reliable: different assessors would reach the same judgement from the same evidence. This is where unclear indicators and untrained assessors show up.
• Current: evidence that the person "still possesses the skills and knowledge being claimed." A certificate from six years ago is history, not currency.
• Sufficient: enough evidence, across enough situations, to support the decision.

Apprenticeship end-point assessment plans show how strict this can get in practice. The Lead Practitioner in Adult Care (Level 4) assessment plan requires a portfolio to underpin its professional discussion, but states that "reflective accounts and self-evaluation cannot be included as evidence," and that employer contributions should focus on direct observation, such as witness statements, "rather than opinions." That is a useful line to borrow for your own framework: a manager's comment that someone is "great with clients" is an opinion, while a note of what they did in a specific meeting is evidence.

Is self-assessment a valid way to assess competence?

Not on its own. Self-assessment is a good way to start a conversation and to plan development, but the evidence that people rate their own competence accurately is weak.

One well-known review comes from medicine.Davis and colleagues (JAMA, 2006) screened 725 articles and found 17 studies that compared doctors' self-assessments with external measures of their competence. Of the comparisons reported, 13 showed little, no or an inverse relationship, and 7 showed a positive one. The authors concluded that physicians "have a limited ability to accurately self-assess." There is no obvious reason to expect other professions to be better at it.

Use self-assessment where it helps: before a review, so the person has thought about their own evidence, and as a way to spot where the person and their manager see things differently. Where the two ratings diverge, that gap is the most useful agenda item you have. Just do not let a self-rating stand as the decision. The same caution applies to 360 feedback, which tells you how behaviour is perceived, not whether it meets the standard.

Where do competency assessments go wrong?

One method for everything

A familiar pattern is a single annual manager rating, applied to every competency in the framework. It is easy to run and easy to report on. It is also the same method for knowledge, skill and behaviour, which the matrix above shows cannot be valid for all three.

Assessing the rare event in the routine

Emergency response, escalation and difficult conversations rarely occur during a scheduled observation, so they are either skipped or quietly marked as met. ORR notes that competence for degraded operations and emergencies may need different methods precisely because these situations "may be rarely encountered." If a competency only matters when things go wrong, plan a scenario for it.

Assessors who have never been assessed

A professional discussion run by an untrained manager becomes a friendly chat, and an observation becomes a tick against a list. ORR expects reassessment to be done by staff "who are competent to undertake the assessment and whose own assessment is current." Calibrating assessors against shared examples, the same way you would calibrate performance ratings, is the cheapest fix for poor reliability.

No expiry date

A competence signed off once and never revisited quietly turns into an assumption. Every assessment needs an expiry and a trigger for early reassessment. Our guide to validation and revalidation at scale covers how to set those frequencies.

Opinions recorded as evidence

"Very capable" in a comments box is not evidence. If an assessment record would not let a second assessor reach the same decision, it fails the reliability test, however senior the person who wrote it.

What does a method plan look like for one role?

Here is a method plan for an illustrative warehouse team leader with five competencies. It is not a template to copy line for line; it shows the six questions applied, so you can see how differently each competency ends up being assessed.

Competency	Dominant element	Primary method	Corroborating method	Reassess when
Safe operation of powered pallet trucks	Skill	Observation in live work	Questioning on hazards and checks	On expiry, after any incident or near miss
Knowledge of site fire and evacuation procedure	Knowledge	Short knowledge test	Walk-through of the route on the floor	After any change to the procedure
Responding to a spill or injury on shift	Skill under pressure	Scenario or drill	Professional discussion of a past event	On expiry; it rarely happens for real
Running a start-of-shift briefing	Behaviour	Observation on more than one shift	Witness testimony from a shift manager	When the role or team changes
Handling an underperformance conversation	Behaviour and judgement	Professional discussion of real cases	Role-play with a trained assessor	Before first leading one alone

Notice that no row relies on self-assessment, and only one relies on a test. The expiry periods are left as triggers because the right interval depends on your own risk assessment, and ORR's point stands: record why you picked the one you did. If you are building the framework first, start with what a competency framework is and the proficiency levels each assessment is judged against.

How StaffCircle helps you assess competence

StaffCircle will not choose your methods for you, and it should not. What it does is hold the framework, the evidence and the decisions in one place, so the plan above runs as a process rather than a spreadsheet.

Competencies, levels and indicators in one framework

Build each competency with its proficiency levels and behavioural indicators in the competency framework module, so every assessor is judging against the same written standard.

Manager and self-assessment side by side

Collect the employee's view and the manager's assessment against the same competencies, and use the differences to drive the conversation rather than to settle it.

Evidence and expiry, not just a rating

Record when a competency was assessed and when it is due again, and see across a team which assessments are close to expiry, through training compliance and skills reporting.

Gaps that turn into development

Where an assessment shows a gap, link it to a development action in skills and development, so the next assessment has something to look for.

Final thoughts

The question is rarely "which method is best?" It is "which method can actually show me this competency?" Answer that per competency, add a second source, give every decision an expiry date, and you will have an assessment process that stands up when someone asks how you know. If you would like to see how that runs in practice, book a demo.

FAQ

What are the methods of competency assessment?

The main methods are direct observation, questioning, professional discussion, work product review, simulation or scenarios, knowledge tests, witness testimony, 360 or multi-source feedback and self-assessment. Direct methods let the assessor see the work first hand; indirect methods rely on a conversation, someone else's account or a staged version of the task.

Which competency assessment method is the most reliable?

No single method is the most reliable for everything. Observation is the strongest evidence for practical skills, tests for knowledge and simulation for rare events. Reliability comes from matching the method to the competency, using trained assessors and corroborating with a second method.

Is a knowledge test enough to prove competence?

No. A test shows what someone knows, not what they can do. The Office of Rail and Road's guidance says questioning should only supplement other methods because knowledge alone cannot be assumed to infer competence. Pair tests with observation or a work product.

Is self-assessment a valid competency assessment method?

Self-assessment is useful for starting a conversation and planning development, but it should not stand as the decision. A systematic review in JAMA found most comparisons showed little or no relationship between doctors' self-assessments and external measures of their competence.

What is the difference between direct and indirect assessment methods?

Direct methods, such as observation, questioning and examining work products, give the assessor first-hand evidence. Indirect methods, such as witness testimony, candidate statements, discussion and simulation, rely on other people's accounts or a reproduced situation, so they usually need corroborating.

What does VARCS mean in competency assessment?

VARCS stands for valid, authentic, reliable, current and sufficient. Evidence should come from a suitable method, be the person's own work, lead different assessors to the same judgement, show the skill is still held and be enough to support the decision.

What is a professional discussion in competency assessment?

A professional discussion is a planned, two-way conversation in which an assessor asks the person to explain real examples of their work against the standard. It tests reasoning and judgement and is often underpinned by a portfolio of evidence, as in apprenticeship end-point assessment.

How do you assess competencies that rarely happen?

Use simulation, scenarios, drills or tabletop exercises, because waiting for an emergency is not a plan. Make the simulation as realistic as possible, including time pressure and communication protocols, and support it with a professional discussion of any real events the person has handled.

How often should competence be reassessed?

There is no single interval. Base it on risk, how often the task happens and how quickly skills fade, give every assessment an expiry date, and reassess early after an incident, a procedure change or a long absence. Record why you chose each frequency.

Can 360 feedback be used to assess competence?

360 feedback is good evidence of how behaviour is perceived by colleagues, which makes it useful for interpersonal and leadership competencies. It is not measurement against a standard, so treat it as one input alongside observation or discussion rather than as the assessment itself.

Who should carry out competency assessments?

Someone who knows the standard and is competent to assess against it, usually a trained line manager, supervisor or specialist assessor. Their own assessment should be current, and assessors should be calibrated against shared examples so different people reach the same decisions.

What should a competency assessment record include?

At minimum, the competency and level assessed, the methods used, the evidence or a reference to it, the assessor's name, the date, the outcome and the expiry or reassessment date. A record should let a second assessor understand why the decision was made.

---