Call calibration means your reviewers score the same calls, compare results, and agree on what each score means. Do it on a fixed schedule. Use observable criteria. Score alone before you talk. Write down every decision. Then feed what you learn into coaching. This guide shows each step.
TL;DR
- Calibration aligns evaluators on the standard. Coaching changes agent behavior. Keep them separate.
- Everyone scores the same calls alone before the meeting. Then you discuss the gaps.
- Meet weekly at rollout. Move to every two weeks once scores agree.
- Write criteria as behaviors a reviewer can see or hear. Vague items cause most disagreements.
- Record the date, the scores, the reasons, the final ruling, and an owner for each follow-up.
Table of Contents
- What call calibration is
- Who should join a calibration session
- How to run a calibration session, step by step
- How often to calibrate
- Write criteria reviewers can score the same way
- Measure agreement
- Common problems and how to avoid them
- Turn calibration findings into better performance
- Our view: calibrate the practice, too
- Practice what calibration finds
- FAQ
- Sources
What call calibration is
Call calibration is a shared exercise. Evaluators and supervisors score the same customer interactions. They then compare results so they apply quality standards the same way. SQM Group describes it as a way to keep performance assessments fair, consistent and accurate.
The goal is consistency. It is not to find a winner.
Without it, a score tells you who reviewed the call. It does not tell you what happened on the call. Coaching plans built on those scores become hard to defend. Agents notice. They stop trusting the scorecard.
Calibration versus coaching
The two are often mixed up. They solve different problems.
| Calibration | Coaching | |
|---|---|---|
| Who is in the room | Evaluators, supervisors, QA leads | One agent and one coach |
| What it fixes | Differences between reviewers | Agent behaviors that hurt results |
| Input | The same calls, scored by all | One agent's own calls and scores |
| Output | A shared reading of each criterion | A skill goal and a practice plan |
| Cadence | Fixed schedule | Ongoing, per agent |
NICE puts it simply: calibration makes evaluators use the same criteria, while coaching targets agent behaviors that affect KPIs. Calibration comes first. If the scores are not reliable, the coaching built on them is not reliable.
The two connect. Calibration can show gaps in training and unclear rubric items. SQM Group notes that it can also give agents feedback they can act on.
Who should join a calibration session
Include everyone who evaluates calls, gives performance feedback, or trains agents. That is the guidance in QATC's call calibration material.
ICMI suggests a team that includes QA, supervisors and leadership. Senior agents or trainers can help too. Name one leader for the process.
A practical group looks like this:
- QA analysts who score calls every day.
- Team leaders who coach from those scores.
- A trainer who knows what new hires were taught.
- A manager who can settle a ruling when the group cannot.
Keep the group small enough to talk. Six to eight people is workable. Larger groups drift into debate.
Multi-site centers need extra care. Benchmark Portal's quality monitoring paper says local point people keep calibration consistent across sites. They also keep quality teams, front-line leaders and agents working together.
How to run a calibration session, step by step
Follow the same steps each time. A fixed routine makes the sessions faster and the records comparable.
1. Set the goal and the facilitator
Pick one purpose for the session. It might be a new rubric item. It might be a drop in agreement last month. Name a facilitator. The facilitator keeps time and moves the group toward consensus. The facilitator does not hand down the "right" score.
2. Choose the calls
ICMI recommends preselecting several calls. They should represent different agents, call lengths and call types. Send recordings to reviewers ahead of time. If calls are not recorded, monitor live together.
Pick three to five calls. Include:
- One clear, strong call.
- One clear, weak call.
- One or two borderline calls where reviewers are likely to differ.
Borderline calls teach the most. Easy calls only confirm what you know.
3. Score alone first
Every participant scores every call before the meeting. No chatting about the calls beforehand. SQM Group's guide says participants should evaluate independently, use a facilitator, and set a scoring baseline.
Set a deadline for submitting scores. Late scores let people copy the group. That hides the real gaps.
4. Compare scores line by line
Start the meeting with item-level results. Do not start with total scores. Totals can match while the reasons differ. A reviewer who is too hard on greeting and too soft on compliance can still land on the same total.
For each item with a gap, ask each reviewer to point to evidence. That means a moment in the audio or a line in the transcript. Opinions without evidence do not count.
5. Focus on why scores differ
The best sessions ask why ratings vary. They do not try to prove one reviewer right. SQM Group calls for respectful, open discussion aimed at shared standards.
Useful questions:
- Which words in the criterion did you rely on?
- What would you have needed to hear to score it differently?
- Is the item unclear, or is the reviewer applying it loosely?
6. Decide and document
End each disputed item with a ruling. Then write it down. MangoApps' template lists useful fields: date, facilitator, call ID, rubric version, scores before discussion, disputed criteria, reasons for variance, final interpretation, participants, follow-up owner and due date.
This step is not optional. A session that reaches agreement but leaves no record has to be repeated. People forget. New reviewers never hear the ruling.
After the session, report results and any action plan to the wider team. ICMI's steps end the same way: decide whether action is needed, then share the outcome.
How often to calibrate
The guidance from QATC is weekly at rollout. Move to once or twice a week while you work toward your variance goal. Go to every two weeks once you reach it.
A simple schedule:
| Stage | Cadence |
|---|---|
| New program or new scorecard | Weekly |
| Scores still far apart | One to two sessions a week |
| Goal met and stable | Every two weeks |
| After any rubric change | One extra session |
| New reviewer joins | Calibrate them in their first week |
Add a session when something changes. A new product, a new policy, a new compliance rule or a new reviewer all shift how people read the rubric.
Write criteria reviewers can score the same way
Most calibration problems start in the scorecard. Vague items produce different scores. Fix the item, not the reviewer.
Follow these rules:
- Describe observable behavior. "Agent states the order total before asking for payment" can be checked. "Agent sounds confident" cannot.
- Define each score level. Say what a full pass, partial pass and fail look like.
- Weight items by business impact. A missed compliance statement should count more than a missed greeting.
- Use auto-fail for critical breaches. Reserve it for items such as skipping a required disclosure.
- Allow N/A. If a criterion does not apply to a call, reviewers need a way to say so.
- Require evidence. A timestamp, a short quote or a note makes every score auditable.
Voxjar's call monitoring guidance makes the same case: criteria based on observable behaviors, weights, auto-fail gates, N/A options and an evidence field. Their point is that disagreement often means a scorecard design problem.
If you want a worked example, see our guide to aligning quality standards to scorecards. For discovery-call criteria, see how to score questioning skills.
Measure agreement
A calibration program needs a number. Otherwise you cannot tell whether it works.
SQM Group recommends storing all QA ratings in one central database and calibrating on a regular basis. Its stated main goal is to keep actual QA scores within 5% variance of each other. Treat that as one reference point. Set your own target and write it down.
Track three things:
- Total score spread. The gap between the highest and lowest reviewer on each call.
- Item-level agreement. Which criteria cause the most splits.
- Reviewer drift. Whether one person scores steadily high or low over time.
Percent agreement can mislead. If almost every call passes an item, reviewers will agree often by chance. A chance-adjusted statistic such as Cohen's kappa gives a fairer picture. Your analytics team can calculate it from the same score sheets.
An erratic reviewer is a different case. If one person scores some calls high and others low with no pattern, a group discussion will not fix it. That person needs one-to-one supervision.
Common problems and how to avoid them
Groupthink
When the senior person speaks first, others follow. Prevent it with blind, independent scoring before discussion. The disagreements then show up on paper. Some will call for reviewer retraining. Others will call for a rubric rewrite.
Ambiguous items
If the same item splits the group three sessions in a row, the item is the problem. Rewrite it. Add an example call. Record the new version number.
Reviewer bias
Some reviewers favor friendly agents. Some punish accents or slow pace. Evidence requirements reduce this. So does rotating who reviews which agents.
No follow-through
Rulings that nobody owns fade. Give each action an owner and a due date. Open the next session by checking last session's actions.
Agents who do not trust scores
Allow appeals. Benchmark Portal lists agent appeals, where an agent can request a re-score by a different person, as a best practice. It builds fairness and buy-in for the quality program. Appeals also feed calibration: a disputed score is a ready-made session call.
Turn calibration findings into better performance
Calibration is a feedback loop. Benchmark Portal's paper says findings expose gaps in training and rubric clarity. Managers can then build targeted coaching plans or rewrite unclear items.
A practical loop:
- Calibrate and fix the rubric.
- Score agent calls with the clear rubric.
- Pick one or two skills per agent from the scores.
- Coach on those skills. Our guide on writing coaching notes shows how to phrase them.
- Let the agent practice before the next live call.
- Re-score and track the change.
Step five is where most programs stall. Agents hear the feedback but have no safe place to try again. Role-play fills that gap. See these call center scenario examples and these mock call scripts for practice material.
To track results over time, use a simple dashboard. Our post on agent performance dashboards walks through a three-step setup. For the metrics that matter most, read about call center performance metrics.
Our view: calibrate the practice, too
We build practice tools for sales and support teams. Here is what we have learned from that work.
Scoring standards should be the same in review and in practice. If agents rehearse against one standard and QA grades them against another, they learn to guess. The rehearsal should use the rubric your QA team already trusts.
That is why we built calibration-aligned scoring into Call Flow. The platform grades practice calls across five performance dimensions. It gives feedback right after the call. Supervisors can review and override a score. Then the grading follows your standard, not ours.
We also think calibration should stay human. A tool can speed up scoring. People still decide what "good" means for your customers. Use automation to give reviewers more calls to look at and less repeat work. Keep the rulings with your team. If you want to test automated scoring, our six-step pilot guide shows a careful way to start.
Finally, keep it small. One clear rubric, one session a week, one written record. Teams that do that beat teams with a complex program they skip.
— Costa
Practice what calibration finds
Calibration shows where agents need work. Call Flow gives them a place to do it. Agents run realistic voice or text role-plays. They get instant grading and coaching. Supervisors see the results in a dashboard.
Plans are billed per team. Starter is $7.42 per month or $89 per year. Growth is $14.92 per month or $179 per year. Scale is $33.25 per month or $399 per year. See all plans on the pricing page. Larger teams can look at the Business and Enterprise options. A free full-feature trial is available.
FAQ
What is the difference between call calibration and a call review process?
A call review process is the day-to-day scoring of agent calls. Calibration is the check on that process. Reviewers score the same calls and compare. The aim is to make sure everyone applies the scorecard the same way.
How many calls should I use in a calibration session?
Use a small set. Three to five calls is workable for a one-hour meeting. ICMI advises choosing calls that represent different agents, lengths and call types. Include at least one borderline call.
Can AI replace calibration sessions?
No. AI scoring can apply a rubric the same way on every call. People still write the rubric and decide what is correct. You still need sessions to review disputed calls and update the standard.
Who makes the final decision when reviewers disagree?
The facilitator guides the group toward consensus. If the group cannot agree, a named QA or operations lead makes the ruling. Write down the reasoning and the rubric version so the decision holds next time.
How long does it take to see better scoring agreement?
That depends on your starting point and how often you meet. Weekly sessions at rollout are the usual advice. Track the score spread per session. You should see it narrow as unclear items are rewritten.
Sources
- SQM Group: Call Calibration Guide
- ICMI: Conducting Effective Quality Calibrations
- QATC: Call Calibration conference material
- Benchmark Portal: Best Practices in Quality Monitoring and Coaching
- NICE: Call Center Quality Assurance Guidelines
- MangoApps: QA Calibration Session Documentation Template
- Voxjar: Call Monitoring Forms