Questioning Skills Assessment: How to Score Agent Discovery

A questioning skills assessment shows whether an agent can find out what a customer needs before offering a fix or a pitch. Score the quality of the diagnosis. Do not score the number of questions. Use a short rubric, a realistic call scenario and trained raters. Repeat it over time.

This guide shows you how to build that assessment for sales and contact-center teams.

TL;DR

• Score five behaviors: fact finding, problem discovery, impact exploration, outcome questions and active listening.
• Score each behavior on its own. One blended score hides the weak spot.
• Use observable evidence in the rubric. Skip vague traits like "good communicator."
• Calibrate raters before you use scores for coaching, promotion or pay.
• Repeat the assessment. One score is a snapshot. A trend shows learning.

Table of Contents

• What a questioning skills assessment measures
• Five behaviors to score
• Open and closed questions: score the fit, not the form
• Look at depth, not only type
• Build the scenario first
• A sample rubric you can copy
• A worked example
• Make the scores defensible
• Use it for hiring and for coaching
• Run a one-week pilot
• Our view: scoring by hand does not scale
• Try it on your own team
• FAQ
• Sources

What a questioning skills assessment measures

Most pages on this topic come from teaching. They cover classroom questions and student thinking. The ideas carry over. But a sales or support call has a different goal. The agent must diagnose a need, then move to a solution.

So the assessment answers one question: did the agent understand the customer well enough to help?

That means you look at what the agent asked, in what order, and what the agent did with the answers. A long list of questions can still be a bad diagnosis. An agent who asks six scripted questions and ignores the answers has not shown skill.

Five behaviors to score

A sales competence assessment developed for a financial services firm is a useful model. Its example report gives separate scores for questioning effectiveness (38%) and active listening (57%). The two skills move apart, so score them apart.

Use these five behaviors:

1. Situation and fact finding. The agent gathers the basics: role, setup, history, volume.
2. Problem discovery. The agent finds the difficulty or dissatisfaction behind the request.
3. Impact exploration. The agent asks what the problem costs: time, money, risk, frustration.
4. Outcome questions. The agent gets the customer to say what a solution would be worth.
5. Active listening. The agent paraphrases, clarifies and confirms what the customer said.

The first four follow the order of the SPIN framework: situation, problem, implication, need-payoff. A consultative selling handout from Northern Arizona University describes the sequence. It warns against overusing situation questions. It also says implication questions matter most in complex consultative sales.

Why this matters: an agent who stops at situation questions collects facts. But the customer may never feel urgency or see the value of fixing the issue.

For support calls, rename the behaviors. "Problem discovery" becomes "issue isolation." "Impact" becomes "what the issue has already cost the customer." "Outcome" becomes "what a good result looks like to the customer." The logic stays the same.

Open and closed questions: score the fit, not the form

A common mistake is to reward only open questions. The Right Question Institute defines closed questions as those answered with yes, no or one word. Open questions need an explanation. Both have uses.

On a customer call:

• Use a closed question to confirm a fact. "Is the error still showing on your screen?"
• Use an open question to find context, causes and priorities. "What happens to your team when that report is late?"

Score whether the form matched the purpose. An agent who asks "Do you like the current process?" gets a one-word answer and learns little. An agent who asks "Is that correct?" after summarizing the issue is doing the right thing.

Look at depth, not only type

Teaching research gives a second lens: cognitive level. The University of Chicago's teaching center lists four common levels: knowledge, comprehension, application and analysis. It notes that observational studies found as many as 90% of teachers' questions focused on low-level recall.

That figure is about teachers, not agents. But the lesson transfers. People often cannot see their own pattern. The center suggests recording a session, sorting the questions by level and repeating the audit to measure change.

For a call, a simple version has three levels:

An agent whose questions all sit in the first row is gathering data, not diagnosing.

Build the scenario first

The test is only as good as the situation you put the agent in. Avoid a knowledge quiz ("Name the four SPIN question types"). It checks memory. It does not check behavior.

Use scenario-based items. Osaka Metropolitan University's Qi assessment is built this way. It has 6 large questions and 18 items, and it allows 85 minutes including the tutorial. It tests reasoning, not recall. You can borrow the design in a shorter form: a realistic customer situation, then one or more valid follow-up questions, then a scored rationale.

Good scenarios share these traits:

• The customer gives an incomplete first answer. This tests follow-up.
• The real problem differs from the stated request. This tests diagnosis.
• There is a hidden cost the customer has not named. This tests impact questions.
• The customer changes direction mid-call. This tests adaptation.

The University of New South Wales teaching guidance says effective questioning needs clear, specific wording, varied levels and follow-up on responses. It also advises planning the overall direction while staying responsive. On a call, that means the next question should depend on what the customer just said. If the agent reads a fixed list, mark it down.

For ready-made ideas, see our posts on role play scenarios that cut ramp time and call center scenario examples with a five-step rubric.

A sample rubric you can copy

Use a 0 to 10 scale for each behavior, or a 1 to 5 scale if you prefer fewer steps. A Florida State University sales role-play judging form uses 0 to 10. On that form, 0 means the skill was not shown, 5 is typical of a new, inexperienced salesperson and 10 is typical of an experienced one. Anchoring the middle of the scale to a real person type helps raters agree.

That form also shows how questioning fits inside a full call. It gives 15% of the score to needs confirmation, 30% to presentation, 15% to objection handling, 20% to gaining commitment, 10% to professional communication and 5% each to the opening and credibility. Within needs confirmation, it looks at probing for additional needs, exploring consequences and costs, summarizing issues and confirming interest.

Here is a questioning-only rubric built on the same idea. Each row lists evidence a rater can see or hear.

Two rules keep the rubric honest:

• Use observable behavior. Paraphrasing, clarifying, probing and confirming are evidence. "Good communicator" is not.
• Score diagnosis quality, not volume. Look for relevant questions, follow-up and a clear progression that helps the customer see the problem.

For more templates, read our role play assessment rubric guide.

A worked example

Take a software support call. The customer says: "Your export keeps failing."

Agent A asks: "Which browser are you using?" Then: "Have you cleared your cache?" Then offers a fix. Three questions, all recall. No follow-up, no impact, no summary.

Agent B asks: "What happens when the export fails?" The customer says it stops at 80%. Agent B asks: "How often does that happen?" Then: "What does your team do when it fails?" The customer says they rebuild the report by hand before a Monday meeting. Agent B summarizes: "So the export fails about half the time, and your team loses a few hours each Monday. Is that right?"

Agent B asked fewer technical questions and still scored higher. The agent found the impact, summarized it and confirmed it. A question count would have ranked A and B about the same.

Make the scores defensible

If scores affect hiring, promotion or pay, the method must be fair.

Tie questions to the job. A University of North Carolina interview development guide builds questions from the knowledge, skills and abilities the job requires. It also pairs each question with a follow-up and scores with a consistent method. Every candidate is then judged on the same competencies, not on a rater's general impression.

Calibrate your raters. A 2019 study in Language Assessment Quarterly looked at contact-center subject matter experts rating customer service reps. The experts reached high intra-rater and inter-rater reliability after a week-long training program with a rubric. The study covered English language skills in Indian contact centers, not questioning. Still, the point holds: leaders can rate reliably, but only after training.

A simple calibration routine:

1. Pick three recorded calls: one weak, one average, one strong.
2. Have every rater score them alone.
3. Compare scores row by row.
4. Discuss every gap of two points or more.
5. Rewrite any rubric line that caused the gap.
6. Repeat until raters land within one point.

Our post on contact center QA and scorecards covers how to align a new rubric with the QA standards you already use.

Use it for hiring and for coaching

The same rubric works in two places, with different stakes.

Hiring. Give candidates a short role-play. Score the five behaviors. Compare candidates on the same scenario. Our sales skills assessment shortlist covers the hiring side in more detail.

Coaching. Run the assessment at the start, then again every few weeks. Pick the lowest behavior for each agent and coach that one. Do not coach all five at once. Our sales call frameworks playbook includes a 30 minute discovery plan you can use in coaching sessions.

Watch for a common trap. Agents who believe questions are "intrusive" ask too few. The FinxS example describes this pattern: no structured approach to asking questions, and a drift into "show and tell." A structured question path fixes it better than a reminder to "ask more."

Run a one-week pilot

Do not wait for a perfect rubric. Start small.

1. Day 1. Pick one scenario and the five behaviors. Write the rubric.
2. Day 2. Calibrate two or three raters on recorded calls.
3. Days 3 and 4. Have 5 to 10 agents run the scenario.
4. Day 5. Score, compare and pick one coaching focus per agent.
5. Week 2 onward. Repeat the same scenario to see change.

Our guide to piloting sales call simulation in one week covers how to set baseline scores.

Our view: scoring by hand does not scale

We build Callflow, so we see the same problem again and again. A manager designs a good rubric. Then two things break it.

First, time. Listening to calls and scoring five behaviors takes longer than most managers have. So the assessment happens once, and the trend never appears.

Second, consistency. Two raters hear the same call and score it differently. The agent gets mixed messages.

That is why we built instant grading across five performance dimensions, with coaching feedback right after each practice call. Agents can practice by voice or text. Supervisors keep a review and override workflow, so a person still has the final say. Scoring is calibration-aligned, so it can match the QA standards a team already uses. Teams can also build custom scenarios for their own customers and products.

Our advice: write the rubric by hand first. Test it with people. Then automate the scoring only when you trust the rubric. A bad rubric run fast is still a bad rubric.

— Costa

Try it on your own team

Callflow lets you run the questioning assessment above as a role-play, with instant scoring and feedback. You can try all features during a free trial.

Plans on the pricing page are published flat-rate:

Business and Enterprise plans have no published price. See the enterprise page for details.

FAQ

How do you assess questioning skills?

Give the person a realistic scenario and record the conversation. Score it against a rubric with observable behaviors: fact finding, problem discovery, impact questions, outcome questions and active listening. Use trained raters, and repeat the assessment over time to see change.

How can agents improve their questioning skills?

Pick the lowest-scoring behavior and practice only that one. Use short role-plays with a customer who gives incomplete answers. Review the recording, sort each question by type and depth, and repeat. Our post on sales role play games has formats to try.

Are open questions always better than closed questions?

No. Closed questions confirm a specific fact. Open questions uncover context, causes and goals. The skill is choosing the form that fits the purpose at that moment.

How many questions should an agent ask on a call?

There is no right number. Count relevance and follow-up, not volume. Research on assessing sales competence points to relevant questions, follow-up on incomplete answers and a clear progression as the signs of skill.

Sources

• FinxS sales competence assessment development
• Consultative selling and SPIN questions, Northern Arizona University
• What is the QFT? Right Question Institute
• Asking Effective Questions, University of Chicago
• Qi assessment, Osaka Metropolitan University
• Questioning, UNSW Staff Teaching Gateway
• Florida State University sales role-play judges form
• Interview Question and Scoring Development Guide, UNC
• Can Subject Matter Experts Rate CSRs at Work in Indian Contact Centres? Language Assessment Quarterly, 2019

Recommended

• Practice First: Probing Questions in Sales, 15–16 per Call for Reps
• Mock Call Scripts That Actually Sharpen Agent Skills
• Sales Skills Assessment: A Hiring Manager's Shortlist
• Pilot Automated Call Scoring in 6 Steps for Managers