How Do I Measure Recommendation Rate When Claude Gives Conditional Advice?

What does a conditional answer look like?

"For a small agency that wants simple task boards, [A] is the easiest option. If you need detailed time tracking and client billing, [B] is stronger. Larger teams with complex portfolios usually choose [C]."

That answer contains three branches, each with a condition and a recommendation. Scoring it as "mentioned" or "not mentioned" misses who wins which buyer. Conditional answers are common across assistants when questions leave room for different needs, and Claude often lays out trade-offs this way.

How should you code each answer?

For every answer, record:

Answers that name no one belong with no-recommendation outcomes, as covered in what refusal means in AI search and how it differs from a no-recommendation.

Which metrics should you report?

• Unconditional recommendation rate: among answers that pick one option outright, the share that pick you.
• Branch presence: among conditional answers, the share where you appear in at least one branch.
• Branch share: the share of all branches that name you.
• Matching-branch win rate: among conditional answers with a branch for your ideal customer's need, the share where that branch recommends you.
• Mismatch rate: the share of your branches that place you with a buyer you do not serve well. It shows positioning errors.

The matching-branch win rate is usually the most useful single figure. It answers the question you care about: when the assistant considers buyers like your customers, does it send them to you?

What does a worked example look like?

Consider an illustrative project management tool for agencies, tested with 100 answers from Claude:

A simple "mentioned" rate would be 40 of 100, or 40%, which says little. The breakdown shows something more useful. The tool is well placed with agencies, winning 40% of the branches that matter. Its unconditional pick rate, 5 of 20 or 25%, is lower, but 20 answers is too few to be sure: the 95% margin is about ±19 points. A rate measured on n answers has a margin of about 1.96·√(p(1−p)/n). The gap between two rates is less certain than either rate: its margin is about √(m1² + m2²), roughly 1.4 times a single margin when the groups are the same size. Here, 25% (20 answers) against 40% (50 answers) would need a gap of about 23 points to be clear, so collect more single-pick answers before acting on it. That points to making the agency fit even more explicit, and to earning the independent comparisons assistants use when choosing a single winner.

How should prompts be written to test branches?

• Keep open prompts, such as "best project management tool", to see how assistants split the market and which branch you are placed in.
• Add need-specific prompts, such as "best project management tool for a 10-person creative agency that bills clients by the hour". These collapse the conditions and test the branch you care about directly.
• Include prompts for adjacent segments to check for mismatch, such as enterprise teams if you serve small agencies.
• Keep the set fixed, so changes in branch outcomes over time are real, as covered in what a golden prompt set is and why to keep it fixed.

How do you label conditional answers consistently?

Write short rules: what counts as a branch, how to handle "also consider" mentions, and when a condition matches your ideal customer. Have two people label a sample and compare, then refine the rules. If you use an AI model to label at scale, give it the rules and examples, and check a sample by hand, as covered in human evaluation versus LLM-as-a-judge for measuring AI recommendations.

What mistakes should you avoid?

• Counting any branch as a recommendation. Being recommended "for enterprises" when you serve small teams is a positioning problem, not a win.
• Ignoring the condition. The condition tells you why you were chosen. Record it.
• Mixing answer types into one rate.
• Comparing assistants without separating answer types, since some assistants give conditional answers far more often than others.

What might an illustrative case look like?

Consider an illustrative accounting software company whose tracking showed Claude "recommending" it in 45% of answers. Branch coding showed most of those mentions placed it under "if you are a larger firm with complex consolidation needs". Its target market was small practices. In the small-practice branch, it won only 12% of answers. The company rewrote its homepage and product pages around small practices and published a comparison page for small firms. It also asked every customer for an honest review in their own words. Four months later, its small-practice branch win rate had risen to 31%, and mismatched enterprise branches had fallen.

What do people ask about conditional recommendations?

Is a conditional mention better than nothing?

Usually, if the condition fits your customers. A mention under the wrong condition can send you the wrong leads.

Should we try to make assistants give single picks?

You cannot control that. Test need-specific prompts to see whether you win when the buyer's needs match yours.

Does this apply to other assistants?

Yes. The same coding works for ChatGPT, Gemini and Perplexity, which also give conditional answers to open questions.