The Bench Test: Tool Map for Comms Teams
A repeatable blind comparison run that settles which AI tool your team should use for which recurring comms task, and turns the answer into a one-page Tool Map that stays current.
Keep in mind A blind test of three tasks is a snapshot: check that the brief was identical for every tool and that no default rests on a narrow result before you pin the map up.
Version 1.1 · Updated 30 October 2026
What you'll get
The opening of a worked example. Anonymised and shortened; yours will follow your inputs.
Fictional example.
Bench Test · In-house comms team of six · July 2026
Tools: Assistant A (team plan) · Assistant B (built into our Microsoft 365 tenant) · Assistant C (built into our Google Workspace)
| Task | Winner | Margin | Note |
|---|---|---|---|
| 300-word internal announcement | Assistant A | Clear (5 of 5 scorers) | "Least editing. The others both wrote like a press release." |
| Monday media summary from 20 clips | Assistant B | Clear | Won mostly because the clips already live in the tenant. Nobody had to copy anything out. |
| Strategy doc to five LinkedIn posts | Tied | Narrow | Two scorers picked Assistant C, two picked Assistant A, one would not have sent any of them. No default set. |
Surprise: the team believed Assistant B was "the weak one". Blind, it won a task outright, because proximity to the data beat raw writing quality.
Next step
The prompt ends with questions for you. Answer them with this, in the same chat.
Here are my answers to your questions: [YOUR ANSWERS] Revise the draft using them. Remove the [Assumed] and [Confirm] marks my answers resolve, keep the rest, and update the questions and checks.
Choosing tools for a particular job? Start with the guide for your comms stream, organised job by job and re-tested every three months.
Human review checklist
Go deeper
Variations
Designing the run
Use this when you have not run the test yet and want to plan it.
I have not run the test yet, so ignore the results lines above. Our team does these tasks most weeks: [SIX TO EIGHT RECURRING TASKS, WITH HOW OFTEN]. Instead of the six sections, write: a recommendation of three tasks to test, chosen for frequency, with one line on why each other task is a weaker choice; then a run sheet table with the columns # | Task | Why this task | Brief to use (anonymised) | How often we do it; then a blind scoring sheet table with the columns Output | Would I send this with light editing? (Y/N) | Any invented facts? (Y/N) | Sounds like us? (1 to 5) | Time it would save me (mins) | Comment. Then what would make a brief favour one tool, and the most common ways a test like this goes wrong. Remind me to remove names, embargoed detail and personal data from every brief.Re-test
Use this when you already have a Tool Map and something has changed.
We already have this Tool Map: [PASTE THE MAP, WITH ITS TEST DATE]. Since then: [WHAT HAS CHANGED, SUCH AS A MODEL UPDATE, A NEW TOOL OR A BAD OUTPUT THAT GOT THROUGH]. Before the six sections, add a section headed "What has changed": which defaults are most likely out of date and why, whether a full re-run is needed or one task is enough, and any task that is now frequent enough to add. Judge only from what I have told you has changed.Full example output
Example output
Anonymised and shortened. Real outputs will vary based on your inputs.
Fictional example.
Bench Test · In-house comms team of six · July 2026 Tools: Assistant A (team plan) · Assistant B (built into our Microsoft 365 tenant) · Assistant C (built into our Google Workspace)
| Task | Winner | Margin | Note |
|---|---|---|---|
| 300-word internal announcement | Assistant A | Clear (5 of 5 scorers) | “Least editing. The others both wrote like a press release.” |
| Monday media summary from 20 clips | Assistant B | Clear | Won mostly because the clips already live in the tenant. Nobody had to copy anything out. |
| Strategy doc to five LinkedIn posts | Tied | Narrow | Two scorers picked Assistant C, two picked Assistant A, one would not have sent any of them. No default set. |
Surprise: the team believed Assistant B was “the weak one”. Blind, it won a task outright, because proximity to the data beat raw writing quality.
Tool Map v1.0, expires three months after the test. Internal announcements: Assistant A. Media summaries: Assistant B. Social repurposing: no default, human first draft. Confidential or embargoed: only tools approved for that data classification and use.
Illustrative example. Your results will differ, and that is the point of running it rather than reading about it.
The template
Part 1: The run sheet
Bench Test run: [Date] Run by: [Name] Tools in the test: [List every sanctioned assistant, with the exact plan and model version, e.g. “Claude, Team plan, Sonnet”] Scorers: [Names and roles]
| # | Task | Why this task | Real brief used (anonymised) | How often the team does it |
|---|---|---|---|---|
| 1 | [e.g. Draft a 300-word internal announcement] | [x per week] | ||
| 2 | [e.g. Summarise 20 media clips into a Monday note] | |||
| 3 | [e.g. Turn a strategy doc into five LinkedIn posts] | |||
| 4 | [Optional] | |||
| 5 | [Optional] |
Rule: the same brief goes into every tool, unchanged. No prompt tuning per tool. You are testing the tools as your team actually uses them.
Part 2: The blind scoring sheet
The runner strips every output of anything identifying the tool, labels them A, B, C, and shuffles the order for each task. Scorers do not know which is which. The runner does not score.
Task [1] scoring sheet
| Output | Would I send this with light editing? (Y/N) | Accuracy: any invented facts? (Y/N) | Voice: does it sound like us? (1-5) | Time it would save me (mins) | One line of comment |
|---|---|---|---|---|---|
| A | |||||
| B | |||||
| C |
Ranking: 1st [ ] · 2nd [ ] · 3rd [ ]
The first column is the one that matters. Everything else is context for it.
Why people, not an AI, should score. When eight AI assistants judged each other’s work blind in the Comms AI Pentathlon 2026, their rankings matched the AI consensus far more closely than a human judge’s did. Measured the same way, the AI judges averaged 0.62 to 0.92 against the consensus and the human judge 0.44: closest on the long strategy brief (0.88), furthest apart on the 150-word media pitch (0.07). The judging prompt is in that record’s download if you want an AI second opinion, but keep the people who would send the work as your scorers.
Part 3: The reveal
The runner unmasks the labels and completes the tally.
| Task | Winner | Margin | Any tool that invented facts | Notable disagreement between scorers |
|---|---|---|---|---|
| 1 | Clear / narrow / tied | |||
| 2 | ||||
| 3 |
Where the result surprised us: [The value of the blind method is that it sometimes contradicts what the team believed. Write down where it did.]
Where the margin was narrow: [If two tools tied, say so. A narrow win is not a mandate, and forcing a default where none exists loses you credibility.]
Part 4: The Tool Map (the actual deliverable)
One page. Pin it wherever the team looks. This is what the Bench Test exists to produce. Copy the block below and fill it in.
[Team name] Tool Map
Version [1.0] · Tested [date] · Expires [date + 3 months]
| If the task is… | Default tool | Because | Never use it for |
|---|---|---|---|
| [Task 1] | [Tool] | [One line from the test, e.g. “won on voice, needed least editing”] | [Red line] |
| [Task 2] | [Tool] | ||
| [Task 3] | [Tool] | ||
| Anything confidential or embargoed | [The tool covered by your organisation’s data agreement] | It is the only one our data agreement covers | Any personal or free account |
| Anything not on this list | Ask [named person] |
Standing rules
- A named human approves anything this produces. The tool does not sign off its own work.
- If a tool invents a fact once, it is off that task until the next Bench Test.
- This map expires on [date]. After that it is unverified, and should be treated as such.
Owner: [Name] · Next Bench Test: [Date]
Part 5: Feeding it back
The Tool Map has two jobs after the run:
- It updates the tool register. If you hold an AI use policy or a tool register, the Bench Test is the evidence behind the tool section. Change the register, note the date, note what changed.
- It closes the shadow-AI gap. If a tool your team is not permitted to use keeps beating the sanctioned one in blind tests, you have learned something important, and suppressing it will not make it untrue. Take it to whoever owns the licensing decision, with the scores.
When to use it
What it is
In some comms teams the hard part is choosing between tools. Several assistants may already be paid for and sitting on the desktop, and each person quietly defaults to whichever one they learned first. The result is that the same task gets done four different ways, quality varies for reasons nobody can name, and when a leader asks “which tool are we using for press releases?” the honest answer is “it depends who’s writing it.”
The Bench Test settles that. You take the tasks that actually fill your team’s week, run the same brief through every sanctioned tool you have, strip the labels, and let the people who do the work rank the outputs without knowing which model produced which. Preference and brand loyalty drop out. What is left is evidence.
The output is a Tool Map: a single page saying which tool is the default for which task, what it is not to be used for, and when the answer expires. That page is what your team actually uses, and it is what keeps your tool register from quietly going out of date.
This is deliberately lighter than the AI Tool Evaluation Framework, which is for procurement: should we buy this. The Bench Test is for the tools you already have: which one, for what, and is that still true.
When to use it
Use this template when:
- Your team has more than one AI assistant available and no agreed rule about which does what
- You suspect people are using AI well but cannot evidence which tool is producing the good work
- A model has just been updated and you want to know whether your defaults still hold
- You are building or refreshing an AI use policy and need the tool section to be based on something
- Shadow AI is showing up and you want the sanctioned tool to win on merit rather than by decree
Don’t use this template when:
- You are choosing whether to buy a tool you do not yet have (use the AI Tool Evaluation Framework)
- You have only one sanctioned assistant (in which case the Tool Map is one line, and that is fine)
- The task in question is confidential enough that it cannot be de-identified for testing
Inputs needed
- Three to five tasks. Real, and done most weeks. Choose the frequent task over the interesting one. Fewer than three gives you too little to go on; more than five makes the scoring hard to finish.
- One real brief per task. Anonymise it. Strip names, personal data, embargoed detail and anything you would not want in a tool you are testing.
- Your sanctioned tools. Only the ones the team is permitted to use. Testing an unapproved consumer account teaches you something you cannot act on.
- Three to six scorers. The people who do the task, rather than those who decide about it.
- A runner. One person who holds the answer key and does not score.
Tips for success
Pick the boring tasks
The temptation is to test the tools on something ambitious. Resist it. The value of a Tool Map is in the tasks your team does many times a month.
Normalise the formatting before scoring
Models have tells. One loves bullet points, another leans on headers, another has a punctuation habit. If you do not strip these, your blind test turns into a quiz about which model your team can recognise.
Let the result be inconvenient
Some of the most useful Bench Tests are the ones that contradict the team’s assumptions, including the assumption held by whoever chose the tools. If the run only ever confirms what you already believed, check whether the method is working.
Run it quarterly
Monthly sounds diligent, but twelve runs a year is a lot to ask of a busy comms team, and a well-chosen default may not need changing every four weeks. Quarterly gives you a regular check on drift at a pace the team can keep up. Re-test early only when something forces it: a model update you can feel, a new tool, or an output that should not have got through.
Set the expiry short anyway
Three months, and put the date on the page. A Tool Map that outlives its evidence is worse than no map, because people trust it.
Make it fast enough to repeat
If every run takes as long as the first, the third may never happen. Keep the same three tasks and the same briefs so the re-run is quicker.
Common pitfalls
Testing tools nobody is allowed to use
It is interesting to know that an unapproved consumer account writes a better announcement. It is not actionable, unless you take it to the person who owns the licensing decision. If you are not going to do that, do not test it.
Confusing the tool with the prompt
A tool that loses badly on a thin brief may win on a good one. That is a real finding about your briefing. Note it, and consider whether the honest conclusion is “our briefs are the problem”.
Setting a default on a tied result
A narrow win is a coin toss with extra steps. Say “no default” and let people choose. An honest gap is easier to live with than a rule that turns out to be arbitrary.
Treating the map as permanent
A Tool Map can be produced, pinned up, admired, and never re-tested. Six months later it is quietly wrong, and it has more authority than it deserves. The expiry date is not decoration.
Doing it once and calling it governance
One Bench Test is a snapshot. Governance is the cadence. If nothing in your calendar says when the next one happens, this was an away day; the cadence is what makes it a system.
Related templates
AI Tool Evaluation Framework
A structured framework for evaluating, comparing, and selecting AI tools for communications work, moving from vendor claims to practical assessment against real workflows, team needs, and risk criteria.
AI Use Principles
A short, foundational statement of how your organisation uses AI in communications: the principles that sit above the operating policy and inform every decision about where AI is and is not trusted.
Comms Workflow Audit
A structured framework for mapping existing communications workflows, identifying inefficiencies and bottlenecks, and pinpointing where AI can add the most value, before buying tools or redesigning processes.
Applied
Tested in practice
From Applied Comms With AI
The Comms AI Pentathlon 2026: eight AI assistants, five comms briefs, 320 blind scoresExperiment
The AI you were given finished near the bottom. 8 assistants, five comms briefs, 320 blind scores: the medal table, and how to run your own test.