The Source Test: six AI operating system demo questions for auto shops

Who this checklist is for

This is for the owner, general manager or operations lead of an auto repair shop or a multi-location group who has a demo on the calendar for software that calls itself an AI operating system, an AI assistant or an AI agent platform. It goes with our short video on the Source Test and with the longer article on what an AI operating system for auto shops should do.

The idea behind it is simple. A real operating layer sits on top of the software you already run and reads from it. So it should be able to show you, on screen, where any fact came from. A tool that can't is guessing, however smart the answers sound.

You don't need to be technical. You need one real item from your own shop in mind, a printed copy of the scoring sheet at the bottom, and permission to interrupt the scripted part of the demo.

What to have ready before the demo

• One real item you know well. An estimate that has been open about a week, a repair order stuck in one stage, or a review that never got a reply. You know the true answer, so you can tell when the tool is wrong.
• A list of the systems you run. Your shop-management system, texting tool, accounting package, review platform and phone system. Ratchet+Wrench's 2026 industry survey found electronic management system use at 59%, so most shops have at least one system of record that any operating layer has to read.
• The person who would approve messages. Bring your service advisor or manager if you can. They will spot a draft that would never go out under their name.
• Fifteen minutes that belong to you. Tell the vendor up front that you want to drive part of the demo.

The three questions from the video

These come first because they are the fastest way to separate a layer that reads your data from a chatbot with a logo on it. Ask them in this order.

1. Source ("Show me where this one fact came from")

Point at a single item on the screen, ideally an open estimate, and ask: Where did this come from, and when was it last updated?

A good answer shows a source line on the item itself: the system it came from, the record number, and a last-updated time. The vendor can click through to the original record or show the same value in your shop-management system.

Red flags: - "It's pulled from your data" with nothing on screen to prove it. - A summary with no record number you could check. - A last-updated time that is missing, or that is the time the demo was loaded rather than the time the record changed.

Ask it twice, on two different items. Demo data can be polished for the first one.

2. Empty field ("What happens when this is blank?")

Pick a field that is often missing in real life, like a parts ETA, a callback number or a promised time, and ask: If this field is empty in my system, what does your screen show?

A good answer is that the item says so in plain words, like "Not recorded", and doesn't fill the gap. Better still, the item is flagged so someone can go and find the missing fact.

Red flags: - The tool shows a confident estimate ("Thursday") with no source. - The vendor says the AI "infers" the value but can't show you how or from what. - The empty field simply disappears from the screen.

A field that quietly gets filled with a guess is the most expensive kind of mistake, because nobody knows to check it.

3. Approval ("Who has to say yes before this goes out?")

Find a draft, like a follow-up text on that open estimate, and ask: Who approves this, and where can I see that they did?

A good answer names a person or a role, shows buttons to edit, approve or dismiss, and keeps a record of who approved what and when. Anything that touches money or your reputation, such as price changes, discounts, refunds and public review replies, starts as a draft.

Red flags: - Messages go out automatically by default and you have to turn approval on. - There is no record of who approved a message. - The vendor can't say what happens when nobody approves a draft.

In its 2023 guidance the Federal Trade Commission told companies to keep their AI claims in check and not overstate what a product does, as a Kelley Drye summary of that guidance explains. An approval step with a record is how you keep your own customer messages honest too.

Three more questions to finish the demo

Once the first three pass, these tell you whether the tool fits your business and what it will take to try it.

4. Reads and writes ("Which of my systems does it touch today?")

Ask for a plain list: which systems it reads, which it writes to, and which are only on a roadmap. A logo wall is not an answer. Ask what happens if one connection stops syncing overnight, and whether the screen tells you.

5. Locations ("What changes between one shop and five?")

If you run more than one location, ask to see two shops side by side. Each item should carry its location, and the same rule should run the same way at every store. Ask who at each site sees what.

6. Pilot ("What does a first project look like, and what does it cost?")

A serious vendor can describe a small first project on one real workflow, with a start, an end and a way to judge it, even if the price is custom. Be wary of a long contract before you have seen the tool run on your own data.

How to score the demo

Print this and mark each question during the call.

#	Question	Pass when	Your mark
1	Source	Every item shows its system, record and last-updated time
2	Empty field	Missing data is labeled, never guessed
3	Approval	A named person approves drafts, with a record
4	Reads and writes	A plain list of live connections, not a roadmap
5	Locations	Location on every item, one rule at every store
6	Pilot	One workflow, a clear end, judged on your data

A tool that fails any of the first three is not reading your shop the way it claims, whatever else it does well. Questions four to six decide fit and timing.

How to tell the Source Test worked

You walk out of the demo able to say three things in one sentence each: where the tool gets its facts, what it does when a fact is missing, and who signs off on anything that goes to a customer. If you can't, the demo stayed on the vendor's script.

It also helps to frame the risk questions the way the NIST AI Risk Management Framework does: who is accountable, what the tool touches, how its output gets checked, and what happens when it's wrong. The Source Test is a shop-sized version of those four questions.

One more check for later: reviews. BrightLocal's 2026 survey found 89% of consumers expect businesses to respond to reviews, so if the tool drafts review replies, run the approval question on a real one-star review before you sign anything.

Where RevvOS fits

RevvOS is built as this kind of layer for independent owners and multi-location groups, and it is designed to pass these six questions: every item carries its source, missing data is labeled, and drafts wait for a named person. See how it works across stores on the multi-location page, or book an assessment to run the Source Test on your own shops.