How Should a Medical Practice Evaluate an AI Vendor Before It Buys?

Start with your own number before you take the first demo: name the one result the tool has to move and measure it on your practice’s own data, then judge every vendor against it. Then get in writing what MGMA tells practices to require of any AI vendor, including evidence of performance and known failure modes relevant to your specialty and patient mix, whether your data can be used to train their models, and how model updates are communicated.

That answer comes from Serious AI Partners, an enterprise AI implementation firm run by Chris Duff, M.D., who trained as a physician at the University of Illinois College of Medicine, spent 2017 through 2019 in healthcare venture capital, and built his judgment in a business where being wrong cost him money. The tools in question here sit on the operating side of a practice (phones, scheduling, documentation, billing), and none of this is clinical advice.

Why is this so hard to do right now?

Before buying a ticket to the 2026 annual conference of the Medical Group Management Association (MGMA), the oldest and largest membership organization representing group practice administration, Chris had Claude pressure-test the idea, and it screened all 171 of the conference’s vendors for direct competitors. The list included plenty of AI software firms, some with $30M to $50M Series A rounds behind them, selling narrow tools into specific corners of a medical practice. In the rooms themselves he found many of them sharp and well-funded, with several measuring ROI on the practice’s own numbers. Plenty were selling virtually the same thing as the next booth.

In the AMA’s 2026 Physician Survey on Augmented Intelligence, released March 12, 2026, physicians named safety and efficacy validation (88 percent) and data privacy (86 percent) as critical for broader AI adoption, and 85 percent want to be consulted or directly involved in decisions about AI adoption. That survey asked physicians rather than practice administrators.

Many medical group leaders also say their organization has no formal AI policy. In a January 20, 2026 MGMA Stat poll of medical group leaders, with 328 applicable responses, 56% said their organization neither has AI governance or a formal AI policy nor is developing one. MGMA Stat is a weekly text-message poll of healthcare leaders who sign up for the panel, so we read it as what those leaders report rather than as a random sample. MGMA’s guidance in the same article is that a practice where anyone can buy or switch on an AI tool without review has no governance.

What did almost buying one teach us?

A few months before April 13, 2026, Serious Land Capital, Chris’s own land investment firm, seriously considered a third-party AI service at a discounted $6K a year. In his words, it was pitching “basically a watered-down version of what SLC Chat is becoming,” SLC Chat being the firm’s internal underwriting assistant. He passed, and gave his reasons in the April 13, 2026 issue of his newsletter: an inferior product, a far longer turnaround on every iteration, and a voice match nowhere close to what the firm needed. In that same issue he wrote the bar down for the firm’s own version: SLC Chat would not go live until it was answering correctly about 85 to 90% of the time at minimum. The firm went on to build a better version itself. The full story is in I Almost Paid $6K/year For A Worse Version.

The same habit runs through the firm’s Deal Engine, the software that now handles its underwriting (working out what a parcel is actually worth and whether the deal clears at that price). Its first fully measured automated run, on August 18, 2026, took 3.5 hours and burned 10 to 15% of a week’s model allowance. Correct answer, unsustainable economics. It now runs in under 30 minutes unattended at about 1% of the weekly allowance. When it disagreed with the manual read by roughly $19K on a test deal, the cause surfaced the same day, and the re-run landed at $87,992 against a hand-built $88,000. We think those numbers carry weight because a hand-measured baseline sat next to them: 45 to 60 minutes per parcel by hand, and a hand-built $88,000.

In our read, a demo proves a tool can get the right answer once. What decides the purchase is how often it is right on your own work, at a cost per run that holds up, and who catches it on the days it is wrong.

What should a practice ask before the demo?

1. What number will prove it works? Pick one result the tool has to move (hold time on the phones, days to close a chart, the share of claims denied) and pull 90 days of it from your own systems before the first sales call. Chris’s October 5, 2026 issue, The AI Slide Said 40%. The Data Said 15%., sets the same test: pick one AI tool you pay for, or are about to, and write down the single number that would prove it is working. If you can’t name it, that’s your finding.
2. Where did the headline figure come from? At MGMA 2026, one session’s slide claimed ambient AI cuts documentation burden 40%, with no source. A later slide showed one health system’s actual rollout across 1,900+ clinicians, where time in notes fell 15%, and charts closed in 0.9 days instead of 15. Ask for results from a deployment that looks like your practice.
3. What happens to your data? MGMA’s minimum vendor list covers clear statements on data ownership and reuse, including whether your data can be used to train models, and security controls with breach notification expectations. Under HIPAA, a covered entity that lets a business associate create, receive, maintain or transmit protected health information on its behalf needs satisfactory assurance, documented in a written contract or other written agreement, that the business associate will appropriately safeguard the information. Confirm the specifics with your counsel and compliance lead rather than from an article.
4. What happens when the model changes? MGMA points out that models get updated, prompts change and performance shifts, and it asks for update and change controls: how model updates are communicated and how you evaluate material changes. Ask who tells you when the tool changes underneath you, and how you re-check it.
5. What does the pilot have to clear? MGMA recommends that any new AI tool come with a pilot plan and success metrics, plus an owner for monitoring and issue escalation. Write the pass mark down before the pilot starts, the way the SLC Chat bar was set before launch.
6. What does it cost at your size, and how do you get out? If the pricing page won’t show a number without a sales call, ask in writing for the all-in price at your provider count, the contract length, and what happens to your data if the pilot does not clear.

Is a vendor the right answer at all?

Often it is. Our view is that as the cost of high-powered AI output keeps falling, it makes more and more sense to shape the underlying models around how a specific practice actually runs and to layer in vendors where they are clearly best, and in Chris’s read AI voice agents are already one of those places. Serious Land Capital made that call on its own assistant, and the in-house build came out ahead of the $6K tool.

Either way, somebody has to own the whole picture, from which tools overlap to whether each one is still clearing its number months after go-live. We would put that with an in-house AI owner, or with a fractional Head of AI when nobody on staff has the capacity.

What to do this quarter

Before the next vendor call, pick the tool you are most tempted by and write down the number it has to move. Pull that number from your own systems. Send each vendor the MGMA minimums as written questions, and judge the answers against your number before you compare one demo with another. If your practice already pays for an AI tool, run the same test on it first. The areas we would look at first in a practice, vetting AI tools before you sign among them, are on the Serious AI healthcare page, and the receipts behind the method are on the Proof page.

AI can produce the analysis. It cannot be wrong, because being wrong requires having something at stake, and that stays with a person. Serious means underwritten.

=====

If your practice is weighing an AI tool and can’t yet name the number it has to move, or already has AI working on a problem without a number that shows it paying off, that is the place to start. The engagement opens with diagnosing the constraint gating the practice and where AI is falling short on it, then building with verification sitting where a person still owns the outcome. You leave the first call knowing what your constraint is whether or not you ever hire us.