Every AI vendor pitch sounds the same after the third demo. Fast, secure, enterprise-grade, HIPAA-ready (if healthcare), integrates with everything. You nod along, sign the order form, and six weeks later your CRM, your call recordings, and your patient intake forms are flowing into a system you can't actually explain to your board or your compliance officer.

I've been on both sides of this. I've bought AI tools that turned out to be thin wrappers with sloppy data handling, and I've sold automation into companies that asked me exactly three questions before handing over admin access. The gap between what operators should ask and what they actually ask is huge. This post closes that gap.

You don't need a security background to vet an AI vendor well. You need a checklist, a little patience, and the willingness to keep asking follow-up questions when the first answer is vague.

Start with what data actually moves

Before you evaluate any vendor, map the data flow on one page. Not a formal diagram. A napkin sketch is fine. Write down:

  • What data leaves your systems (call recordings, patient names, credit card tokens, contract PDFs, appointment history)
  • Where it goes (the vendor's servers, a sub-processor, an LLM provider like OpenAI or Anthropic, a transcription service)
  • How long it stays there
  • Who at the vendor can see it
  • What happens to it when you cancel

If you can't fill in those five lines from the vendor's marketing site and a 30-minute call, that's already a signal. Not necessarily a red flag, but a "get more answers before signing" flag. In practice, the vendors who have their act together will fill this in for you without being asked. They've done it a hundred times.

The questions that actually separate serious vendors from resellers

Most AI vendors today are building on top of foundation models from OpenAI, Anthropic, Google, or a handful of open-source options. That's fine. What matters is how they handle your data on top of that stack. Ask these, in this order, and listen carefully to the answers.

1. "Where does my data get processed, and by whom?"

You want a specific answer. Something like: "Your data hits our API on AWS us-east-1, we store it encrypted in Postgres, and we send prompts to Anthropic's API with zero data retention enabled." If the answer is "it's secure" or "we use enterprise-grade encryption," push harder. Ask for the sub-processor list. Any serious vendor has one and can email it to you in under an hour.

2. "Is my data used to train models?"

This is the question everyone should ask and almost nobody does. The answer you want is no, with specifics. "We have zero data retention agreements with our LLM providers, and we do not train any models on customer data." If they hedge, or if they say "you can opt out," dig in. Opt-out defaults mean somebody, somewhere, is training on customer data by default. Decide if you're comfortable with that.

3. "What's your incident response process, and have you had one?"

Everyone has incidents. The question is whether they detect them, disclose them, and learn from them. A mature vendor will describe their monitoring, their notification SLAs (usually 24 to 72 hours), and, if pressed, will tell you about a real past incident and what changed after. A vendor who claims they've never had a security event either has been operating for six months or isn't looking.

4. "Who at your company can access my data, and under what conditions?"

The good answer: role-based access, logged, requires customer support tickets to trigger, engineers can't browse production data casually. The bad answer: a shrug, or "just our engineers, they're all trusted." Trust is not a control.

5. "What certifications do you have, and can I see the reports?"

SOC 2 Type II is the baseline most operators should ask about. HIPAA compliance matters if you're in healthcare, and comes with a Business Associate Agreement (BAA) that the vendor should sign before any protected health information moves. ISO 27001 shows up more often with international or enterprise-focused vendors. Ask to see the actual report or letter, not just a badge on the website. Any real vendor will send it under NDA within a day or two.

6. "What happens when I cancel?"

You want to hear: your data gets exported on request, deleted from production within 30 days, and purged from backups within 90. If a vendor can't tell you their deletion timeline, they probably don't have one.

Special notes for healthcare operators

If you run a medical or dental practice and you're looking at AI tools for scheduling, patient calls, reminders, review requests, or intake, the bar is higher. Not because the technology is different, but because the legal exposure is.

Before any patient data (names, phone numbers, appointment info, insurance details) touches a vendor, you need a signed BAA. That's non-negotiable. If a vendor tells you they're "HIPAA-compliant" but won't sign a BAA, they are not HIPAA-compliant. Full stop.

Ask specifically about call recordings and transcripts. Many AI phone agents record calls, and those recordings often contain PHI. Where do those live? For how long? Who transcribes them? Is the transcription service also covered under a BAA?

And confirm compliance specifics with your own healthcare counsel. I'm giving you the operator's lens here, not legal advice, and your practice's specific setup (state laws, payer contracts, EMR integrations) may add requirements.

Red flags I've learned to walk away from

A few patterns that, in my experience, correlate strongly with pain later:

  • The vendor can't name their sub-processors on the first call
  • Their security page is one paragraph long and mentions "military-grade encryption"
  • They require you to email them a spreadsheet of customer data to get started
  • They can't produce a SOC 2 report but insist they're "SOC 2 aligned"
  • Their sales rep answers technical security questions and their engineer isn't available
  • The contract has no data processing addendum, or the DPA is a two-page template

None of these are automatic disqualifications on their own. Early-stage vendors with genuinely good practices sometimes lack the paperwork. But if you see three or more, keep shopping.

What to do after you sign

Vetting doesn't end at contract signature. Two habits worth building:

First, keep a simple inventory of every AI tool touching your data, what it accesses, and who owns the relationship internally. A shared spreadsheet is enough for most operators. When something breaks or a security question comes up, you'll thank yourself.

Second, review access annually. Vendors add features, expand permissions, and swap sub-processors. Ask for an updated sub-processor list once a year and see what changed. If a vendor quietly added a new AI provider you don't recognize, that's worth a conversation.

Vetting AI vendors well is not about being paranoid. It's about knowing enough to make an informed trade-off. Sometimes the right call is to accept a little more risk for a lot more speed. Sometimes it's to pass on a slick demo because the data handling isn't there yet. Either way, you want to be the one deciding, not discovering.

If you're evaluating automation for your operations and want to work with a team that answers these questions before you ask them, talk to our team at Qintara Corp. We build custom AI agents and automations for real operational work, and we'll walk you through exactly how your data moves before we touch anything.

Frequently Asked Questions

Do I need a security background to vet AI vendors well?

No. You need a written data flow, a short list of the right questions, and the discipline to keep asking follow-ups until answers get specific. If a vendor can't explain their setup to a capable operator in plain terms, that's information too.

What's the difference between "HIPAA-compliant" and signing a BAA?

A BAA (Business Associate Agreement) is the legal contract that actually makes a vendor accountable for handling protected health information under HIPAA. A vendor saying they are "HIPAA-compliant" without signing one is marketing language. If they won't sign a BAA and you're handling PHI, they're not a real option.

How long should vendor vetting take?

For a small operational tool with limited data exposure, a few days. For something that touches your CRM, phone system, or patient records, plan on two to four weeks including security review, sub-processor list review, contract redlines, and a pilot with limited scope before full rollout.

Is it safe to use AI tools built on OpenAI or Anthropic?

It can be, if the vendor has configured the integration properly, specifically enterprise or API tiers with zero data retention and no training on your data. The foundation model provider matters less than how your vendor uses it. Ask about the retention and training settings on their upstream contracts.

What if a vendor I already use doesn't meet these standards?

Don't panic-cancel. Document the gap, ask the vendor for a remediation timeline, and consider limiting what data flows through them while you evaluate alternatives. Most operators discover a few of these gaps when they start looking. Fixing them is a project, not an emergency.