Five questions to ask before AI touches your customer data

Putting AI into the way your business works usually means sending it your business information: customer emails, order histories, contracts, notes from calls. Often that's worth doing. It's also the point where a lot of companies stop asking the questions they'd ask any other vendor that handles customer information.

These are the five questions we ask before AI goes anywhere near something a client cares about. You should ask them of anyone who proposes putting AI into your systems, us included.

Where does the data go?

When someone says "we use AI," they usually mean "we send your data to an AI company's servers." Find out which company, under what agreement, and where the processing happens. A business agreement with a named provider is a very different thing from a consumer chatbot account somebody opened with a credit card. If the answer is vague, the way your data is handled probably is too.

Will it be used to train anything?

Consumer AI products have often used people's conversations to improve their models unless you opt out. The major providers' business agreements generally don't train on your data. Don't assume that, though. Get it in writing for the specific product being used. We use business-tier providers that don't train on client data, and we'll tell you which ones.

What's the least information the job needs?

The safest data is data you never sent. If AI is drafting a reply to a customer email, it needs that email and a bit of background, not the customer's whole account history. If it's sorting support tickets, it has no use for payment details. Done well, AI work starts by cutting each request down to what the task requires. That gets decided while the system is being designed, rather than being a setting someone turns on afterward.

Is there a record of what was sent?

When something goes wrong, or a customer asks what happened to their information, you want to be able to answer. That means keeping a record of what was sent to the AI, when, for which task, and what came back. If a system can't tell you that, it can't tell you much else about what it's been doing either. We keep that record as standard, for the same reasons we explain on our security page.

What happens when it gets something wrong?

AI will get things wrong sometimes, so what matters is what a mistake costs you. AI that drafts a reply for a person to check and send saves time, and there's still a person between the software and your customer. AI that sends replies by itself carries a very different level of risk. For anything that reaches a customer or moves money, we build it the first way. In one system we built, the rule that the software drafts and a person always sends is built into how the system works, so nobody can switch it off as a preference.

None of this means you should avoid AI

When you ask these questions properly, the answer is usually a clear yes. AI is good at the work people shouldn't be doing by hand, like reading, sorting, summarizing and drafting. We wrote about where it's worth using, and where it isn't, in where AI belongs in business software.

An AI tool is a vendor handling your data, and it deserves the same scrutiny as any other. If a proposal to bring AI into your business can't answer these five questions in plain language, that tells you something. If you'd like to see how we'd answer them for a particular part of your business, our AI and automation work starts with exactly this conversation, and the first call is free. What the AI provider charges for usage is billed separately at cost, so you can always see what the AI part of the system is costing you.