Where AI belongs in business software, and where it doesn't

Nearly every business software vendor has added AI in the last two years, and most of them put it in the same place, a chat panel down the side of the screen. It's the easiest thing for them to build and the least useful thing for you to use, because it asks the person doing the work to stop, explain what they want, and then copy the answer back to where they were.

The AI worth paying for is built into the work itself. It reads the invoice when it arrives, or sorts the email before anyone opens it, so nobody has to remember to go and ask it something.

What it's good at

There's a clear pattern to what works. These tools do well wherever the input is messy, written by people, and what you want back is something organized or a first draft.

Reading documents is the obvious one. Invoices, applications, purchase orders, inspection forms: pulling the details out of a PDF someone emailed used to take either a person or a fussy template that broke whenever the layout changed. Now it takes neither.

Sorting and routing is another. Which department does this belong to, is this request urgent, does this email promise something to a customer. Those are all questions about language, and AI handles them well.

Drafting works too: the follow-up email, the summary, the first version of a description. Your team edits the draft, which is a better start than a blank page, and their judgment stays in the edit.

Then there's searching your own material. The answer is often buried in a spec, a contract or six years of job notes, and finding it is most of the work. That's the job AI does best.

All of these have something in common. A person looks at the result, and when it's wrong, it's obvious.

What to keep away from it

Our rule of thumb is simple. If a mistake could slip through without anyone noticing, don't use AI for that job.

That rules out anything involving arithmetic, like pricing, tax, commissions and quantities. Those rules belong in ordinary software, where they can be tested and give the same answer every time. Something that's right 98% of the time is a disaster on an invoice.

It also rules out compliance and eligibility decisions. If you might have to explain to a regulator or a customer why a decision came out the way it did, you need to be able to show the exact logic behind it.

And keep it away from anything that can't be undone, unless someone reviews it first: sending, paying, deleting, committing to an order. AI can get those ready. A person should be the one who presses the button.

It's tempting to use AI because it's quicker to set up than writing the rules out properly. That trade is fine for a draft and a bad idea for a calculation.

Four things to settle first

You don't need a policy binder or a committee, but you do need answers to four questions before AI goes anywhere near your business data.

First, know where the data goes. Use the business versions of AI services, whose terms say your data isn't used to train their models, and keep track of which provider handles what.

Second, send as little as the job needs. There's rarely a good reason to hand over a whole customer record when the task only needs two fields.

Third, keep a record of what was sent and what came back. Six months from now, when someone asks why the system did something, that record is your answer.

Fourth, decide who checks the results. For anything that goes outside the company or changes a record, that should be a person. Plan the review step in from the start, before a bad result forces you to.

It costs money every time it's used

Most software costs the same whether you use it once a day or a thousand times. AI is different. The AI provider charges for every request, so a feature that looks cheap in testing can cost a lot more once a whole department is using it every day.

That should affect how you buy it. A vendor who folds AI usage into a flat fee is either limiting your use somewhere you can't see, padding the fee to cover the worst case, or planning to renegotiate later. We pass the AI provider's usage charges through at cost, separately from the monthly fee, because usage varies and you should be able to see the real number.

If you're working out where AI fits in your own business, start with the jobs where someone spends the day reading messy documents or emails and a mistake would be easy to spot. Keep the calculations in ordinary software, keep a person in charge of anything that matters, and budget for the running cost as its own line.