Before you trust an AI workflow, decide what “done” means

An AI assistant can sound finished before the work is finished. I keep returning to that gap: the distance between a reassuring answer and something another person can actually use.

For a small organization, the distinction is practical. A meeting summary needs to preserve the decision. An inbox assistant needs to leave an important message where someone will see it. A task marked complete needs to point to the thing that was completed.

I’m learning to make those expectations visible before deciding how much of the work to automate.

At Luminosa, I bring together research, workflow design, and experiments with AI tools. Much of the learning described here comes from my own internal projects. I frame the problem, direct AI collaborators, and test the results. These are working lessons from development, rather than claims about client savings or organization-wide adoption.

Begin with an ordinary task

“We should use AI” leaves too much undecided. I find it more useful to start with a sentence someone could recognize from their working day: “I spend the first hour sorting messages,” or “I cannot tell which version of this document contains the decision.”

That gives us something to examine. What information arrives? What does someone do with it? Where does the work wait? What would a good result look like to the person receiving it?

Sometimes the answer calls for a language model. Sometimes it calls for a clear form, a shared record, or two existing tools exchanging information. Anthropic’s 2024 engineering guide makes a similar recommendation: begin with the simplest workable approach and add complexity when evaluation shows that it improves the result. Building effective agents

I want to apply that discipline before an organization inherits another system to maintain.

Give uncertainty somewhere to go

One internal project has involved designing an assistant to screen incoming email. The narrow question was whether a message appeared to be junk, legitimate, or uncertain.

The third answer mattered. In the screening rules we tested, an uncertain result leaves the message in the inbox. An answer that fails the required format also leaves it there. The system needs an explicit way to stop short of acting when the evidence is inadequate.

Testing changed the model choice, too. One candidate ran faster in this small test battery but failed several deliberately difficult examples. That made speed a poor reason to select it. These were limited tests on constructed examples; they did not establish everyday inbox accuracy, and the replacement workflow was still unfinished in the records reviewed for this article.

The lesson I take from that work is simple: decide what happens when the AI is unsure before celebrating what it does when the example is easy.

For an organization, that might mean returning an uncertain expense category for review, highlighting an ambiguous sentence in a meeting note, or keeping a proposed reply in draft. Those are possible applications, not outcomes we have measured. Each needs a person who can review it and a clear place for that review to happen.

Make the result inspectable

Another internal project, Work Atlas, grew around the difficulty of keeping tasks, context, and AI-assisted work together.

In the version developed for that project, proposed changes were checked against a defined set of permissions. Applied changes were recorded and returned alongside the response. Archiving kept the underlying task so it could be restored.

That gives me something more useful to inspect than a sentence saying everything went well. I can look at the task, the change, and the record of what happened. It does not eliminate the need to check whether the result makes sense.

I think the same principle applies to ordinary administrative work. If a tool summarizes a document, keep the document within reach. If it extracts a date, preserve the passage that supports the date. If it prepares a follow-up, make the draft easy to inspect before it is sent.

The person reviewing the work should not have to reconstruct the entire process just to find out what changed.

Test the experience someone will have

Working on a voice interface for my assistant, Noti, made another distinction difficult to ignore. A response can arrive successfully while the conversation still feels awkward: audio cuts out, speech stutters, or a pause makes it unclear whether anything is happening.

My own feedback during development included those problems. Automated checks helped examine particular failures, but a successful simulated exchange could not answer every question about using the assistant on a phone.

I want the word “tested” to carry its limits with it. A small example, a browser check, and repeated use by the intended person tell us different things.

For an organization, I would ask the person doing the work to try the whole sequence: find the result, correct a mistake, recover from an interruption, and continue their day. That is where we can begin to see whether the tool reduces effort or merely moves it somewhere less visible.

Keep learning attached to evidence

Our internal workflow notebook, which we call the Slipbox, was designed to hold short records of work we had built or tested. Its format asks why a workflow exists, what it does, who is responsible, where the evidence is, and when a person must approve the next step. It also leaves room for what remains unknown.

I find that useful because an idea can otherwise become more certain each time it is retold. A record of one successful test should remain a record of one successful test. A promising prototype should remain a prototype until there is evidence for a stronger claim.

For a nonprofit considering a first AI project, I would start with one recurring task and write down four things:

• What a useful result must contain.
• What the system may change on its own.
• What should happen when it cannot decide.
• How the person responsible will check the result.

Then try it on a small, representative set of work. Keep the original material. Record the corrections and the review time, along with the successful outputs. Those observations can tell us whether to continue, revise, or stop.

I want technology to make the work clearer and leave people better able to exercise judgment. For me, that begins with being precise about what happened—and honest about what we still need to learn.