An AI audit is a structured examination of whether a particular business task can be supported reliably by AI. It starts with the actual process, available data, permissions and the person accountable for the result. A useful audit may recommend implementation, preparation of the data, or a decision to delay automation. Its value lies in turning a broad ambition into a decision grounded in evidence.
Define the scope around a real task
“Are we ready for AI?” is too broad a question. A company may be ready to classify incoming requests while being unprepared for automated financial commitments. Describe the task through its input, desired output, frequency, owner and consequences of error. Establish where the system's responsibility ends and human judgement begins.
In a hypothetical example, a technical support company wants to recommend repair procedures using old manuals. The audit examines more than the existence of PDFs. It checks whether each manual matches the correct equipment model, whether newer instructions exist, and whether an incorrect recommendation could create a safety issue. Providing information and instructing someone to perform an intervention require different designs, even with the same tool.
A data inventory should reveal more than volume
Record each source, format, owner, review date and access method. A folder containing a thousand files may be less useful than fifty approved documents. Check duplicates, scanned pages without readable text, broken tables and attachments containing critical information. Test extraction before assuming the entire archive is usable.
Assess quality using a representative sample. Select ordinary, difficult and recent cases, rather than only well-organised documents. For each sample, record the answer an experienced employee would consider correct and its supporting source. When two employees disagree, the issue needs a policy decision or clarification before it needs a better model. Include the evidence necessary for another reviewer to understand that decision later.
Usage and access permissions are part of readiness
Being able to read a file does not mean it may be transferred to any service or exposed to everyone in the company. Map access groups, confidential content and approved processing destinations. The appropriate business owner must approve the scope. Where specific legal obligations arise, obtain suitable professional advice rather than treating the audit as a compliance certification.
For the repair example, public manuals can form a collection separate from internal customer reports. This reduces exposure and simplifies testing. If the assistant needs fault histories, access can be applied by role and individual request. The audit should record this choice and its additional operating cost, including how access will be maintained when employees change roles.
Establish process rules before adding automation
Follow a task from receipt to completion. Who corrects missing details? How is an urgent case identified? Where is the final decision recorded? If the answers vary between employees, automation will multiply inconsistency. First agree on a workable minimum: required fields, completion status and a clear exception route.
Observe informal work too. A spreadsheet may appear complete because someone remembers telephone clarifications that are never recorded. That knowledge needs structured capture or a human checkpoint. It should not be presented as information the system already possesses. Ask employees to demonstrate a difficult case, because normal cases often conceal these dependencies.
Measure the starting point and review capacity
Record current time per accepted task, corrections, delays and common causes of failure. Include information searches and communication, rather than only final typing. Comparison with a pilot should involve equivalent quality. Faster provisional answers cannot fairly be compared with approved instructions.
Also establish who will review the pilot's results. If the team's specialist has no time for verification, the project has a capacity constraint. Allocate evaluation hours, correction records and an approval owner. Team training may need to precede implementation so users understand when to challenge an answer. Review capacity is a practical project requirement, rather than an optional activity after launch.
The deliverable should support a concrete decision
A useful result includes a process map, source inventory, known gaps and a bounded candidate pilot. Every gap needs an owner, an action and a completion condition. “Data quality is average” does little to help. “Two conflicting instructions require the technical manager's approval before entering the search collection” is actionable.
For the hypothetical project, acceptance may require excluding withdrawn manuals, supporting each model-specific answer with evidence and involving a person whenever safe instructions are unavailable. Exact limits depend on the business risk. AI Consulting connects this assessment with an implementation plan, while AI Knowledge organises the sources the system will use. The decision should also identify which work remains manual and the circumstances that would justify reconsidering it.
Frequently asked questions about AI audits
Must we clean all our data before starting?
No. You need a sufficient approved collection for the chosen task. The audit helps determine what to clean first, so preparation serves an actual use rather than an indefinite ambition to perfect the entire archive.
Can interviews alone establish readiness?
Interviews reveal rules and expectations, but actual examples must be checked. The stated process may differ from what happens when a field is missing or an exceptional situation arises. Observing work helps expose those differences.
What happens when sources conflict?
Record the conflict and request an official decision from the responsible person. Do not let the model choose the most persuasive wording. Until the issue is resolved, exclude the topic or communicate uncertainty clearly.
Is an audit unsuccessful if it recommends a delay?
It is useful when it prevents spending without the conditions needed for success. A delay should come with specific tasks and a reassessment method, rather than a vague recommendation that the company become “more mature”.
For an audit of your own process, contact DIGITALNOW. For further technical reading, Anthropic's discussion of effective agents explains why architecture should follow the task. The assessment framework above is our own practical editorial methodology.