Most AI governance writing is written for companies with a General Counsel, a Chief Risk Officer, and a compliance team that reads EU regulations for fun. That is not most small businesses. What follows is a working checklist — concrete, non-legalistic, actionable by a 20-person company without a dedicated IT department.
This is not about checking boxes. It is about avoiding the specific failure modes that actually hurt small businesses when AI goes wrong: data leaking to a vendor’s training pipeline, employees using unapproved tools that expose customer records, automated decisions that nobody can explain after the fact. All of these have happened, and none required negligence — just the absence of a few deliberate policies.
1. Know What AI You Are Actually Running
Before writing any policy, inventory your AI surface area. This means the tools you deliberately chose, plus the AI features quietly baked into software you already use.
Ask about every SaaS product you pay for:
- Does it have AI features enabled by default?
- What data does it send to AI systems?
- Is it using a third-party AI provider (OpenAI, Anthropic, Google, etc.) as a subprocessor?
This matters because 63.6% of AI vendors do not disclose their third-party AI subprocessors by default. You may have signed a data processing agreement with a CRM vendor without knowing that vendor pipes your customer records through a model provider’s API. The inventory is not paranoia — it is due diligence.
Practical step: add a line to every new vendor evaluation asking which AI systems process your data, whether your data is used for model training, and what contractual controls govern that.
2. Data Classification Before Tool Deployment
Not all data is equal. Before connecting any AI tool to your business data, categorize what you are exposing:
- Public-safe: marketing copy, general research, anonymous analytics
- Internal-only: internal memos, pricing strategy, unreleased product details
- Regulated or sensitive: customer PII, payment card data, health information, employee records
AI tools should only receive data at the appropriate classification level. A general-purpose chat assistant can work with public-safe content freely. It should not receive a spreadsheet of customer names, emails, and purchase histories unless the vendor’s data processing terms explicitly cover this and you have verified where that data goes.
This is where most small businesses have unexamined exposure. It is not that someone made a bad decision — it is that nobody made a decision at all.
3. Shadow AI Policy
Roughly 55% of employees report using AI tools not approved by their organization. Only 37% of organizations have any policy to address it. The math here is uncomfortable.
Shadow AI is not primarily a security theater problem. The practical risks are:
- Customer data pasted into a free-tier tool with no data processing agreement
- Proprietary information entered into a consumer product that uses inputs for training
- Inconsistent outputs in customer-facing work that nobody reviews
A shadow AI policy does not need to be restrictive. It needs to be explicit. Define:
- Which tools are approved, and for what data classifications
- Which tools require approval before use
- What “approval” actually means (who reviews, what they check)
- What employees should do if they want to use a tool that isn’t on the list yet
The policy fails if the answer to “I want to use this new AI tool” is a bureaucratic slog. Make a short approved list, a simple request process, and a fast turnaround. The goal is that employees route through a lightweight check rather than quietly using whatever they found.
4. Human-in-the-Loop for Consequential Decisions
Define which decisions require human review before acting on an AI output. This does not mean every output needs a human — that defeats the point. It means identifying the specific cases where the cost of an error is high enough that a human needs to be in the loop.
Candidates:
- Contract terms sent to customers
- Credit or payment decisions
- Employee performance inputs
- Any automated communication on a sensitive topic (complaints, disputes, terminations)
- Regulatory filings that include AI-generated analysis
The practical test: if an AI system made an error on this decision and a customer or regulator asked who reviewed it, what would your answer be? If the answer is “nobody,” that decision probably needs a checkpoint.
Automation is valuable precisely because it removes friction. Human-in-the-loop controls add friction deliberately, which means they should be applied selectively where the asymmetry justifies it — not uniformly. For a framework on where AI actually adds value versus where simpler controls suffice, see when AI makes sense for your business.
5. Vendor and Data Residency Due Diligence
When evaluating any AI tool, get answers to four questions before signing:
- Where is my data processed? Which data centers, which countries. This matters for GDPR if you serve EU customers, and increasingly for state-level U.S. privacy laws.
- Is my data used to train models? Many consumer-tier AI products train on user inputs by default. Enterprise and API tiers typically do not — but verify in writing, not marketing copy.
- Who are the AI subprocessors? The vendor you sign with may pass data to OpenAI, Google, or another provider. You need to know the full chain.
- What happens to my data if I cancel? Deletion timelines, data portability, and residual data in training corpora are all negotiable upfront and nearly impossible to address after the fact.
For SMBs selling into the EU, the relevant regulatory backdrop is the EU AI Act. The good news: most AI tools used by small businesses fall into minimal-risk or limited-risk categories — CRM automation, content generation, support ticket routing. The high-risk rules (which apply to employment screening, credit scoring, and similar consequential decisions) were pushed to December 2027 by the EU Omnibus negotiated in May. We covered the EU AI Omnibus timeline changes here. For most SMBs, the immediate obligation is transparency and data handling — not full compliance apparatus.
For a broader look at the AI landscape including what open-weight models might reduce your vendor dependency, the open-source AI models guide is worth reviewing.
6. Logging and Audit Trail
You need to be able to answer: what did the AI do, when, and on what data?
This is less about liability (though that too) and more about debugging. When an AI-assisted output is wrong, you need to know whether the error was in the prompt, the context provided, the model’s output, or the human review step. Without logs, you are debugging blind.
Minimum viable logging:
- Which model or tool produced an output
- What prompt or instructions were used (or a reference to the prompt version)
- What data was provided as context
- Who reviewed it (if human review is required)
- The date and outcome
You do not need to build a compliance system. A shared document or a simple database row per significant AI-assisted action is sufficient for a small team. The key is that logs exist and are findable if you need them six months later.
7. Acceptable Use Statement
Write down what AI is for inside your company and what it is not for. Keep it to one page. This forces clarity that most governance discussions avoid.
Specifics that belong in an acceptable use statement:
- AI is not a substitute for professional review in legal, financial, or medical contexts
- AI outputs used in customer-facing materials require human review before publishing
- Employees are responsible for the accuracy of AI-assisted work, not the tool
- Specific prohibited uses (e.g., generating synthetic customer testimonials, automated communications that impersonate a specific person)
The statement is also the right place to define accountability: who owns AI governance decisions when something goes wrong? For a 15-person company, this is probably the owner or CEO. The point is that somebody owns it, not that the role is formal.
AI governance for a small business is not a compliance project. It is a set of deliberate decisions that prevent the specific failure modes that actually occur — data exposure, shadow tool risk, unreviewed automated decisions, and vendor entanglement that becomes visible only after something breaks. For help thinking through which AI investments make sense before adding governance overhead, our AI solutions practice is where we typically start those conversations.
The checklist above is a minimum viable posture, not a ceiling. Companies that handle regulated data, serve EU customers, or use AI in employment or financial decisions will need to go further. But for most SMBs, getting these seven things right closes the majority of the practical risk.
Sources