From "Hello World" to "Create an Invoice" — The Safety Architecture Behind AI That Actually Ships
It started as a side project. I wanted to see if I could build something that let me talk to my ERP system the way I talk to a colleague — just describe what I need and have it done.
Create a leave application. Show me pending approvals. Find that invoice from last week. All from a chat interface. No digging through menus, no remembering which module has which form.
The idea was simple: a chat that understands what you mean and acts on it while respecting what you're allowed to do. If you can approve leave requests in the system, the chat should let you do it too. If you can't, it should gracefully decline.
What I didn't expect was how quickly this side project pulled me into the hardest engineering problem I have faced in a while. Not building the chat itself. Not connecting to the API. The hard part was making sure the chat only did what it was supposed to do, for the right person, at the right time, with nothing slipping through.
The First Attempt
When I started building the agent's brain, the first approach was a decision tree — rigid, predictable, brittle. It would walk through a predefined set of questions and branch based on keywords. The tools required were simple: a state machine, some regex patterns, and a lot of if-else statements. It worked for exactly the questions I anticipated, and failed on everything else. One slightly creative phrasing, and the entire flow collapsed. Like teaching someone to recite answers without understanding the material.
Enter the LLM
That changed when I introduced an LLM into the mix. Specifically, DeepSeek's function-calling model. The tooling here was different — instead of a decision tree, I needed an API client that could handle streaming responses, a prompt engineering framework to define system context, and a structured output parser to extract tool calls from the model's response.
The model brought genuine intent understanding to the table. Ask for customers and it delivers, not because you coded every possible variation of the request, but because the model actually understands what you mean. This was the first real breakthrough moment, but it introduced a new problem. The agent could understand requests, but it couldn't do anything about them yet.
Giving the Agent Tools
The next step was giving the agent tools to act on the world. This meant building a set of whitelisted API endpoints that the LLM could call — search_records to find data, get_doctype_schema to inspect field structures, create_draft to build documents, submit_document to finalize workflows. Each endpoint required careful input validation, permission checks, and structured error responses that the model could interpret and relay to the user.
The LLM decides which tool to use based on what you ask. It's like giving someone a toolbox and watching them reach for the right tool every time. But with that power came an uncomfortable question. How do you stop this thing from creating the wrong invoice?
The Safety Architecture
The answer was a multi-layered safety architecture, and this is where the real engineering work happened.
Layer 1: The Permission Gate
Every tool call checks against your actual Frappe role before it touches the database. If your role can only view Customers, the agent will gracefully decline when asked to query Sales Invoices. It doesn't even attempt the operation.
You need proper permission management in your ERP system, role-based access control that's actually configured, and a middleware layer that intercepts every tool call to verify authorization before execution. This isn't a prompt-level guardrail. It's hard enforcement at the code level, and it never delegates security decisions to the LLM.
Layer 2: Explicit Confirmation
Draft documents never submit automatically. The agent creates a pending action, records it in an audit record, and waits for you. Creating this required an action queue system, a confirmation workflow that stores pending actions with their context, and an audit log schema that captures every decision.
You see something like: "Created draft invoice ACC-SINV-2026-00004 for $1,000. Reply with 'yes submit it' to confirm." No silent operations. Every write to the database is a two-step process. The agent proposes, you approve, and the audit trail captures every micro-decision along the way.
Layer 3: The DocType Whitelist
The allowed document types are configured in a single settings table — one list, in one place, no exceptions. This required building a configuration DocType, a schema inspection service that dynamically learns the structure of each allowed type, and field-level filtering to expose only safe fields.
Add Account to the list and the agent dynamically inspects its schema and can query accounts intelligently. Remove it and it gracefully responds that it can't help with that. The scope of what the agent touches is explicitly declared, not guessed at runtime.
Layer 4: The Audit Trail
Every tool call, every question asked, every draft created leaves a record. Not for compliance theater — for debugging. This required a dedicated audit DocType with structured fields for conversation context, tool name, input parameters, response data, and timestamps.
When something goes wrong — and it will — you trace the exact conversation turn and tool input that caused it. The data model needs to be compact enough to query quickly but structured enough to be useful during an incident review.
What's Next
What comes next is a fully configurable settings table where you add any DocType and the agent dynamically learns its structure. No more hardcoded lists. The agent inspects the schema, figures out the safe fields, and queries intelligently. The tooling for this phase involves automated schema discovery, dynamic field-level permission inference, and a self-configuring agent that adapts to whatever data model you throw at it.
The end result is an agent that can create sick leaves, draft Sales Invoices with real line items, query customer data, and filter support tickets — all while respecting your permissions, asking before acting, and leaving an audit trail you can actually read. This is not science fiction. It is basic engineering discipline wrapped around an LLM, built one layer at a time, with the right tools for each step.
What is your approach to AI safety? Permission layers, confirmation gates, or something completely different? Leave a comment below.