agentum
An AI assistant can prepare a proposal, classify incoming requests and summarize a document quickly. But speed is not quality. A confident response that invents a delivery date, includes someone else's confidential data or takes an unauthorized action may cost a business more than the time it saves.
The mature question is not “which model sounds smartest?” but how to assign a task, validate the deliverable, restrict permissions and measure whether automation is economically justified. This guide covers an operating matrix, three review layers, a fictional case, worked cost scenarios and a practical implementation checklist. All numbers are illustrative, not AGENTUM pricing, benchmark data or measured performance.
Why “the AI did great” is an inadequate quality standard
A fictional sales manager asks an assistant to draft offers from client correspondence and likes the professional tone. A week later the team discovers that the drafts promised unavailable discounts and delivery dates and cited an outdated price list. The messages looked polished, but the claims were wrong.
This need not mean that the language model is poor. Task specification, data provenance, version control and permissions can matter more than presentation. Early successful examples are not permission to let an assistant commit the business to binding terms.
Define acceptable output first: facts must match approved sources, timelines be supported, uncertainties identified and the person responsible for release recorded. Make it possible for another employee to reproduce the review.
Separate tasks by the consequences of mistakes
| Task class | Example | Who approves | Suitable automation |
|---|---|---|---|
| Informational | Public document summary or idea list | Request owner, as needed | Drafting and classification |
| Operational | Internal customer inquiry summary or plan | Responsible employee before use | Suggested actions, audit trail, limited rights |
| Critical | Contract signature, payment change, data deletion or formal legal opinion | Authorized person under a defined process | Decision support only, unless separately controlled execution is explicitly permitted |
This is an organizational framework, not a universal legal classification. The same operation may become critical depending on the customer, jurisdiction or potential damage. Payments, access changes, medical decisions and legal obligations require appropriate professional safeguards and technical controls.
A useful task specification has six fields
Describe goal, context, permitted sources, required output, restrictions, and acceptance criteria. Example: “Summarize these three emails for our account manager. Use only the attached correspondence. Quote any existing obligation and deadline from its source. Do not invent a price or send a message to the customer. Return a table containing task, owner, date and source reference.”
This tests completeness and compliance, not personal taste in writing. If the source has no deadline, the right answer is “not specified,” rather than a fabricated date.
Create templates for repeatable low-risk operations. Avoid a single universal prompt purporting to run an entire company without contextual limitations.
Sources and permissions: AI should not see everything
Customer support staff rarely need access to every confidential quotation from other departments. AI tools should follow the same least-privilege principle. Connect only relevant and authorized data sources.
For Google Drive, CRM, spreadsheets and corporate records, distinguish reading, editing, creating, sending, and deleting. Read-only access reduces some risks but does not eliminate data leakage: sensitive content can still be reproduced in a response.
Document which data is transmitted to external processors, retention rules and authorized decision makers. The European Data Protection Board explains data minimization and security. Do not infer compliance simply from the presence of an API integration.
Three layers of output verification
Layer 1 — automatically testable properties. Verify format, required fields, allowed lengths, arithmetic and structured output. A valid JSON object may still contain false claims.
Layer 2 — factual comparison. Check prices, dates, document titles and quotations against approved source versions. A citation should really support the claim. A convincing-looking URL is not proof.
Layer 3 — business decision. A person evaluates tone, commercial context, contractual constraints and consequences before an external commitment. Correct arithmetic is not sufficient authority to contact a customer.
No “smart mode” setting replaces all three layers. Autonomous execution of higher-risk actions needs a separate justification, access controls and meaningful confirmation.
Fifteen test cases instead of one impressive demonstration
Before a pilot, assemble a small synthetic test set:
- Five normal repeatable cases using fictional input data.
- Three incomplete requests requiring a caveat or clarification.
- Three requests with conflicting documents dated differently.
- Two structured or numerical exercises that are easy to check manually.
- Two permission-boundary cases where the assistant should refuse or request human authorization.
Fifteen examples form an initial diagnostic, not a statistical reliability proof. Production use requires broader variation, repeated tests and continued observation after changes to models or integrations. Specify the expected behavior and maximum tolerable risk before viewing the results.
Accuracy, usefulness and critical violations are different metrics
| Metric | What it measures | Not the same as |
|---|---|---|
| Accepted draft rate | Share of results meeting agreed acceptance criteria | Total number of generated responses |
| Time to usable work | Input + generation + verification + rework | Model latency |
| Factual errors | Wrong amounts, dates, citations or facts | Stylistic preferences |
| Critical incidents | Unauthorized actions, disclosure, invented commitments | Average answer rating |
| Cost per accepted outcome | API + tools + human work | Price per model request |
| Appropriate refusal | Honest indication of insufficient evidence | An assistant that always answers |
An average satisfaction score of 4.8/5 does not excuse a single dangerous payment or breach of confidential records. Use a separate stop threshold for critical failures rather than averaging them into successful drafts.
Fictional case: manager Oleh and a four-person team
Oleh leads a fictional business with four employees answering recurring customer inquiries. They spend significant time finding facts in documents and rewriting standard explanations.
Oleh lets AI draft internal replies only. Its sources are restricted to currently approved terms. Human employees send all customer messages; price changes, refunds and payment operations are prohibited for the automation.
During the first pilot, the tool identifies questions well but sometimes favors an outdated document when old and new sources conflict. Instead of prompting “be more careful,” Oleh establishes one authoritative version and requires source evidence for each important deadline. This reduces process ambiguity without proving zero risk.
Worked economics: cost of completed work, not tokens
Suppose an employee needs 14 minutes to produce and check a reply without AI. For 200 replies per month, 200 × 14 = 2,800 minutes, or 46 hours 40 minutes.
In a fictional AI-supported process, entering information and drafting takes 4 minutes and review takes 5 minutes. Then 200 × 9 = 1,800 minutes, or 30 hours. The hypothetical saving is 1,000 minutes, or 16 hours 40 minutes.
At a hypothetical value of $18/hour, the freed time is worth $300. Assuming $80 for AI and tools, the example balance is $220, before errors, taxes or downtime. It is not demonstrated profit or AGENTUM savings.
Now add an average of three minutes of additional review to every response: 200 × 12 = 2,400 minutes. Only 400 minutes remain saved—six hours 40 minutes, around $120—and $80 in tools leaves $40. A small rise in review time substantially changes the economics.
Intelligent routing is not magic or permission to disclose data
AGENTUM's design includes selecting appropriate processing for different requests instead of sending everything to one costly, complex agent. “Intelligent routing” is not evidence that every selection is optimal, safe or cheap.
Define when a simple response is enough, when approved documents are needed, when a specialist must review, when external processing is forbidden and when the person in charge must decide. Test both normal and edge cases; retain enough audit information to investigate a wrong route.
Internal model names do not need to be public in every reply, but data processing, service boundaries and accountability should remain understandable. Technical optimization must not compromise privacy or output quality.
Documents and spreadsheets: what read-only mode improves
When an employee asks AI to summarize a policy, it helps to link the result to the current document without allowing the AI to change the original. When filling a spreadsheet, define which fields may be suggested and which must be confirmed by a person.
Keep proposed edits separate from actual write operations. Avoid silently overwriting source records or posting confidential files to group chats. Use access logs and an approval step for sensitive actions.
For larger tables, check units, blanks, formulas and totals separately. A single percentage mistake can make a convincing report unusable for a business decision.
Incident response when AI produces an unsafe answer
Have a short incident procedure: stop the harmful operation, record only the needed evidence in authorized logs, identify affected data, correct messages sent to customers, improve acceptance criteria and rerun tests before reactivation.
Logging must not become a second privacy incident. Never include secrets or unnecessary customer correspondence merely for diagnostics. If personal data is involved, comply with applicable notification duties and deadlines.
The NIST AI Risk Management Framework offers a practical structure: govern risk, map context, measure outcomes and improve safeguards. Following its concepts does not by itself certify a product.
A seven-day manager's pilot plan
- Day 1. Choose one repeated, low-risk task and measure the existing workflow.
- Day 2. Write the six-field assignment, restrictions and named human owner.
- Day 3. Prepare fifteen synthetic test cases with acceptance criteria.
- Day 4. Restrict sources and permissions; verify that prohibited actions cannot execute.
- Day 5. Test results and separately record factual and critical errors.
- Day 6. Measure input, generation, correction, review and tooling costs.
- Day 7. Allow only a limited pilot at acceptable risk with an error log and stop control.
Seven days are a suggested initial structure, not sufficient assurance for finance, medicine, legal decisions or other critical systems.
Twelve questions before approving automation
- What exact task is assigned to AI?
- Which person is accountable for the result delivered to customers?
- Which sources are approved and current?
- What data must never reach external providers?
- Are read and write permissions separated?
- Which actions require a human confirmation?
- Are numbers, dates and references independently checked?
- Is there a distinct rule for critical failures?
- Are model settings and instruction versions recorded?
- Is complete time to accepted work measured?
- Can the process be stopped and recovered?
- Is quality retested when a model or source changes?
Conclusion. Strong AI automation depends less on model power than on disciplined assignments, authorized sources, measurable outcomes, human review and incident handling. A leader must be able to say “not ready to deploy” when the system answers quickly but makes dangerous promises or requires too much correction.
Frequently asked questions
Can AI send customer emails autonomously? Only after assessing risk, permissions, confirmation steps and relevant requirements. For an initial pilot, leave sending to a person.
How can we measure savings? Compare end-to-end time and costs per accepted output, not just generation time.
What if the assistant does not know the answer? It should ask for clarification or mark uncertainty instead of inventing data.
Why not simply choose the “most powerful” model? Tasks differ in analytical depth, tool requirements and risk. Select using measured quality and cost, not a model name.
Are fifteen tests sufficient? No. They are a first diagnostic; continued usage requires more scenarios and monitoring.
Can work documents be connected? Only with appropriate access controls, data-processing terms and restrictions. Read-only access helps but does not guarantee confidentiality.
Official references
- NIST AI Risk Management Framework - NIST AI RMF Playbook - EDPB — data protection basics - OWASP — LLM Top 10 - AGENTUM — AI business tools