MagicTeams.ai · Sample deliverable
Sample AI audit readout
What leadership receives at the end of the 4 to 6 week company-wide AI audit: one page that says what agents you run, who owns them, where the risk is, and what to fix first.
Example Co. is a fictional company with 400 locations. Every name and number on this page is illustrative.
Summary
Top five risks
- High
The phone ordering agent has no eval suite or pass bar, so a model update could change order accuracy without anyone noticing.
Owner: VP Operations. Fix: Seed evals from 40 real failed calls; gate every change on them (G2.14 to G2.16).
- High
The delivery dispatch optimizer runs on a vendor platform with no owner, no record, and no way for a site to override it.
Owner: Unassigned. Fix: Name an owner, register it, and add a per-site override before the next platform update (G1.2, G4.3).
- High
The shift scheduling assistant makes employment-related recommendations with no notice or appeal path.
Owner: HR Operations. Fix: Add notice, a human reviewer, and a risk assessment before it is used for decisions (G1.7).
- Medium
41% of AI spend is untagged, so nobody can say what any one agent costs per successful outcome.
Owner: Finance. Fix: Route model calls through one gateway with per-agent identity and cost tags (G3.4, G3.5).
- Medium
Staff-built tools hold personal and financial data in accounts the company does not administer.
Owner: IT Security. Fix: Move them under company identity or retire them; record each in the inventory (G1.1, G2.1).
Agent inventory (8 of 14 shown)
| Agent | Source | Business owner | Risk tier | Data | Lifecycle gate | Kill switch |
|---|---|---|---|---|---|---|
| Phone ordering agent | Vendor | VP Operations | 3 | Voice, personal | Stuck at G2: no evals | Not tested |
| Delivery dispatch optimizer | Platform | None | 3 | Operational | Not registered | Unknown |
| Shift scheduling assistant | Bought | HR Operations | 3 | Employee | Intake only | Not tested |
| Recruiting screener | Vendor | None | 3 | Employee, personal | Not registered | Unknown |
| Guest support chat | Vendor | Director, Guest Care | 2 | Personal | G3 passed | Tested |
| Finance close helper | Staff-built | None | 2 | Financial | Not registered | Unknown |
| Store operations Q&A | Built | VP Operations | 1 | Internal | G3 passed | Tested |
| Marketing copy assistant | Staff-built | Marketing | 1 | Public | G1 passed | n/a, no write access |
Costed 90-day plan
- 01
Owners and controls Days 0 to 30
Name an owner for every agent, publish risk tiers and minimum controls per tier, pick the identity approach, and write the kill-switch procedure.
- 02
Instrument and evaluate Days 31 to 60
Register the two pilot agents, give each its own identity, route calls through one gateway, emit traces, and build eval suites from real failures.
- 03
Gate and pilot Days 61 to 90
Turn on CI eval gates and approvals, start 6 pilot sites next to 6 control sites, and hold a go, iterate, or stop decision on day 90.
The plan comes with a fixed-price pilot proposal for the two agents, the pass bars each must meet, and the date of the go or stop decision.
How the readout is produced
Interviews with operations, technology, security, and finance; read-only access to existing AI tools and usage data; one Open Agent Lifecycle record per agent, checked against its gates; and, for voice agents, a baseline run of Voice Order Evals. You keep every record, test, and script we produce.