Define the workflow
Start with a task your team understands well enough to judge.
Free evaluation resource
Private AI Evaluation Checklist
Choose one workflow, collect evidence your team can review, and make a clear decision before expanding a private AI deployment.
Free and ungated. No email address is required.
The six checks
Evaluate the workflow, not just the model.
Build a representative test set
Use approved examples that resemble the work, including difficult cases.
Map the complete data path
Private inference is one part of the workflow. Trace every connection.
Freeze the tested configuration
A result applies to the combination that produced it.
Check control and operations
Prove both intended access and day-to-day ownership.
Set the decision gate before testing
Agree on evidence before seeing a polished answer.
A simple stopping rule
Do not advance without evidence.
Pause when the workflow has no owner, the test set cannot be judged, the data path is unclear, or the result cannot be reproduced. A convincing response is not enough by itself.
Record the decision
- What passed and failed?
- Which limitations remain acceptable?
- What must change before another evaluation?
- Who owns the next action and date?
Srasta Field Notes
Keep evaluating AI with practical guidance.
Get a short note every two weeks for IT and engineering leaders evaluating, deploying, and operating AI.
Subscribe when it is useful to you.
Subscription is optional and handled by Substack, including confirmation and unsubscribe controls.
Subscribe to Field Notes →