Who this is for
Developer or data engineer whose live model step for extraction or tagging must not silently get worse.
A provider update or a prompt edit changed the output once and went unnoticed for weeks.
The result
Every month you receive a report that scores the outputs of your golden-set run, plus 20 fresh rows from last month's live outputs that you label. It shows the scores against the floor, and the rows that changed. Any fall below the floor is raised in the report with the evidence.
What is included
- One live model step with a golden set, from our golden-set job or your own
- A monthly scoring of the golden-set outputs you send (you run the harness, ours or yours, with your own key, or send recorded outputs)
- 20 fresh rows a month from last month's live outputs, labelled by you
- A scored report: floor, scores, changed rows and shape check
- A golden-set history, kept as the durable record
You receive
- A monthly scored report
- The list of rows that changed since last month
- The updated golden-set history
- A short note on any fall and its evidence
What is not included
- Access to your production step, data or provider key
- Improving or retuning your prompt or model
- Fixing a regression: findings are reported, and any fix is a separate agreed job
- Out-of-hours cover or any guaranteed response time
- A model as the grader
What we need from you first
- Which step, which tool it runs in and how often it runs
- Whether you have a golden set already
- No real records, personal data or credentials in the first enquiry
Never send passwords, keys, customer records or confidential code in the first enquiry. Secure handover is agreed after scoping.
How we check it is done
- Each month's report covers the golden set and the 20 fresh rows, and its scores can be recomputed from the files returned.
- Any score below the floor is raised in the report with the rows that changed.
- No live access was used: the sample came from you, and nothing was changed in your systems.
- The report states the rows scored and the rows that changed, and no model graded any output.
You read each report and tell us whether the marks and rules still suit you. A month counts as delivered when you have received the report.
When we would stop or decline
- No golden set exists and none can be built
- Fresh rows cannot be redacted
- You want us to hold your provider key or production access
- The output shape changes every month so scores cannot be compared
Questions
Do you run my step?
No. You send the golden set results and a monthly sample. We score them.
What if the score drops?
The report says so with the rows that changed. A fix is a separate agreed job.
Do I need the one-off golden-set job first?
Not if you already have a labelled golden set and thresholds.
Price and terms
£99/month · untested offer price. Billing is monthly under the written terms we agree with you. Nothing is charged until you have agreed them.
This is a new service with no published client results. The price is a starting point we have not yet tested with buyers. Nothing is ordered or charged by the enquiry. The full specification is on the Synthetic Industry catalogue.