How do you measure and prove accuracy over time?
Three layers: refuse to guess at question time, record every model call in an append-only ledger, and score releases against golden question sets in a standalone evaluation service.
At question time
The system is built to decline rather than improvise. When a question does not map cleanly onto the schema it asks, with candidate columns drawn from the live schema. An answer that arrives without a clarification is one where the mapping was unambiguous.
Continuously
An append-only credit ledger records every model operation with per-agent attribution, latency and outcome. When a number is disputed, there is a trace showing which agent produced it, on what, and how long it took.
Per release
A standalone evaluation service with golden question sets per vertical, six specialized scorers, and regression tracking across seven accuracy dimensions. This is the framework listed on this page as in development, and it is deliberately separate from the product so it cannot grade itself on the runtime's terms.
The honest version
Accuracy claims from a text-to-SQL vendor should be treated as a hypothesis until they are tested on your schema and your vocabulary. That is precisely what the pilot is for: golden questions from your domain, scored against your data, before anything converts.
Where this comes from
See it against your own data
A pilot is scoped to one governed use case and time-boxed to eight to twelve weeks, with success criteria agreed before it starts.
Get started freeRelated questions
- What is dataface.ai?dataface.ai is on-source conversational analytics. Ask a question in plain English, and it generates read-only SQL from your schema metadata and runs it on the systems you already own. No warehouse copy, no pipeline.
- How do you run analytics without moving our data?dataface.ai reads your schema and a few example values per column, generates read-only SQL, and executes it on the source system itself. Your tables are never copied: no pipeline, no warehouse copy, no second place to secure.
- What does the LLM actually see from our data?By default the model sees your catalog: table names, column names, types, and a few example values per column. It never receives your tables, and it only sees query results if a user turns on AI analysis.
- How do you stop an AI from writing to or damaging our database?Generated SQL is SELECT-only. INSERT, UPDATE, DELETE and DDL are blocked, queries are parameterized against injection, and every request passes both a workspace gate and a datasource gate.