- A small open model fine-tuned on your schema, terms and past queries can match a larger general model on your questions, at a lower cost per query.
- H2O Cascade is H2O's tool for curating training data, fine-tuning and evaluating models on infrastructure you own.
- AT&T fine-tuned an 8B-parameter model for text-to-SQL that beat GPT-4 and ranked #1, then #2, on the global BIRD-SQL benchmark, cutting cost per query from about $0.02 to near $0.
- Running the tuned model and your current model on the same held-out questions from your own data tells you whether the smaller model is accurate enough to switch.
Stop waiting on someone who knows SQL for every question
Your business teams have questions the data can answer: which customers in a region to target, which services a client has not bought yet, how a metric moved last quarter. Each one waits in a queue for an analyst who knows the schema and can write the query.
A general large language model can write SQL, but it does not know your table names, your business terms or which column really holds the number. Getting it right means long prompts on every call, and the bill grows with every question your users ask.
How text-to-SQL works
H2O Cascade is the fine-tuning tool at the center of this workflow. You tune a small model for one job, measure it against the model you use today, and route the traffic when it is ready.
Step 1: curate examples from your own data
Collect the questions your users ask and the SQL your analysts wrote for them. Data profiling and semantic search help you find the tables, columns and terms each question depends on, and H2O Cascade can add synthetic examples and distill answers from a larger model.
Step 2: fine-tune a small open model
Fine-tuning teaches a small language model your schema and vocabulary. H2O Cascade supports LoRA, QLoRA and full fine-tuning, plus reinforcement learning with LLM-as-a-judge, on open models you keep.
Step 3: evaluate against the model you use today
Run the tuned model and your current model on the same held-out questions and compare the results. Evaluating on your own questions, not a public benchmark, tells you whether the smaller model is accurate enough for your data.
Step 4: serve it where your data lives
H2O Cascade serves tuned models from its Model Zoo, or you move the weights to the serving platform you already run.
Results
AT&T fine-tuned a small model for plain-English database queries against its own schema. The tuned, 8B-parameter model beat GPT-4 and ranked #1, then #2, on the global BIRD-SQL leaderboard, cutting the cost per query from about $0.02 to near $0.