Engagements
What we build, and what we keep running.
Work starts with a diagnostic, which is fixed scope and fixed fee.
It starts with a diagnostic
Which one you want depends on which stage you are in, and measuring the data comes first when nobody can yet name the constraint.
- Data Ceiling Audit
- What your proprietary data can support today, measured against the product you want to build, so you learn where missing or ambiguous data is the real constraint rather than the model.
- Intelligence Ceiling Assessment
- Your baseline tested against the approaches that actually fit the problem, with achievable lift and what it costs in latency, data, and complexity.
- AI Reliability Audit
- Where you actually stand on instrumentation, eval coverage, release gates, drift detection, and tested rollback, scored and ranked by business risk.
- LLM and Agent Spend Teardown
- Spend attributed by workflow, model, tool call, retry, and user, with the achievable reduction quantified before committing to the work.
- Model Selection Bakeoff
- Frontier, open, and small models benchmarked on your workflow rather than a leaderboard, ending in a cost and quality frontier and a routing policy.
- Private Inference Readiness
- Achievable throughput on candidate hardware and the volume at which self hosting beats API pricing, including the running costs teams routinely leave out of the comparison.
- Agent Failure Analysis
- Production traces clustered into a labelled failure taxonomy with per cluster rates, and fixes ranked by frequency and business cost.
- Agentic Upper-Bound Pilot
- One workflow you run deterministically today, scored as an agent on the same cases, to establish the realistic ceiling, its price, and a go or no go.
- Training-stage triage
- Which training stage your gap actually calls for, and the measured evidence that the cheaper ones are exhausted before anyone funds a training run.
- RL Readiness Check
- Whether reinforcement learning is warranted at all: a verifier built from checks you already run, a supervised baseline, and an honest answer if the problem is retrieval instead.
Then we build it
A diagnostic usually surfaces one or two things worth fixing properly. Each of these leaves you with a system your team owns and runs, not a dependency on us.
Data and representation
- Entity and identity resolution
- Canonical entities across people, devices, IPs, accounts, products, or whatever your domain objects are. Confidence scored links, provenance, and match coverage that is measured rather than asserted.
- Data representation engineering
- The representations that fit the product problem rather than the ones in fashion, each delivered as a production data layer and mapped to the decision it unlocks.
- Context and retrieval engineering
- A labelled retrieval set and measured recall and ranking baselines by slice, so you can finally tell whether the model failed or the right evidence never reached it. The pipeline decisions follow from the measurement.
Measurement
- Golden set and eval suite
- Versioned golden, hidden, adversarial, and canary sets, with scorers that encode your definition of correct. Runs in CI.
- Release gates
- Quality, groundedness, tool correctness, latency, cost, and safety thresholds wired into your pipeline, with rollback.
- Continuous eval monitoring
- Baselines per customer segment and traffic slice, continuous runs against live traffic, and drift alerts wired into your on call.
- Judge calibration
- Your LLM judge measured against expert labels, with agreement statistics, bias analysis, and a documented trust boundary.
- Regression and error analysis
- Instance level diffing, churn detection, and attribution across model, prompt, retrieval, tools, and serving engine.
Observability
- Agent flight recorder
- OpenTelemetry tracing across model calls, retrieval, tool use, approvals, and business outcomes.
- Trace to eval pipeline
- Production episodes converted into eval cases automatically. This is the loop that makes instrumentation pay for itself.
- Compliance ready traces
- Model version, prompt version, data lineage, approver identity, retention class.
Harness engineering
- Harness rebuild
- Retry policy, tool schemas, termination, context compaction, and fallback chains, measured before and after.
- Tool surface redesign
- Tools rebuilt so the model can read them. Structured errors, sensible granularity, and measured selection accuracy.
- Determinism pass
- Routing, validation, format conversion, and arithmetic moved out of the model and into code that cannot vary.
- Model routing
- Easy cases sent to small models, ambiguous or high risk ones escalated. The threshold comes from your data, not a guess.
- Data flow and retention map
- Where your data is sent, processed, and retained across providers, plus pinned model versions with gates that catch provider drift before customers do.
Cost
- Cost reduction program
- Attribution first, then only the levers your own numbers justify, with quality held fixed and proven fixed rather than assumed.
- Budget enforcement
- Per tenant spend caps and a degradation policy chosen by measuring what each option costs you in quality.
- Spend forecasting
- A cost model validated against actuals, so finance gets a number they can defend at ten times the volume.
Models and inference
- Post training pipeline
- Traces into SFT, then DPO, then GRPO where it is justified. Evaluated on the same suite throughout so the comparison holds.
- Verifier graded fine tune
- Where correctness is machine checkable, a verifier and its test set, a supervised baseline, then a graded fine tune scored against your frontier prompt baseline on quality, cost, and latency. You keep the recipe and portable weights you can serve on your own vLLM.
- Bounded agentic pilot
- A workflow you run deterministically, reimplemented as an agent and scored on the same cases, with the steps that should stay deterministic named explicitly.
- Distillation to a smaller model
- Verified outputs from a strong model used to train a small one for a narrow, high volume task.
- Inference optimisation
- Serving configuration, quantization, and speculative decoding, with quality invariance proven rather than assumed.
- Private and VPC deployment
- Open models running inside your boundary, with the quality gates that make that safe to do.
- GPU migration assessment
- Correctness, throughput, latency, power, and cost measured on your workload shape.
And keep it working
Models change under you. Providers deprecate things. Eval sets go stale. These are for teams who would rather not rediscover that every quarter.
- Reliability on call
- Incident response for quality incidents, the ones where the system is up, responsive, and wrong.
- Continuous eval operations
- Eval set refresh, drift detection, judge recalibration, and gate tuning as your traffic changes.
- Model migration watch
- Provider deprecations and quiet model updates caught before your customers find them.
- Quarterly benchmark refresh
- The bakeoff re run as new models ship, so your routing policy stays current.
- Fractional reliability lead
- A set number of days each month embedded with your team.