A Compile-Time Safety Contract for LLM-Authored Fitness Plans
A Reproducible 240-Trial Benchmark of WPL Governance Across the OpenAI Lineup
Alex Filatov · Gymbile · May 2026 · 23 pages
Abstract
Consumer-grade AI fitness products increasingly let large language models (LLMs) emit free-form workout and nutrition plans directly to end users. We present a reproducible, version-controlled safety benchmark of this deployment pattern. The corpus release (v0.5.0) contains fifteen trainer-voice scenarios spanning medical contraindications, menstrual-cycle adaptation, and constraint adherence. Each scenario is run through two pipelines using the same model and the same prompt: Lane A is the raw LLM behavior found in commercial products today; Lane B routes the model through the Wellness Plan Language (WPL), a domain-specific language with compile-time validation, a canonical exercise vocabulary, and a rule evaluator that re-applies client constraints on every regeneration.
We evaluate the four OpenAI models in the v0.5.0 corpus (gpt-5, gpt-5-mini, gpt-5-nano, gpt-4.1) across single-turn and 8-turn multi-turn protocols, yielding 240 trials at a total inference cost of $37.27. Raw LLM output produced 207 safety violations across 43/120 trials (36%); the WPL public layer reduced this to 28 violations across 6/120 trials (5%) — an 86% reduction on both metrics. Multi-turn drift, where the model loses track of a stated contraindication at some turn N > 1, occurred in 25/60 (42%) raw conversations and in 0/60 WPL-governed conversations.
A four-arm prompt ablation shows the safety-guarantee mechanism is not what the architecture diagram suggests: the explicit "do-not-prescribe-contraindicated" instruction has no measurable effect, while the DSL’s commitment-forcing structure and compile-time fail-closed behavior together drive the result. We publish every prompt, every model output, every score, and the deterministic scorer; every quantitative claim is offline-reproducible without further API spend.
Key results
- → 207 → 28 safety violations under WPL governance (−86%)
- → Unsafe trials: 43/120 (36%) raw → 6/120 (5%) governed
- → Multi-turn drift: 25/60 (42%) raw → 0/60 governed
- → Ablation: the "do not prescribe" instruction has no measurable effect; the DSL structure and fail-closed compile gate drive the safety result
About this version
Initial draft. OpenAI-only lineup, 15 scenarios, 240 trials. Superseded before arXiv submission by the v0.6.0 cross-vendor version; published here as part of the research record.