Why Speech-Regex Action Supervisors Miss Live Output: An Empirical Evasion Taxonomy and Held-Out Structured Action Contract Evaluation
Sean Maxwell — Roll Solo, Melbourne, VIC, Australia (ORCID: 0009-0004-0133-0463)
Abstract
Autonomous desktop and cloud agents must not assert that state-modifying actions have occurred when no verifiable execution receipt exists. While deterministic speech-regex pattern matchers are widely deployed as low-latency safety guards, live production audits reveal substantial failure rates on paraphrased side effects. We establish an empirical taxonomy of four grammatical evasion modes: active progressive participles, passive and telegraphic confirmations, unlisted verb lemmas, and markdown formatting syntax. Across a pre-registered held-out benchmark of N=120 unseen prompts and a 720-turn blinded semantic re-audit (validated by a second independent rater at kappa = 1.0000), plus 1,000 completions across four Gemma-4 scales, we compare regex supervision, expanded grammar matching, and Structured-Output Action Contracts (SOAC) verified against cryptographic tool receipts in 6.82 microseconds mean lookup latency.
Citation (BibTeX)
@techreport{maxwell2026evasion,
author = {Maxwell, Sean},
title = {Why Speech-Regex Action Supervisors Miss Live Output: An Empirical Evasion Taxonomy and Held-Out Structured Action Contract Evaluation},
institution = {Roll Solo},
number = {LUCCA-TR-2026-02},
year = {2026},
doi = {10.5281/zenodo.23901605},
url = {https://doi.org/10.5281/zenodo.23901605}
}