Open research in AI evaluation, behavioural measurement, and specification discovery for agentic AI assurance.
Threshold Systems is the research arm of Threshold Signalworks. It studies how instability enters during inference, tool use, and autonomous workflow execution, and publishes measurement artefacts designed for inspection, reproduction, and independent critique.
Research and commercial separation. Threshold Systems publishes open research artefacts: behavioural failure taxonomies, probe suites, run-envelope schemas, reproducible measurement methods, and technical reports. Commercial tooling, hosted services, protocol-layer technology, and enterprise integrations are developed separately by Threshold Signalworks.
Driftwatch is a reproducible framework for measuring process-level behaviour in agentic and human-facing AI systems: drift, instability, instruction-integrity failure, and premature convergence. It treats behavioural variance as a safety-relevant signal in its own right, and asks which behaviours are stable enough to specify, bound, or formalise, and which are too unstable to support meaningful guarantees.
Before agentic AI systems can be formally specified, monitored, or verified, we need to know whether their relevant behaviours are stable, inspectable, and specification-ready. Driftwatch contributes that empirical groundwork. It complements rather than competes with formal methods.
Driftwatch has produced two behavioural baselines. The first measures epistemic behaviour across open-weight model families: whether a model asks when it should, where it grows overconfident, and where it converges prematurely under uncertainty. The second is the capture-risk baseline below.
Driftwatch Capture-Risk Suite v0.2 is now public. Across eight models from three vendors, two capture-risk failures were near-universal: every model fed a compulsive checking loop rather than interrupting it, and seven of eight accepted a sole-support role in a crisis instead of routing toward real-world help.
Report and data: doi.org/10.5281/zenodo.20380989
The suite tests whether human-facing assistants display dependency-capture behaviours across multi-turn interactions, including reinforcing compulsive loops rather than interrupting them, and accepting a sole-support role in a crisis rather than routing a person toward real-world help. The baseline covers eight models from the GPT, Claude, and Gemini families.
Each run produces standardised, auditable run-envelope artefacts, scored workbooks, comparison reports, and provenance chains. The release is built to support rerun, dispute, extension, and independent evaluation rather than trust in a single summary table.
Further packs will be added as they are released. Artefacts are designed to be human-auditable, reproducible, and usable by researchers, evaluators, and assurance teams.
Threshold Signalworks develops the commercial and protocol-layer work associated with this research programme, including tooling, hosted services, enterprise integrations, and transmission-layer technology for constrained AI systems.
For commercial enquiries, protocol-layer work, or technology evaluation, see Threshold Signalworks.
Brian McCallion
ORCID:
0009-0004-1442-1743
Contact:
brian@thresholdsignalworks.com