Agent Training Needs Isolated Practice Environments

Agent training is moving closer to real tools and business processes. Microsoft Research has released Agent Lightning v1.0, a compact reinforcement-learning framework for existing agent harnesses. In parallel, OpenAI and Ironclad are building realistic contract workspaces in which agents complete and are evaluated on complex legal workflows.

Learning from the actual workflow

Agent Lightning separates agent execution from training. Existing harnesses can report interactions and outcomes to a separate training process without a complete architectural rewrite. Microsoft describes a coding example using Qwen3.5-9B: with about 6,000 training samples, Pass@1 on SWE-bench Verified increased from 41.8% to 56.4%. This is a vendor research result, not a general productivity benchmark.

OpenAI and Ironclad take a domain-specific approach. Their assessment suite recreates contract work in workspaces containing realistic documents, emails, context, and objectives. Evaluation covers more than the final answer: the agent must find relevant information, use tools across several steps, and change the workspace state correctly.

Why training needs its own boundary

Once agents learn from failures, scores, or trajectories, a test question in a chat window is no longer enough. The system needs a controlled practice environment with a defined starting state and a verifiable final state. Production systems are unsuitable for this purpose: a training run must not modify real contracts, close live tickets, trigger payments, or expose customer data.

A robust training harness should include at least five controls:

• Isolation: dedicated accounts, data copies, credentials, and network boundaries instead of production access.
• Reset: every run starts from a reproducible state and leaves no hidden residue.
• Success criteria: evaluate the actual system state, not only a plausible textual answer.
• Edge cases: explicitly test timeouts, missing data, incorrect tool choices, and cancellation.
• Approval: an improved training score reaches production only after regression testing, security review, and business acceptance.

The DACH issue is data and evidence

DACH organisations must define the purpose, retention, and access rules for training data, traces, and evaluator feedback. Teams should prefer synthetic or controlled anonymised data when real content is unnecessary. Every release also needs evidence: version, test set, success thresholds, known exceptions, and a named owner.

Train agents where failures remain visible, reversible, and harmless. An agent may fail in the practice environment — it must not learn in production.