Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards

Summary

The video presents TEF, an approach to training agents to handle long, multi-step workflows. Its central idea is to address two problems at once: the shortage of scalable long-horizon training data and the lack of useful feedback when an agent receives a score only at the end of a task.

Why long-horizon training is difficult

• Long-running agent tasks—such as scientific research, software and infrastructure work, security, or machine-learning research—require agents to maintain goals, recover from mistakes, and make progress over many steps.
• Existing agent-training data consists mostly of short- or medium-length tasks with simple pass/fail outcomes.
• Realistic long-horizon benchmarks are often small, expensive, human-authored evaluation sets rather than large training corpora.
• Terminal-only rewards are especially unhelpful early in training: a rollout can fail after many steps without revealing whether the agent made useful progress.
• This creates a problem for group-relative policy optimization (GRPO). If every sampled rollout in a group fails, there is no meaningful difference in rewards to guide learning.

TEF Gym: Generating verifiable environments

TEF introduces TEF Gym, a dataset of 1,170 verified long-horizon environments across seven domains:

• Software engineering
• Data engineering
• AI and computer-science research
• Mathematics and operations research
• Security and forensics
• Infrastructure administration
• Quantum computing

The environments are built using a “latent world first” strategy: each task is generated from a complete, hidden specification that serves as its ground truth.

The process is:

1. Generate a latent world. A seeded program creates the complete underlying state of the task, such as records, distributed-system shards, or files in a research workflow.
2. Validate the world. The generator checks that the underlying state satisfies consistency rules.
3. Define a workflow. The task is represented as a directed acyclic graph (DAG), with stages, dependencies, and possible parallel work. Its structure determines the task’s depth and horizon.
4. Render the public inputs. The latent world is converted into files or artifacts—such as CSVs, JSON, databases, binaries, or infrastructure manifests—designed to preserve enough information to recover the world unambiguously.
5. Apply controlled corruption. The agent receives inputs with declared, invertible defects, such as missing or duplicate records, schema drift, scrambled indices, or stale caches.
6. Verify the result. Because the task, inputs, and checker all derive from the same latent world, the final output can be compared with known ground truth.

This shared foundation is intended to prevent artifact drift, where a synthetic prompt, its files, and its checker do not describe the same coherent task.

TEF Gym also labels environments along multiple dimensions, including domain, scale, workflow stage, tool requirements, artifact roles, relationships, and corruption types. Scale and workflow structure can be varied to create a difficulty ladder: larger worlds require more reading, joining, and writing, while deeper or differently structured DAGs change the dependencies and repair order.

Before inclusion, tasks go through qualification checks. These test whether gaming strategies fail, whether progress scores behave appropriately on partially correct states, and whether the score, verifier, and reference solvers are independent.

Progress Potential Reward Shaping

TEF’s second contribution is Progress Potential Reward Shaping (PPRS). Because the latent world can verify the final answer, it can also assess intermediate workspace states.

• A progress potential score from 0 to 1 measures how close the agent’s current state is to correct intermediate states.
• The agent receives feedback throughout the workflow rather than only at completion.
• Progress raises the potential; regressions lower it.
• The resulting step-by-step signal can provide useful variation among rollouts, even when none reaches full task completion.

The intended benefit is to make long-horizon tasks learnable earlier in training, including for GRPO, which needs differences in reward among sampled rollouts to form an advantage signal.

Models and reported results

The video describes training TEF 9B and TEF 27B with the recipe, then applying it to a larger base model to produce TEF 320B. Evaluation includes Agent’s Last Exam’s computing and mathematics sciences split (ALECM) and subsets of Terminal Bench.

The video reports that TEF 27B makes substantial gains on ALECM and compares favorably with leading models on that benchmark. It also describes improvements over base models on Terminal Bench subsets and transfer to tasks beyond the training environments. The stated interpretation is that TEF may teach broader workflow competence, rather than simply improving performance on its synthetic tasks.

Limitations and cautions

• Training emphasis is concentrated on ALECM because of resource constraints, even though TEF Gym covers multiple domains.
• The latent-world approach works best when a complete specification can be constructed and intermediate progress can be scored meaningfully. That may be difficult for open-ended research or problems with ambiguous success criteria.
• Dense reward shaping can create incentives to optimize the metric rather than the real objective if the metric is imperfect. The paper’s qualification checks and hidden tests are intended to reduce this risk, but it remains a general concern.

Overall, TEF’s proposed recipe is to generate long-horizon tasks from a known, coherent ground truth and use that ground truth both to verify final results and to reward intermediate progress.

Speakers and sources featured

• Unnamed narrator: Presents and explains the paper and its results; no individual speaker is identified in the subtitles.
• The TEF paper and its reported methods and results: The research source being summarized.
• Visual materials described in the subtitles: Performance charts and rankings, a TEF Gym task-generation diagram, and an AWS billing-dashboard screenshot. No separate speaker is identified for these visuals.