Harness Engineering 101

Harness Engineering: The Next Frontier in AI

Introduction to Harness Engineering

• The discussion begins with an introduction to harness engineering, a term gaining traction among users of AI tools like Claude Code and Codex.

• It highlights the evolution of engineering practices in AI, particularly focusing on prompt engineering as a foundational concept.

Evolution from Prompt to Context Engineering

• In 2023 and 2024, prompt engineering was emphasized as crucial for optimizing model interactions.

• Context engineering emerged as a significant development, stressing the importance of providing models with relevant information for better performance.

• An example is given where access to past marketing campaign data can enhance chatGPT's assistance in creating new campaigns.

Divergent Perspectives on Context Engineering

• For engineers, context engineering involves designing systems that improve interaction with AI by addressing issues like memory and state management.

• Non-technical users focus more on how to effectively provide necessary information to AI for optimal results.

The Rise of Harness Engineering

• Harness engineering encompasses all elements surrounding a model—systems, tools, and access—that enable effective operation.

• Cursor's recent launch of Cursor 3 exemplifies harness engineering by integrating multiple agents into a unified workspace for software development.

Recent Developments in Harness Engineering

• Cursor 3 aims to streamline workflows by allowing parallel agent operations and seamless transitions between local and cloud environments.

• Claude Managed Agents were introduced, emphasizing the pairing of performance-tuned agent harnesses with production infrastructure.

Industry Insights on Harness Engineering

• A blog post titled "Scaling Managed Agents" discusses the metaphorical distinction between the 'brain' (model) and 'hands' (harness).

• The ongoing debate within harness engineering revolves around the balance between powerful models versus robust harness systems.

Conclusion: The Future of Agent Labs

• The conversation reflects broader industry trends regarding the value derived from both human expertise and system capabilities in achieving successful outcomes.

Harness Engineering: Understanding Its Role in AI Development

The Concept of Harness in AI

• The term "harness" refers to a layer that connects, protects, and orchestrates components in engineering disciplines without performing the work itself.

• Boris Churny emphasizes that the harness for Claude Code is minimal, allowing the model to express its full potential as intended by its creators.

Insights from Industry Experts

• Noam Brown from OpenAI notes that before reasoning models emerged, complex systems were built around non-reasoning models. Now, simpler reasoning models can handle tasks directly without additional scaffolding.

• Latent Space's Jerry Liu argues that while agent reasoning is improving rapidly, users' ability to engineer context and workflows remains a significant barrier to maximizing AI value.

The Importance of Harness Engineering

• Harness engineering is recognized as valuable despite some biases towards large model approaches. It focuses on how users can effectively configure models for better performance.

• A post by Kyle from humanlayer.dev highlights that many failures with coding agents stem not from model limitations but configuration issues.

Configuration Challenges and Solutions

• Users often mistakenly believe they need better models (e.g., GPT-6), but the real issue lies in how they configure existing models for specific tasks.

• Kyle suggests focusing on optimizing current models rather than waiting for future advancements to solve existing problems.

Enhancing Coding Agents Through Harness Engineering

• Many users have unknowingly engaged in harness engineering by configuring their coding agents through various methods like using skills or memory files.

• A coding agent consists of both AI models and a harness; understanding this relationship is crucial for improving output quality and reliability.

Defining Harness Engineering Practices

• Harness engineering involves leveraging configuration points to enhance coding agents' capabilities beyond just prompts.

• This practice addresses how to teach coding agents about codebases not included in training data and improve task success rates.

Diverse Applications of Harnesses

• Harnesses are designed based on what models cannot do natively, creating components that fill those gaps.

• Techniques such as auto research or specific loops are examples of harness additions aimed at achieving desired behaviors in long-term projects.

Harness Engineering and the Future of AI Development

The Concept of Harness Engineering

• OpenAI's post on harness engineering emphasizes building software products with zero manually written code, highlighting a shift in engineering approaches.

• A key experiment revealed the need for "progressive disclosure," allowing agents to access only necessary context without overwhelming their information capacity.

• The challenges now focus on designing environments and feedback systems that enable agents to build and maintain complex software reliably at scale.

Three-Layer Architecture of Harnesses

• An etna labs post outlines a three-layer architecture for harnesses:

• Information Layer: Manages what an agent can see and its capabilities (memory, context management).

• Execution Layer: Handles work decomposition, collaboration among agents, and failure recovery (orchestration).

• Feedback Layer: Ensures system improvement over time through evaluation, verification, and observability.

Performance Insights from Blitzy

• Blitzy achieved a performance score of 66.5% on SWE Bench Pro, outperforming GPT 5.4's score of 57.7%, indicating the effectiveness of harness layers.

• The findings suggest that the infrastructure around foundation models (agent scaffolding and orchestration) can yield greater performance gains than improvements in the models themselves.

• Blitzy's success stemmed from its knowledge graph providing deep contextual understanding that raw models could not achieve.

Consensus Around Harness Power

• Nicholas Charrier notes growing consensus about the significance of harnesses in AI development; many companies are converging towards similar product shapes.

• Various tech companies are developing coding agents or focusing on similar architectures, indicating a trend towards unified solutions across different platforms.

The General Harness Concept

• Charrier introduces the idea of a "general harness," where user input interacts with context engineering before reaching the model, creating a loop until task completion.

• This new technique is versatile enough to apply to various computer-based tasks beyond coding due to its scalable architecture.

Predictions for Software Development Trends

• By 2026, many software companies may appear similar as they adopt self-improving systems capable of achieving business outcomes through goal-oriented processes using tools.

• The convergence in architecture will lead to faster progress for companies that control more aspects of these loops; success will depend not just on better models but also on effective orchestration.

Harness Engineering and Managed Agents

The Evolution of Harness Engineering

• Anthropic's Managed Agents signify a shift in harness engineering, focusing on distribution, trusted workflows, and proprietary context to enhance the observation-to-improvement process.

• The blog post emphasizes that as models improve, assumptions about harnesses and code can become outdated; thus, managed agents are designed to maintain stable interfaces amidst these changes.

• A key insight is that harnesses should adapt to address agent behaviors not natively present in models, highlighting the need for continuous questioning of existing assumptions.

• An example illustrates how Claude 4.5 exhibited "context anxiety," leading to premature task completion; adjustments made for this behavior became unnecessary with newer model iterations.

• Managed agents are described as a cloud-hosted service that runs long-horizon agents through interfaces designed to remain relevant despite evolving implementations.

Meta-Harness Concept

• Anthropic aims to create a meta-harness that remains unopinionated about specific designs due to anticipated changes in harnesses as models advance.

• This separation of the agent loop (brain), execution environment (hands), and event log (session) allows for independent failure or replacement of components, impacting discussions around big harness versus big model debates.

• The infrastructure being developed by Anthropic suggests that while harness engineering is crucial, it should be viewed as temporary; the discipline itself is what endures over time.

Importance of Harness Engineering

• Users engaging with tools like Claude Code or Codex are already participating in harness engineering by structuring their environments effectively for agent performance.

• Brigida Bocular differentiates between user-built outer harnesses and inner ones created by developers like Anthropica or OpenAI; the outer harness significantly influences output quality based on specific goals.

• For enterprise leaders, understanding this framework shifts focus from merely selecting models to creating optimal environments where AI capabilities can flourish.

Broader Implications

• The concept extends beyond individual tools; it highlights the necessity of designing systems where AI can thrive rather than relying solely on technology implementation.

• Observing trends across products reveals a convergence towards similar functionalities driven by general-purpose core loops in AI models—illustrating why various companies are developing coding and work agents.