Ornith (35B,9B) + Hermes,Zed: THE FULLY PRIVATE LOCAL AGENT is ACTUALLY HERE!

Introduction to Ornith: A New Family of Coding Models

Overview of Ornith

• Ornith, developed by Deep Reinforce, is a new family of coding models ranging from a small 9B model to a large 397B model. The larger model is not practical for local use.

• The 9B and 35B versions are more accessible for users wanting to run models locally or on servers. These models build upon existing frameworks like Gemma and Quinn.

Key Features of Ornith

• Ornith's main innovation lies in its ability to create its own scaffolding while solving tasks, enhancing the coding process beyond just generating answers.

• It focuses on planning, error handling, tool usage, file checking, and testing—critical aspects for effective coding agents. A strong agent loop can significantly improve user experience with coding models.

Addressing Reward Hacking in Model Training

Concerns with Benchmarking

• There is a risk that models may learn to "game" benchmarks rather than genuinely solve tasks (e.g., hardcoding outputs). Deep Reinforce aims to mitigate this through fixed environments and monitoring systems.

• Real-world testing remains essential as benchmark results may not reflect actual performance in practical applications.

Performance Metrics of Different Models

Model Comparisons

• The 397B model scores approximately 77 on Terminal Bench and around 82 on SweBench, performing comparably to Claude Opus in some areas but impractical for most users due to size constraints.

• The more manageable 35B model achieves scores of about 64 on Terminal Bench and around 75 on SweBench, indicating solid performance for its size if it functions well in real tools.

• The smallest version, the 9B model, scores around 43 on Terminal Bench and approximately 69 on SweBench; these numbers suggest it could be an excellent option for local use if performance translates effectively into practice.

Important Setup Considerations

Running the Models Effectively

• As a reasoning model, Ornith can produce reasoning tags and tool call blocks; improper runtime handling may lead to suboptimal performance indicators such as broken calls or formatting issues. Users should follow recommended setups closely for best results.

• Suggested methods include using recent runtimes and appropriate templates when deploying the models locally via platforms like LM Studio or Ollama; starting with the 9B version is advisable for general users seeking ease of use.

Personal Testing Experience with the Models

User Insights

• The tester found that the 35B model works well with agents like OpenCode without frequent glitches compared to similar models (e.g., Qwen 3.6), making it a reliable choice for local applications despite limitations in casual chatting capabilities.

• While comparisons are made against other advanced models (like Opus), it's emphasized that smaller versions need only excel at tool calls rather than compete directly with high-end counterparts; they perform adequately without significant glitches—a common issue among many similar offerings today.

Conclusion: Overall Impressions of Ornith

Final Thoughts

• Despite some shortcomings (e.g., issues with specific simulations), overall impressions indicate that post-training has been executed effectively within these models.

• When integrated with tools like Hermes Agent, the potential utility increases significantly compared to other available options—highlighting their value as robust local coding solutions moving forward.

Let me know your thoughts! If you enjoyed this video or found it helpful consider supporting through Super Thanks or membership options! Don't forget to subscribe!