The 5 Levels of AI Coding (Why Most of You Won't Make It Past Level 2)

Name: The 5 Levels of AI Coding (Why Most of You Won't Make It Past Level 2)
Uploaded: 2026-02-18T15:01:07.000Z
Duration: 1 h 24 min 15 s

The Future of Software Development: AI's Role

The Paradox of AI in Coding

90% of cloud code is generated by Claude Code, yet developers using AI tools often experience slower performance initially.

Strong DM operates a software factory where no human writes or reviews code; only specifications are provided, and outcomes evaluated.

Anthropic reports that nearly all code produced is AI-generated, with leadership noting that Boris Triny hasn't personally written code in months.

A study found experienced developers using AI tools took 19% longer to complete tasks compared to those not using them, raising questions about efficiency.

Developers overestimated their speed increase with AI by 24%, indicating a disconnect between perception and reality.

Levels of Vibe Coding Framework

Level Zero: Spicy Autocomplete

This level involves basic autocomplete features like GitHub Copilot, where the human still primarily writes the code while the AI suggests completions.

Level One: Coding Intern

At this stage, humans assign discrete tasks to the AI (e.g., writing functions), but they still review everything produced by the AI.

Level Two: Junior Developer

The AI can handle multifile changes and navigate dependencies. Humans review more complex outputs but remain involved in reading all generated code.

Level Three: Developer as Manager

Here, developers direct the AI's work without writing much code themselves. They focus on reviewing pull requests rather than implementation details.

Level Four: Developer as Product Manager

In this level, developers write specifications and check outcomes after tests pass without delving into the actual coding process.

Level Five: Dark Factory

This advanced stage represents an autonomous system where no human involvement occurs in coding or reviewing; specifications directly translate into working software.

Understanding Level Five Software Development

The Concept of Agentic Software Development

Vendors often refer to their tools as code-writing solutions, typically indicating a basic level of automation.

Startups claiming to engage in agentic software development usually operate at a more advanced level (level two or three).

Strong DM's assertion that their code must not be human-written signifies they are functioning at the highest level (level five), known as the "dark factory."

The Gap Between Marketing and Reality

There exists a significant disparity between marketing language and actual operational capabilities in software development.

Addressing this gap requires comprehensive changes beyond merely selecting better AI tools; it is fundamentally a people problem rather than just a tool problem.

Strong DM Software Factory: A Case Study

Strong DM's software factory exemplifies level five software development, recognized by Simon Willis as an ambitious form of AI-assisted coding.

This case study illustrates how visionary concepts can be realized with current technology, particularly looking ahead to advancements expected by 2026.

Team Composition and Key Milestones

The Strong DM team consists of three members: Justin McCarthy (CTO), Jay Taylor, and Nan Chowan, who have been operating since July last year.

They identified Claude 3.5 Sonnet as a pivotal point for agentic coding, marking when correctness began to compound rather than errors.

Innovative Testing Approaches

Unlike traditional testing methods that reside within the codebase, Strong DM employs external scenarios for evaluation, preventing the AI from gaming the system during development.

Scenarios serve as behavioral specifications stored separately from the codebase, ensuring unbiased assessment of software functionality post-development.

Digital Twin Universe and Integration Testing

Strong DM utilizes what they call a "digital twin universe," creating simulated environments for every external service interaction (e.g., Octa, Jira). This allows full integration testing without affecting real systems or data.

Their output includes substantial lines of production-ready code across various programming languages (Rust, Go, TypeScript), demonstrating successful end-to-end autonomous development by agents.

Cost Efficiency in Software Development

Strong DM emphasizes that if organizations aren't investing $1,000 per human engineer daily in their software factories, there is room for improvement in efficiency and output quality.

Codeex and AI-Generated Code: A New Era in Software Development

The Evolution of Code Generation

Codeex analyzes training logs, flags failing tests, and suggests fixes to training scripts, showcasing a significant advancement in AI capabilities.

Claude Code has built 90% of its own code, with expectations to reach 100%, indicating a shift in developer roles towards specification and oversight rather than direct coding.

Anthropic's approach emphasizes that their engineers are now architects directing machines to implement code, highlighting the efficiency gained through AI assistance.

Impact on Software Development Speed

The rapid development of co-work was facilitated by directing machines rather than manual coding by engineers, leading to faster project completion.

Claude Code achieved a billion-dollar run rate within six months post-launch, demonstrating the commercial viability of AI-generated code tools.

Feedback Loops and Productivity Challenges

The feedback loop for improving AI is closing; the focus is on how quickly this acceleration will impact software developers globally.

A study found that open-source developers using AI tools completed tasks 19% slower due to workflow disruptions outweighing generation speed.

Trust Issues with AI Tools

Developers face challenges such as evaluating suggestions from AI, correcting errors in generated code, and context switching between their mental models and model outputs.

Approximately 46% of developers express distrust in AI-generated code due to reliability concerns despite being experienced professionals.

Adoption Curve and Workflow Redesign

The adoption curve illustrates that productivity often dips before improving when integrating an AI tool into existing workflows without redesigning them.

Many organizations are currently experiencing this dip but misinterpret it as evidence against the effectiveness of AI tools.

Real-world Implications of GitHub Copilot

GitHub Copilot boasts significant user numbers but presents complexities in production environments like larger pull requests and increased review costs.

While Copilot may reduce writing costs, it raises ownership expenses due to security vulnerabilities introduced by generated code.

Successful Integration Strategies

Companies achieving substantial productivity gains (25%-30%) have redesigned their entire development workflows around AI capabilities rather than merely adopting new tools.

Effective integration involves changing specifications, review processes, expectations for different engineer levels, and CI/CD pipelines to accommodate new error categories from AI-generated code.

The Impact of AI on Software Development and Organizational Structures

The Rise of Dark Factories

The emergence of tools like Opus 4.6 and Codeex 5.3 empowers software engineers, but 95% lack the knowledge to utilize these effectively.

This situation highlights a deeper issue within software organizations, which were originally designed for human collaboration in coding.

Human Limitations in Software Development

Traditional processes such as stand-up meetings and sprint planning exist to address human limitations in coordination and task management.

As AI takes over coding tasks, existing structures become friction points rather than necessities.

Shifting Roles in Software Organizations

In an AI-driven environment, traditional roles like scrum masters and engineering managers are challenged; their focus shifts from coordination to specification clarity.

Engineering managers must redefine their value proposition from team coordination to articulating clear specifications for AI agents.

New Skills Required for Success

The transition necessitates new skills focused on articulation rather than mere coordination among teams.

Coaching engineers to write precise specifications becomes crucial, as ambiguity can lead to incorrect implementations by AI.

Understanding Legacy Systems

Most enterprise software is built on legacy systems that cannot easily adapt to dark factory models; they rely heavily on institutional knowledge.

Effective specification is hindered by the lack of documentation regarding implicit decisions made over years of development.

The Path to Autonomous Software Development

Understanding the Transition to Autonomous Systems

The journey towards more autonomous software development begins with creating a specification that accurately reflects existing software functionality, rather than immediately deploying an AI agent for code writing.

Reverse engineering the implicit knowledge within a running system is challenging and requires human expertise, including insights from engineers familiar with specific edge cases and architects who recall historical decisions made under pressure.

Essential human capabilities such as domain expertise, honesty, customer understanding, and systems thinking are increasingly vital in the evolving landscape of software development.

Migration Steps for Organizations

The migration path varies by organization but typically starts with leveraging AI at levels two or three to enhance developers' productivity in writing features and fixing bugs.

Next steps involve using AI for documentation purposes—generating specifications from code and building scenario suites that reflect actual system behavior.

Organizations must then adapt their CI/CD pipelines to accommodate AI-generated code, which includes implementing new testing strategies and review processes.

Eventually, organizations can shift new development efforts towards higher levels of autonomy while maintaining legacy systems; this transition is time-consuming and requires honest documentation practices.

Talent Dynamics in Software Engineering

A significant decline in junior developer employment has been observed following widespread adoption of AI coding tools; projections indicate further drops in job availability across various regions.

The traditional apprenticeship model of software engineering is threatened as AI takes over tasks typically assigned to junior developers, leading to concerns about mentorship opportunities diminishing.

As juniors rely on hands-on experience for learning through simple feature development and bug fixes, the absence of these opportunities due to automation raises questions about future skill acquisition pathways.

Evolving Skill Requirements

Despite a reduction in entry-level positions, there remains a high demand for skilled engineers; however, expectations have shifted significantly regarding the competencies required for junior roles.

Future junior developers will need advanced systems design skills previously expected of mid-level engineers due to automation taking over simpler tasks that once provided foundational learning experiences.

Engineers must now possess deeper judgment capabilities—identifying potential system failures or security gaps—skills that cannot be easily automated by AI tools alone.

Effective communication skills are essential; juniors must articulate specifications clearly enough for autonomous agents to implement them correctly while anticipating unasked questions.

The Shift Towards Generalists in Tech

Changing Career Landscape

The concept of being "adequate" is no longer sufficient for career advancement in tech; the industry is moving towards hiring generalists who can think across various domains rather than specialists focused on narrow technologies.

Value of Generalists Over Specialists

As AI takes over implementation tasks, the value of humans lies in their ability to understand broader problem spaces and direct AI effectively, making generalists more valuable than specialists with deep but narrow expertise.

New Training Models for Engineers

Some organizations are adopting a medical residency model for junior engineers, where they learn by collaborating with AI systems and developing judgment about outputs instead of traditional coding from scratch.

Importance of AI-Native Skills

Companies are increasingly hiring juniors who are "AI native," as they bring fresh perspectives into teams that may have been established before the rise of advanced AI tools like ChatGPT.

Upskilling Necessity in Software Engineering

Gartner predicts that 80% of software engineers will need to upskill in AI-assisted development tools by 2027, though the speaker believes this number will reach 100%, emphasizing the urgency for training infrastructure to keep pace with rapid changes.

Organizational Changes Driven by AI

Talent Pipeline Challenges

The rapid evolution of technology means that software engineers must continuously update their skills; those who do not engage with new models risk becoming obsolete quickly.

Characteristics of AI Native Startups

Successful AI-native startups demonstrate significantly higher revenue per employee compared to traditional SaaS companies, indicating a shift in operational efficiency and organizational structure.

Flattening Organizational Structures

The emergence of small teams adept at understanding user needs and directing AI implementations leads to flatter organizational structures, reducing layers typically found in traditional software companies.

Evolving Roles Within Organizations

Middle management roles may either evolve or disappear entirely as automation takes over engineering tasks; only those whose judgment cannot be automated will remain essential within these organizations.

Transformation Needs for Existing Roles

Specific roles such as junior developers and QA engineers face significant transformation challenges due to automation; individuals must adapt their workflows around developing with AI to remain relevant.

The Future of Software Development in the Age of AI

The Role of Agents in Career Development

Emphasizes the importance of agents in personal development, which varies based on technology stack, budget for token spend, and willingness to learn.

Demand for Software and Intelligence

Highlights that there has never been a ceiling on the demand for software or intelligence; as computing costs decrease, software production increases exponentially.

Economic Viability of New Software Categories

Discusses how lower costs have made new categories like SaaS and mobile apps viable, transforming industries by enabling solutions previously deemed economically impossible.

Addressing Unmet Needs in Various Industries

Notes that many companies cannot afford custom software due to high labor costs; traditional systems are often replaced with spreadsheets as a workaround.

Exploding Total Addressable Market

Asserts that the total addressable market for software is rapidly expanding due to reduced production costs, making it feasible to serve markets previously ignored by traditional firms.

Structural Observations on Job Displacement

Acknowledges that while market growth does not alleviate job loss concerns, it indicates an increase in demand as intelligence becomes cheaper.

Shifting Constraints in Software Production

Argues that as resources become cheaper, the focus shifts from "can we build it?" to "should we build it?", emphasizing decision-making over mere implementation.

Amplifying Human Talent with AI Tools

Suggests that AI tools enhance rather than replace skilled individuals who can navigate ambiguity and understand customer needs deeply.

The Reality of Dark Factories

Introduces the concept of dark factories where teams produce code without human intervention; these teams are advancing rapidly compared to others still reliant on traditional methods.

Cultural and Organizational Gaps

Identifies a significant gap between advanced teams and those lagging behind—not due to technology but because of cultural resistance and organizational structure issues.

Importance of Systematic Understanding

Stresses that successful organizations will be those investing time into understanding their systems deeply rather than just acquiring better coding tools.

Evolving Definition of Better Engineers

Defines 'better engineers' as those capable of articulating what should exist clearly enough for machines to create effectively while ensuring user needs are met.

Conclusion: Navigating Transition Challenges

Concludes with a call for awareness regarding the transition from traditional software development practices to automated processes, highlighting available resources for further learning.