Your coding agent should be doing end to end human simulating app testing

Human-Simulating End-to-End App Testing

Importance of Human-Simulating Testing

• Coding agents should perform human-simulating end-to-end app testing rather than just running unit or integration tests on their own code.

• Consider the various surfaces (web, iPhone, desktop apps) that users interact with when testing applications.

Tools for Different Surfaces

• Utilize specific tools for each type of application surface:

• Playwright CLI for browser testing.

• Native computer use tools for desktop app testing.

• Xcode MCP for iPhone app testing.

Mapping User Flows

• Identify the branching tree of potential user flows within your app. Each user decision creates a new branch in this flow.

• Combine the understanding of surfaces and user flows to enable comprehensive human-simulating end-to-end testing.

Advantages Over Traditional Testing

• Unlike traditional unit or integration tests, which may overlook edge cases, human-simulating tests provide a full map of potential user interactions.

• This approach allows coding agents to navigate through all branches, simulating real human usage effectively.

The Role of Unit and Integration Tests

• While unit and integration tests are valuable during development to prevent regressions, they do not replace the need for thorough human-like driving scenarios in testing.

• Just as a car needs to be driven under various conditions by a human to uncover bugs, so too must an app undergo rigorous human-simulated testing.