Codex Just Became the Everything App

OpenAI's Codex: The Everything App?

Major Upgrades to Codex

• OpenAI has released a significant upgrade to Codex, transforming it into an "everything app" that extends beyond coding tasks.

• The new version allows Codex to interact with all applications on the user's computer by clicking and typing with its own cursor, utilizing multiple agents in parallel without disrupting other work.

New Features and Capabilities

• A built-in browser feature enables Codex to render web pages directly, allowing users to provide feedback on those pages for collaborative development. This is particularly beneficial for front-end and game development.

• Image generation capabilities have been integrated into Codex, similar to Google's offerings, enhancing design ideation processes for front-end projects.

Plugin Integrations

• OpenAI is introducing 90 additional plugins that combine skills app integration and MCP servers, which will enhance context gathering and action-taking across various tools essential for knowledge work.

• Available plugins include popular applications like Jira, CircleCI, GitLab issues, and Microsoft Suite, catering not only to developers but also non-developers. This marks a strategic refocus by OpenAI towards creating a comprehensive super app rather than diversifying too broadly.

User Experience Feedback

• Initial user experiences indicate some bugs in the new MacOS version of the app; however, improvements in UI design are noted compared to previous versions. Users express optimism about the rivalry between OpenAI and Anthropic leading to better features overall.

• After successfully upgrading the app, users can trigger the in-app browser functionality which allows direct interaction with web elements for enhanced contextual information during coding tasks. This feature is expected to significantly improve usability for developers working on specific projects or tasks.

Future Developments

• The ability to index specific folders within the application was demonstrated as a new feature; however, further enhancements such as click-and-select options are desired by users for improved functionality. Overall satisfaction with these updates suggests positive momentum moving forward in product development at OpenAI.

Testing Computer Use Capabilities

Initial Setup and Navigation

• The speaker initiates a test of computer use capabilities by opening a new incognito tab in Chrome and navigating to google.com, confirming that the browser opened successfully.

• The speaker realizes that the browser may have already navigated to google.com as the landing page, prompting them to try accessing openai.com instead.

Exploring Application Functionality

• The speaker tests whether the system can open an application on their computer by requesting it to launch the notes app and add a note, which it successfully does.

• They note that this process appears faster compared to previous cloud-based code execution methods, indicating improved performance in local application handling.

Observations on Model Capabilities

• The speaker reflects on the convergence of capabilities among different AI models, suggesting that differentiation is diminishing as companies compete for similar functionalities. Execution quality will be key in distinguishing providers moving forward.