I Cut My OpenClaw Costs by 97%

OpenClaw Token Optimization Guide

Introduction to OpenClaw

• Matt Ganzac introduces the OpenClaw token optimization guide, emphasizing that it is an AI personal assistant deployable locally.

• He advises against attempting this if the user lacks development experience or has not previously deployed apps locally.

Deployment Recommendations

• Users should deploy OpenClaw in a controlled environment, preferably on a dedicated PC or Mac, rather than their primary machine.

• A cautionary tale is shared about a user who inadvertently allowed OpenClaw to make expensive purchases while trying to rebuild his brand.

Customization and Risks

• Matt stresses that following his steps may risk breaking the application for those unfamiliar with coding; he provides these insights based on his own experiences.

• The guide aims to provide more comprehensive information beyond short social media videos where he has been sharing tips.

Initial Experiences with OpenAI

• Initially, Matt attempted to use OpenAI for version 1 of his app but found it ineffective and frustrating, leading him to create version 2 using Sonnet instead.

• He notes that deploying and configuring the app with Sonnet cost around $3 and provided a solid foundation for further development.

Cost Reduction Strategies

• Matt discusses how he managed to reduce costs by 97% through specific optimizations related to token usage during daily operations.

• He mentions providing a downloadable link for viewers to access the detailed guide accompanying this video.

Daily Usage Insights

• Before optimizations, running basic tasks cost $2-$3 daily, which could lead up to $90 monthly even when idle.

• After implementing changes, he reports achieving zero costs while idle and explains how the AI assists in finding business opportunities without being fully autonomous yet.

Core Logic Issues Identified

• A significant issue identified was that OpenClaw loads all context files with every message sent, leading to excessive token consumption (2–3 million tokens wasted).

• By adjusting how context files are loaded—reducing unnecessary data retrieval—Matt saved approximately 80% of token expenses associated with context overload.

AI Model Management and Optimization

Context Size and Memory Management

• The context size on startup was initially around 50 kilobytes, which increased with each prompt and heartbeat, leading to unnecessary memory bloat.

• Users may experience escalating context file sizes (from 50 to over 100 kilobytes), resulting in inflated data usage without any productive output.

Running Multiple AI Models

• Contrary to popular belief, it is possible to run multiple AI models simultaneously; the speaker mentions running four models effectively.

• Configuration files allow users to set up different primary models for various tasks, enabling segmentation of workloads across models.

Task Segmentation and Cost Efficiency

• Tasks can be categorized based on their complexity; simpler tasks can utilize less expensive models like Haiku instead of more costly ones like Opus or Sonnet.

• By routing tasks appropriately among different models based on their criticality, users can significantly reduce token consumption while maintaining output quality.

Heartbeat Functionality

• The "heartbeat" function pings the system to check for active tasks. If not running, the system may enter sleep mode and fail to complete ongoing tasks.

• Each heartbeat sends context files and session history data; however, excessive session history uploads can lead to inefficiencies.

Session History Management

• A token audit revealed that session history could accumulate significant data (e.g., 111 kilobytes), which is uploaded every time a prompt is made.

• Implementing a command to clear session history before new prompts helps avoid loading unnecessary past interactions into the current context.

Local LLM Installation

• Users are encouraged to install the latest version of local LLM software (like Olama), which allows for better management of AI model operations.

Understanding Heartbeats and Token Management

The Importance of Efficient Heartbeat Configuration

• The speaker emphasizes the need to avoid unnecessary token expenses for heartbeats when using Opus, suggesting that misconfiguration could lead to costs as high as $5 a day for idle services.

• A local heartbeat configuration is recommended, which checks system memory and tasks without making API calls, thus saving tokens.

• The speaker advocates for integrating this local heartbeat check into OpenClaw as a core feature to prevent unnecessary API token usage.

Addressing Rate Limits and Session History Issues

• Initial rate limits from Anthropic provide 30,000 tokens per minute; however, the speaker encountered issues with excessive token consumption due to session history being uploaded repeatedly.

• A command was created to manage session history effectively by dumping previous sessions while retaining them in memory, preventing redundant uploads during API calls.

Optimizing Token Usage

• Built-in pacing mechanisms were established to avoid hitting rate limits frequently; the speaker noted receiving error code 429 due to excessive submissions.

• Testing on different platforms revealed that messaging apps like Slack compile entire message histories each time a prompt is sent, leading to increased token usage.

Strategies for Reducing Token Consumption

• The speaker compressed workspace files and defined key metrics focused on low token usage. This allows prompts to specify desired outcomes while monitoring expected token expenditure.

• By optimizing requests based on anticipated costs (e.g., finding leads), users can better manage their budget and evaluate actual versus projected token use.

Monitoring and Calibrating Token Usage

• Users are advised not to allow automatic billing until they have fine-tuned their setups; unexpected charges can occur if not monitored closely.

• Keeping track of dashboard metrics helps calibrate expectations regarding token usage accuracy over time through comparative analysis of predicted versus actual costs.

Cost Efficiency of Automated Research Tasks

The Importance of Caching

• Caching significantly reduces costs; the cache API is much cheaper than other methods. A task that utilized cached tokens resulted in a minimal cost.

Comparison with Traditional Methods

• The speaker highlights the stark contrast between automated research costs and traditional human labor, noting that a task costing $6 would typically cost tens of thousands when done by people.

Performance and Time Savings

• The efficiency of automated systems allows for tasks to be completed overnight, which would normally take a month’s worth of work from a company, emphasizing the value of automation.

Workflow Automation with Sub Agents

• The setup involves multiple sub-agents performing different tasks: crawling data, writing emails, and organizing files using APIs like Brave search and hunter.io.

Task Execution Process

• Three agents were employed: one for research (Haiku), another for email drafting (Sonnet), and an LLM (Olama) for file organization. This division of labor enhances productivity.

Cost Analysis of Automation

• Running these tasks overnight cost only $6, equating to about $1 per hour. This level of efficiency is unmatched by traditional workers who cannot operate at such low costs effectively.

Token Optimization Awareness

• Emphasizes the need for token optimization to avoid excessive costs; inefficient use can lead to high expenses without significant output.

Encouragement for Engagement

• Invites viewers to ask questions in comments and provides links to documents for optimizing their own setups with multiple sub-agents.