Supermicro CEO Keynote at COMPUTEX 2024

Super Micro: Accelerating AI and Sustainability

The Role of Super Micro in AI Advancement

• Super Micro builds the compute infrastructure that enhances GPU capabilities, accelerating machine learning, predictive analysis, generative AI, natural language processing, and deep learning.

• Their technology supports real-time data processing through trained AI models for faster predictions and task solutions.

Commitment to Sustainability

• Emphasizes the importance of protecting nature as humanity's survival depends on a flourishing planet; Super Micro aims to build a better world through high-performance computing.

• The company integrates sustainability into its product design and performance, utilizing platforms like Nvidia for optimized server solutions.

Infrastructure Solutions Offered

• Super Micro provides a wide range of performance-optimized AI infrastructure solutions essential for businesses relying on real-time data delivery.

• They offer an extensive selection of rackmount servers and generative AI superclusters tailored for modern workloads including ML and LLM applications.

Vision for Green Computing

• The future is framed around green computing initiatives that aim to save the planet while delivering enterprise-ready server solutions.

• For 30 years, Super Micro has committed to sustainability efforts in technology development.

Personal Mission and Community Engagement

• Charles Liang introduces his personal mission focused on environmental sustainability through tree propagation projects targeting millions of trees.

Data Center Efficiency and Direct Liquid Cooling Solutions

Introduction to AMD Gen 5 Touring Base Solution

• The latest high-performance chip from AMD, the Gen 5 touring base solution, is ready for early deployment, offering improved performance at a similar cost but with increased thermal design power (TDP).

• There is a rising demand for compute and storage capabilities, particularly in generative AI training. The mission is to enhance data center efficiency while reducing total operational costs.

Challenges of Traditional Air Cooling

• Traditional air cooling methods are becoming less viable due to their high energy consumption and associated costs.

• Direct liquid cooling (DLC), which has been around for over 30 years, can utilize room temperature water for optimal server cooling at significantly lower costs and reduced environmental impact.

Benefits of Direct Liquid Cooling

• Implementing DLC in an average-sized data center could save up to 7% on total costs by minimizing space requirements and allowing smaller UPS systems.

• Operational savings can reach up to 40%, potentially totaling $60 million over five years depending on location, making green computing more accessible through DLC.

Environmental Impact and Power Requirements

• DLC not only reduces CO2 emissions but also lowers power requirements; a typical air-cooled data center needs about 15 megawatts of power compared to just 10 megawatts with DLC.

• This reduction opens opportunities for new classes of data centers in regions with limited power capacity.

Overcoming Historical Barriers to Adoption

• Historically, there were concerns regarding the long lead times (4–12 months), higher costs, reliability issues, maintenance difficulties, and unproven market viability of DLC solutions.

• Supermicro aims to revolutionize the industry by addressing these concerns through improved delivery timelines (2–4 weeks), lower costs than traditional air-cooled solutions, enhanced reliability, and targeting significant adoption rates within the next few years.

Future Outlook for Direct Liquid Cooling

• The goal is for DLC solutions to capture 15% of global new data center developments within one year and potentially reach up to 30% thereafter.

• This shift represents a dramatic increase from less than 1% market share over the past three decades.

Conclusion: A New Era in Data Center Technology

• While DLC requires more complex integration into existing infrastructures compared to traditional air cooling systems, it enables the development of powerful AI systems in smaller spaces.

AI and Green Computing: A New Era

Introduction to AI and Green Computing

• The discussion opens with the potential of DLC liquid cooling being offered for free, emphasizing a partnership in "Green Computing" driven by advancements in AI.

• The speaker expresses excitement about the transformative power of AI, hinting at future developments and concerns regarding its control over humanity.

Keynote by Jensen Huang

• Jensen Huang, CEO of NVIDIA, is introduced as a key figure sharing his vision on AI's impact on computing.

• Huang humorously admits to feeling groggy before his keynote but emphasizes the importance of the current technological moment.

Accelerated Computing and Energy Efficiency

• Huang discusses two significant trends: accelerated computing and green computing, highlighting their relevance in today's data processing landscape.

• He clarifies that "green computing" refers to energy-efficient computing practices aimed at reducing waste in data centers.

Generative AI's Impact

• The emergence of generative AI is highlighted as a revolutionary development that will influence every data center globally.

• Huang explains generative AI encompasses various forms such as text generation, image generation, and video generation, marking a shift in how data centers operate.

Modernization of Data Centers

• There is an urgent need to modernize existing data centers due to the anticipated growth in demand for advanced systems by 2030.

• Supermicro’s role in shipping new technologies like liquid cooling solutions is emphasized as essential for lowering power consumption while increasing production capabilities.

Technological Marvel of GPU Servers

• The complexity and innovation behind GPU servers are discussed; they consist of numerous components working together seamlessly.

• Huang notes that while GPUs are central to these systems, the overall architecture involves intricate engineering beyond just chips.

Energy Efficiency and Revenue Generation in Data Centers

The Shift to Liquid Cooling Systems

• The goal is to increase the DLC M share from 1% to 15% this year, significantly improving energy efficiency for data centers.

• Liquid-cooled systems drastically reduce costs in data centers by capturing waste energy, which can be repurposed for computing tasks.

New Commodities and Revenue Models

• A new commodity has been introduced: token generation, which equates to revenue similar to how energy is measured in dollars per kilowatt hour.

• This model emphasizes that AI factories generate direct revenues rather than merely serving as file servers or email exchanges.

Financial Implications of AI Factories

• Investing $3 million can yield substantial annual returns through large language model (LLM) token generation due to embedded intelligence.

• Key factors such as reliability, throughput, and startup time are crucial for maximizing revenue potential in these AI-driven environments.

System Integration and Performance Metrics

• Building proprietary data centers and supercomputers ensures high startup speed, utilization rates, and throughput—all vital for revenue generation.

• Factory output relies on complex metrics certified by NVIDIA software licenses, ensuring compatibility across various systems.

Software Compatibility and Networking Innovations

• All components are designed for full software compatibility; this is critical since software complexity is a major challenge in high-performance computing.

• CUDA serves as the foundational software stack while advanced networking technologies enable efficient workload distribution across computing fabrics.

High-Speed Interconnectivity

• The system utilizes high-speed interconnect technology that connects CPUs with GPUs efficiently while maintaining low power consumption during operations.

AI Performance and Energy Efficiency

Importance of Energy Efficiency in AI Systems

• High-performance AI systems require low energy consumption, particularly during training and inference phases. Efficient use of CPU resources is emphasized.

• Energy efficiency directly correlates with higher performance in computing, highlighting the significance of "Green Computing" initiatives.

Current State and Future Goals for AI

• AI technology has reached a plateau where it is deemed "good enough" for many applications, but there is still a need for improvement to enhance functionality.

• The focus remains on advancing AI technology while ensuring safety measures are in place, similar to autopilot systems in aviation.

Safety Measures and Policies for AI

• Just as aviation employs various technologies to ensure safety (e.g., monitoring systems), future AI implementations will require similar oversight mechanisms.

• Establishing guardrails and policies for safe AI operation is crucial; discussions around good practices in science, engineering, and business are necessary.

Conclusion on Safety and Purchasing

• A key takeaway is that increased investment leads to enhanced safety measures within the realm of AI development.

Advancements in Liquid Cooling Technology

Introduction to Liquid Cooling Solutions

• Transitioning from traditional air cooling to direct liquid cooling can significantly benefit data centers by improving efficiency.

Benefits of Direct Liquid Cooling

• Super Micro's direct-to-chip liquid cooling solutions can support large-scale deployments (e.g., 8,000 H100 GPUs), offering substantial power savings.

• Customers switching to liquid cooling experience up to 33% power savings compared to air-cooled configurations.

Rapid Deployment Capabilities

• Super Micro has achieved a two-week rapid rack delivery promise, allowing data centers to scale quickly without long wait times typically associated with new builds.

Dominance in Liquid Cooling Market

• The company reports shipping over 50 liquid-cooled racks per day, indicating strong market presence and commitment to this technology.

Innovations in GPU Deployments

Overview of GPU Shipping Milestones

• Super Micro has shipped over half a million Nvidia GPUs, surpassing the computational capacity of the top 20 supercomputers combined.

Versatile GPU Rack Configurations

• New versatile rack designs support multiple GPU types (Nvidia H200S, AMD MI300X), catering to diverse customer needs while maintaining deployment flexibility.

Enhanced Deployment Strategies

AI Solutions and Data Center Management

Overview of Supermicro's AI Solutions

• Supermicro presents a comprehensive monitoring system for data centers, integrating GPU chip sensors to liquid cooling towers into a single interface.

• The introduction of the legal coin system aims to enhance thermal management while reducing environmental impact by redirecting energy from cooling to computing.

Energy Efficiency and Performance

• The DLC solutions can dissipate heat efficiently at temperatures up to 45°C, optimizing thermal management and enhancing hardware performance.

• Supermicro's DLC solutions reportedly double computing density compared to traditional air cooling methods, supporting high TDP GPUs while promoting sustainability through heat reuse.

Customer Support and Deployment

• Supermicro offers total solutions including monitoring tools and on-site deployment services, facilitating quicker transitions to DLC data centers for customers.

• Early adopters of green computing with DLC can achieve significant savings, emphasizing the financial benefits alongside environmental considerations.

Expansion and Capacity Growth

• Recent expansions in Taiwan have increased server production capacity significantly, allowing for global service support in data center deployments.

• The new facility is capable of designing and delivering up to 5,000 racks per month, showcasing an increase in operational capabilities.

Advancements in Edge Computing

Focus on Edge Computing Solutions

• Supermicro has been dedicated to edge computing for over two decades, leveraging design expertise across various systems including compact form factors.

• New hyper compact systems are designed specifically for AI inferencing applications in retail environments, integrating multiple GPUs for enhanced performance.

Outdoor Edge Systems Capabilities

• Outdoor edge systems are equipped with IP65 ratings for protection against water and dust while operating under extreme temperature conditions (-47°C to 47°C).

Future Goals and Environmental Initiatives

Market Share Aspirations

• Aiming to grow the market share of DLC from less than 1% currently to potentially 15% within one year or even 30% within two years.

Financial Impact of Green Computing

• Transitioning data centers can save up to $60 million while significantly reducing CO2 emissions—equivalent to preserving billions of trees globally.

Commitment to Sustainability

• Emphasizing that green computing initiatives not only provide financial benefits but also contribute positively towards environmental conservation efforts.

Environmental Protection Efforts

Founding the Green Earth Foundation

• Inspired by environmental concerns, Charles founded the Green Earth Foundation focused on planting billions of drought-resistant trees worldwide as part of a broader initiative against climate change.

Implementation Plans