Linux on the New Framework Desktop PC!

Is This Finally the Year of the Linux Desktop?

Introduction to the Machine

• The speaker reflects on the ongoing joke about whether this is finally the year for the Linux desktop, expressing optimism after using a new machine.

• The machine features an AMD Ryzen AI Max Plus 395 CPU with 16 cores and a powerful GPU, boasting 128 GB of unified memory, which has garnered attention from AI entrepreneurs.

Performance Overview

• Despite its small form factor, this desktop performs exceptionally well. It utilizes a standard ITX motherboard and includes a robust 400-watt power supply.

• The system supports various connectivity options including 5 gig Ethernet, dual USB 4 ports, HDMI display port, and PCIe Gen 4 M.2 slots with built-in heat sinks.

Customization and Design

• The front of the case features customizable tiles that can be 3D printed; users can design their own tiles for personalization.

CPU Specifications

• The CPU is a Zen 5 variant similar to the desktop's 9950X but operates within a quad-channel memory configuration providing around 200 GB/s bandwidth.

• Operating within a power envelope of 140 watts allows performance close to that of higher-end CPUs like the Ryzen 9 series.

GPU Performance Insights

• The integrated Radeon 860S GPU offers performance comparable to Nvidia's RTX 4060, capable of handling games at respectable frame rates (1440p at 60 fps).

• Unified memory architecture allows dynamic allocation of VRAM from system memory rather than pre-allocation, enhancing efficiency in gaming scenarios.

Software Compatibility

• Framework supports multiple operating systems including Windows and various Linux distributions such as Ubuntu and Fedora.

• For optimal performance on bleeding-edge hardware, Arch or Arch-based distros like Cachi OS are recommended due to their ease of setup while retaining access to Arch’s ecosystem.

Configuration Options

• Available configurations include models with varying RAM:

• 128 GB: Ideal for AI workloads.

• 64 GB: Suitable for general use and gaming.

• 32 GB: Not recommended due to limited upgradeability.

AI Workloads Capabilities

• Three main methods exist for running AI workloads on this platform:

• Utilizing all available CPU cores (16 Zen 5 cores).

AI Development and Performance Insights

Overview of AI Paths and Technologies

• The discussion begins with the introduction of the 8060S APU, highlighting its capabilities for compute tasks and large language models, particularly emphasizing the Vulcan backend as a fast option.

• Testing results show that OpenAI's new 120 billion parameter model runs at approximately 33 tokens per second on this platform using Vulcan, demonstrating impressive day zero support.

• Clarification is made that Vulcan is distinct from Rockom; while both are related to AMD technologies, Vulcan serves as a graphics compute API whereas Rockom is tailored for machine learning on CDNA GPUs.

Rockom Framework and Performance

• The speaker notes significant advancements in Rockom over the past six months, suggesting it as an affordable alternative to cloud instances for developers needing CDNA access.

• A recommendation is made to explore community resources such as forums and GitHub repositories for tracking model performance and setup steps related to Rockom.

Hybrid Setup Options

• An innovative hybrid setup involving an Nvidia SFF4000 ADA GPU connected via USB4 or Oculink allows workload distribution between Vulcan/Rockom and Nvidia for enhanced processing efficiency.

• This hybrid configuration can handle large models (up to 500 billion parameters), combining power from both systems with a total of 250 watts consumption.

Personal AI Lab Configuration

• The speaker describes their ideal personal AI lab setup, likening it to a blend of sci-fi elements with practical applications while maintaining low heat and power requirements.

• Emphasis is placed on the flexibility of the Framework ITX board design, which supports external GPU configurations and customizable features appealing to tech enthusiasts.

Market Implications and Future Directions

• Discussion shifts towards AMD's strategic positioning in the market; leveraging APUs could disrupt traditional desktop CPU markets by reducing component counts while maintaining performance levels.

• Concerns are raised about Rockom’s current limitations being designed primarily for CDNA rather than RDNA but acknowledges rapid progress in its development over recent months.

• The need for continued momentum in developing Rockom is stressed; accessible compute options like NPU integration are highlighted as beneficial when other resources are occupied.

Conclusion: Building Versatile Systems

• The BIOS quality within this framework is praised as superior compared to competitors, enhancing user experience significantly.