Mengenal “AI PC” & NPU : Marketing, Realita, dan Masa Depan (ft. Core Ultra 7 155H,Zenbook 14 OLED)

What is AIPC and NPU?

Introduction to AI PC and NPU

• The discussion focuses on the concepts of AI PC (AIPC) and Neural Processing Unit (NPU), clarifying that this is not a CPU or laptop review.

• The speaker emphasizes that their knowledge of AI is at a basic level, akin to an ordinary user, indicating a shared learning experience with the audience.

Understanding Artificial Intelligence

• The definition of AI is explored, describing it as the capability of computer systems to perform tasks typically requiring human intelligence.

• AI encompasses various forms including hardware, software, and algorithms; its scope in computer science is broad and diverse.

Subsets of AI: Machine Learning and Deep Learning

• Two main subsets within AI are identified: machine learning (ML) and deep learning (DL).

• ML involves computers learning from data labels to recognize patterns (e.g., identifying cats in images).

• DL uses neural networks to analyze data through multiple layers for deeper pattern recognition.

Applications of AI

• The necessity for AI arises from its ability to perform certain tasks more efficiently than traditional methods.

• An example illustrates image upscaling using traditional resizing versus an AI tool called Waifu2x, highlighting improved clarity with the latter.

Modern Uses of AI

• Advanced functions beyond simple tasks are discussed; for instance, images can be generated by tools like Stable Diffusion or OpenAI's GPT chat.

Defining an AI PC

Characteristics of an AI PC

• An "AI PC" refers to a computer capable of handling modern computational workloads associated with artificial intelligence effectively.

Types of Workloads in AI

• There are two types of workloads:

• Online Workloads: Require internet access for processing via cloud servers (e.g., GPT chat).

• Local Workloads: Execute entirely on-device without internet reliance (e.g., generating images using Stable Diffusion).

Training vs. Inferencing in AI Models

• Key terms such as training and inferencing are explained:

• Inferencing: Executing existing models akin to following a recipe.

Understanding AI Training vs. Inference

The Complexity of AI Training

• The process of training an AI model is significantly more complex than running inference, which most users typically engage in.

• While companies usually handle the initial training of models, advanced users may fine-tune them for specific applications.

Inference and Hardware Requirements

• Inference can occur on local hardware or cloud servers; generating images with different setups yields varying performance times.

• Heavy workloads like image generation require substantial processing power; for instance, generating an image using Stable Diffusion on a laptop CPU takes about 2 minutes.

Performance Comparison: CPU vs. GPU

• Using an integrated GPU can drastically reduce processing time for tasks like image generation to around 25 seconds compared to a CPU.

• Modern GPUs are equipped with dedicated AI accelerators, enhancing their ability to perform AI-related tasks efficiently.

Introduction to Neural Processing Units (NPUs)

• NPUs are specialized hardware designed for executing AI operations independently from CPUs and GPUs, focusing on low-power consumption rather than high throughput.

• For example, while a GPU might consume over 30 watts during operation, an NPU can operate at around 5 watts when performing similar tasks.

Applications and Limitations of NPUs

• NPUs are particularly beneficial for battery-powered devices as they conserve energy while running lightweight AI applications.

• However, desktop PCs may not benefit as much from NPUs due to other components consuming more power than the savings provided by the NPU.

Software Utilization and Performance Metrics

• Effective use of NPUs requires compatible software; without it, the potential benefits cannot be realized.

Performance of AI on Different Hardware

Software Optimization Impact

• The transition from direct ML fire to Open Vino optimization reduces image processing time from 25 seconds to 13 seconds, demonstrating the significant impact of software optimization on performance.

• Emphasizes that software optimization can be as crucial as hardware specifications (e.g., NPU), suggesting users should focus on the software capabilities rather than just hardware specs.

GPU vs. NPU in AI Applications

• Intel's integrated GPU Arc is currently more effective for a wider range of applications compared to its NPU, highlighting the importance of application compatibility with hardware.

• An example shows that using the Arc GPU can halve music separation processing time from 2 minutes to 1 minute, indicating its efficiency over CPU.

Limitations and Challenges of NPU

• The current lack of widespread software support for NPUs limits their effectiveness; many applications still rely heavily on GPUs.

• Discusses challenges in assessing NPU performance due to limited benchmarking tools and databases available for comparison.

Understanding NPU Functionality and Use Cases

Current Capabilities and Performance

• NPUs have potential for running local large language models (LLMs), but most software is still in early development stages.

• There are few benchmarks available for NPUs, making it difficult to evaluate their performance against other technologies.

Practical Applications and Efficiency

• Continuous background tasks like audio noise cancellation benefit significantly from NPU acceleration, showcasing its utility in specific scenarios.

• While NPUs are not yet useful in gaming contexts, they excel at tasks like AI upscaling which typically utilize GPUs.

AI PCs: Are They Necessary?

Evaluating AI Utility

• A general assessment suggests that while some AI applications prove beneficial, others may be perceived as gimmicks depending on user needs and experiences.

• Large language models like ChatGPT enhance search capabilities beyond traditional engines, illustrating practical benefits of AI technology.

Future-Proofing with AI PCs

• The necessity of an AI PC depends on individual use cases; if daily tasks require local AI processing, investing in an accelerator makes sense.

Understanding the Role of NPU in AI PCs

User Perspectives on AI Capabilities

• Users have varying opinions on AI capabilities like image generation and video upscaling; some find them useful for content creation, while others see them as unnecessary, particularly gamers who use PCs solely for entertainment.

• It's essential to choose software based on personal needs rather than others' preferences, emphasizing that utility is subjective.

The Importance of Hardware Components

• The discussion highlights the roles of CPU, GPU, and NPU in AI applications; CPUs can handle light applications without accelerators, while NPUs are beneficial for power-saving in continuous light applications.

• Currently, the use of NPUs in Core Ultra processors is limited; there are more applications utilizing GPUs compared to NPUs. Thus, NPUs aren't seen as mandatory yet.

Adoption and Future Potential of NPU

• Increased adoption of NPUs could encourage developers to create more compatible applications. However, this potential has not yet materialized into widespread functionality.

• The importance of software support is emphasized; having powerful hardware is futile without corresponding software or APIs that utilize these capabilities effectively.

Standards and Recommendations for AI PCs

• Intel's Openfin presents a significant advantage by encouraging developer adoption of their technologies (Arc GPU and NPU), which may enhance usability.

• A call for 32GB RAM as a minimum standard for AI PCs by 2024 is made due to resource-intensive nature of many AI applications. Current 16GB setups may be inadequate.

Conclusion and Future Considerations

• The speaker expresses hope that future discussions around NPUs will provide valuable insights without succumbing to hype or unrealistic expectations regarding their capabilities.