C++ Zero Cost Conf 2025 / Belgrade

Welcome to the Zero Cost C++ Conference

Introduction and Overview

• The conference is introduced by Anton Kalen, who works at YDB in Yandex, emphasizing the presence of numerous C++ developers.

• The program consists of six talks in both Russian and English, covering diverse topics from robotics to memory optimizations based on real production cases.

• Attendees can use headsets for simultaneous translation; online viewers can select their preferred language settings for translation.

• Coffee breaks and lunches are highlighted as essential parts of the conference experience, with directions provided for after-party locations.

• Attendees are reminded to keep mobile phones on silent mode and encouraged to participate in Q&A sessions following each talk.

Speaker Introductions

Andre's Talk

• Andre introduces himself as a speaker who will present in Russian, apologizing to non-Russian speakers for the language barrier.

Technical Discussions Begin

Memory Management Insights

• Discussion shifts towards advanced memory management techniques including performance metrics related to various indexing methods like HNSW (Hierarchical Navigable Small World).

Upcoming Talks

Alexi's Presentation on TCM Allocator

• Alexi discusses his background in previous conferences focusing on memory management topics such as sanitizers and profiling techniques.

• He emphasizes that today's talk will focus specifically on memory performance using TCM allocators.

Memory Managers Overview

• Alexi explains that different benchmarks compare memory managers primarily based on allocation speed and fragmentation metrics.

Hot/Cold Allocation Strategy

Enhancements in TCM Allocator

• The internal Google TCM allocator has been optimized for multi-threading applications with features like hot/cold hinting for allocations introduced in 2021.

• Programmers can provide hints about how frequently allocated memory will be accessed, which helps improve data locality.

Profile Guided Optimization (PGO)

Integration with LLVM Tools

• Discusses how PGO aims to enhance heap access speed through runtime profiling information used by compilers during optimization processes.

Expectations from New Features

Performance Predictions

• No degradation is expected when using default operations; however, cold-marked objects may see performance drops due to less frequent access patterns.

Implementation Details

Separation of Hot/Cold Objects

• Explains how separating hot from cold objects could lead to better cache utilization and improved swap efficiency within operating systems.

Benchmark Results

Performance Analysis

• Initial benchmark results indicate significant speed improvements when correctly marking hot versus cold objects under specific conditions.

Understanding DTLB Store Misses and Performance Implications

DTLB Store Misses Overview

• The chart illustrates the percentage of DTLB (Data Translation Lookaside Buffer) store misses, which occur when a write operation to main memory fails due to a missing entry in the DTLB.

• A lower percentage of hot objects significantly reduces DTLB misses; for instance, with 1% hot objects, there's a 40% improvement, and at 0.1%, it can eliminate misses entirely.

Sequential Access Performance

• When hot objects are rare (less than 50%), sequential access performance degrades as it resembles random access due to gaps between hot objects.

• Separating hot from cold objects can yield up to an 80% speedup for accessing hot objects, effectively eliminating DTLB misses under optimal conditions.

Impact of Huge Pages on Performance

• Using huge pages introduces performance degradation with inverted settings but still shows improvements with correct configurations—up to 20% for low percentages of hot objects.

• Default settings perform better with huge pages compared to small pages because they result in fewer DTLB store misses.

Sequential Access with Huge Pages

• The performance characteristics of huge page sequential access mirror those of small pages; however, using correct or inverted settings yields similar results.

• The rate of DTLB store misses is negligible (less than 0.02%) for huge pages during sequential access, indicating minimal impact on application performance.

Memory Management Strategies and Allocator Options

Custom Allocators and Adjustments

• If Google’s internal TCM lock cannot be used, alternatives include creating custom allocators backed by mapped devices; however, this approach may introduce complexity and potential vulnerabilities.

• Adjusting existing memory managers like Jaloc is feasible since it supports custom arenas for allocations; however, Jaloc's status as "dead" limits its viability.

Memory Pool Strategies

• Two strategies emerge:

• Creating a memory pool for cold objects that are rarely created or destroyed allows slower allocation speeds while ensuring reliability.

• Establishing a memory pool for hot objects where only specially marked items are treated as such can enhance processing efficiency by copying data between pools when necessary.

Benchmarking and Performance Gains

Importance of Benchmarking

• Combining both strategies—cold and hot object pools—is encouraged while emphasizing the need for benchmarking decisions based on empirical data rather than assumptions about performance gains.

Maximizing Performance Efficiency

• Efficient use of DTLB impacts overall application performance significantly; understanding this relationship can lead to gains up to 50%.

Security Concerns in Memory Allocation

Potential Risks in Allocation Techniques

• Questions arise regarding security risks associated with allocation methods potentially leaking sensitive information due to differing access times between hot and cold objects.

Technical Details on Hint Values in Memory Management

Hint Value Mechanism Explained

• The hint value range from zero to 255 serves as an indicator; values above a certain threshold classify entries as "hot," while others are deemed "cold."

Initialization Variables and Benchmark Considerations

Variable Initialization Insights

• Preheating or initializing variables before benchmarks ensures predictable cache behavior during tests.

Application Suitability for Optimization Techniques

Identifying Beneficial Applications

• Profiling tools like PGO help identify which allocations should be classified as hot or cold without manual intervention.

This structured summary captures key insights from the transcript while providing timestamps linked directly to relevant sections.

Understanding Lightweight Threads and Scheduling

Overview of Lightweight Threads

• The discussion begins with the challenges of implementing lightweight threads, emphasizing that nothing is free in computing. A custom scheduler is required to manage these threads effectively.

• The speaker explains the hierarchy of scheduling, where lightweight threads operate on top of OS threads, creating a more complex scheduling environment.

Locking Mechanisms and Deadlocks

• An example illustrates a deadlock scenario involving one OS thread and two lightweight threads competing for a lock without proper yield instructions.

• The absence of yield leads to a situation where the scheduler cannot reschedule the blocked thread, resulting in a deadlock.

Scheduler Efficiency and NUMA Considerations

• The speaker discusses how schedulers can be designed to be NUMA-friendly or not, affecting performance based on thread locality.

• Different pools for each NUMA node can enhance efficiency by attempting to schedule from local pools before resorting to work stealing from other nodes.

Waiting Strategies for Lightweight Threads

Types of Waiting Mechanisms

• Various waiting strategies are introduced: active spin, yield (context switch), and suspend/resume mechanisms.

• Suspend/resume allows threads to avoid being scheduled until explicitly resumed, which can improve performance but may face library support issues.

Implementing Mutexes for Lightweight Threads

• The main focus shifts towards updating mutex implementations suitable for lightweight threads while addressing potential problems with existing methods.

• A common waiting loop strategy is discussed, highlighting different backoff strategies used in various locking mechanisms like test-and-test-and-set (TTS).

Challenges with Mutex Implementation

Limitations of Test-and-Test-and-Set Locks

• TTS locks face limitations as they only allow yielding due to their reliance on registers without additional information about the lock state.

Enhancements through MCS Locks

• MCS locks provide better flexibility by incorporating specific notes per thread that facilitate suspension and resumption during contention scenarios.

NUMA Challenges in Thread Scheduling

Issues with Thread Locality

• When suspending and resuming threads across different NUMA nodes, there’s a risk of losing locality which can degrade performance.

Potential Solutions for NUMA Scheduling

• Suggestions include using specialized schedulers that maintain thread locality or waking up next queued tasks preemptively to ensure efficient handoffs between threads.

Implementation Details and Library Choices

Selected Libraries for Testing

• Three libraries were chosen: Argobots (academic focus), Boost Fibers (popular general use), and Userver (widely used in Russia).

Supporting Suspend/Resume Functionality

• Each library's approach to supporting suspend/resume varies; Argobots has built-in support while others require creative solutions due to lack of native functionality.

Experimental Results and Performance Analysis

Benchmarking Methodology

• Experiments involved running lightweight threads performing simple operations under varying conditions including critical sections and parallel workloads.

Performance Insights Across Libraries

• Results indicate that TTS performs well under low contention scenarios while MCS locks show promise when combined with Q-lock structures.

Key Takeaways from the Discussion

Importance of Choosing Appropriate Libraries

• Selecting the right library impacts both ease-of-use and performance significantly; experimentation is crucial before implementation decisions are made.

This structured summary captures key insights from the transcript while providing timestamps for easy reference.

Introduction to Key Concepts

Initial Thoughts and Feelings

• The speaker expresses a sense of uncertainty and desire for clarity, indicating a struggle with the topic at hand.

• There is an emphasis on "heat," possibly referring to intensity or urgency in the discussion.

Technical Discussions

• Introduction of technical terms such as "latency" and "index," suggesting a focus on performance metrics in computing.

• Mention of C++ and Google Chrome, hinting at discussions around programming languages and web technologies.

Data Structures and Algorithms

Ordered Sets

• Discussion about ordered sets, emphasizing their importance in managing data efficiently.

• The concept of stable sorting is introduced, which is crucial for maintaining order during data processing.

Similarity Measures

• Locality-sensitive hashing is mentioned, indicating methods for efficient similarity searches within datasets.

Trading Systems Insights

Market Dynamics

• Reference to trading practices like "buy and hold," highlighting strategies used by traders in financial markets.

• Introduction of concepts like market makers, which are essential for understanding liquidity in trading environments.

Computational Framework Discussions

Open Source Initiatives

• Mention of open frameworks suggests a focus on collaborative software development practices.

Device Concurrency

• Discussion about concurrency in devices indicates challenges related to resource management during operations.

Future Technologies: CXL and Beyond

Innovations in Memory Management

• Inquiry into how CXL (Compute Express Link) could enhance memory management solutions within computing systems.