Fundamentals of GPU Architecture: SIMT Core Part 4

Introduction to the Three Loop Approximation in GPU Architecture

Overview of Previous Loops

• Nick introduces the topic of three loop approximation, building on previous discussions about scheduling loops.

• The first loop focused on instruction selection from a warp, while the second loop dealt with selecting instructions from an instruction buffer.

Transition to Register File Scheduling

• The discussion shifts towards scheduling accesses to the register file, moving past dependency tracking.

• Emphasizes the need for numerous warps to hide memory latencies, leading to large register files (around 256K in certain architectures).

Naive Banked Register File Microarchitecture

Challenges with Naive Implementation

• A naive approach would require one port per operand per instruction issued per cycle, resulting in excessive ports.

• Instead of this, multiple banks of single-port memories are used; however, this leads to bank conflicts.

Example of Bank Conflicts

• An example is provided where two instructions access registers mapped to the same bank, causing execution stalls due to conflicts.

• The execution order illustrates how simultaneous reads can lead to stalling when accessing shared banks.

Utilization Issues and Bank Conflicts

Analysis of Bank Usage

• Observations show that some banks remain unused while others are over-utilized, indicating poor distribution across banks.

• If all reads go to a single bank, it negates the benefits of having multiple banks.

Motivation for Collector Units

• Introduces collector units as a solution by allowing multiple instructions to overlap during operand reading.

Enhancing Throughput with Operand Collectors

Functionality of Operand Collectors

• Each instruction is assigned a collector unit upon entering register read phase which helps reduce bank conflicts and improve throughput.

Scheduling Reads Effectively

• This third scheduling loop aims at avoiding bank conflicts during register file access by optimizing read schedules.

Swizzle Banked Register Layout

Addressing Inter-Warp Conflicts

• Proposes a swizzle layout where registers are offset for different warps ensuring better parallelism and reduced conflict chances.

Example Execution Flow

• Illustrates how three warps executing similar instructions can still achieve effective utilization through varied bank mapping.

Handling Write After Read Hazards

Potential Issues with Operand Collectors

• Discusses challenges like write after read hazards when later instructions become ready before earlier ones complete execution.

Proposed Solutions

• Suggestion includes "release on commit" work board limiting active executions per warp but negatively impacts performance significantly.

• Alternative "release on read" work board allows only one instruction collecting operands at once reducing slowdowns.

• Finally proposes using bloom filters for tracking outstanding reads minimizing performance impact effectively.

Conclusion and Future Directions

Summary and Next Steps

• Nick concludes by hinting at future topics including structural hazard handling and research directions related to GPU architecture improvements.