How to Choose an NVIDIA GPU for Deep Learning in 2023: Ada, Ampere, GeForce, NVIDIA RTX Compared

What GPU to Buy in 2023

In this video, the speaker discusses what Nvidia GPU to buy for deep learning and machine learning in 2023. The speaker assumes that the viewer will be buying an Nvidia GPU and that it will be going into a machine that they are building or installing.

Nvidia RTX vs. Nvidia GeForce

• The first thing to consider is whether to go for the Nvidia RTX or the Nvidia GeForce line.

• The Nvidia RTX is more expensive and considered the professional line of GPUs.

• There are only two cases where one might want to consider the professional line over the GeForce: if you need a lot of RAM on your individual GPU or if you're going to get more than one of them.

Professional Line Considerations

• If you need a lot of RAM on your individual GPU, consider getting an Nvidia A6000 which comes with 48GB of RAM.

• If you want compactness and stacking ability, consider getting two A6000s as they take up only four slots compared to six for two 3090s or equivalent.

Ampere vs. Ada Lovelace

• We are at a turning point between two technologies: Ampere (30 series) and Ada Lovelace (40 series).

• Both are worth considering depending on price point.

• If maximum performance is desired regardless of price point, get a 4090 or an A6000.

• To get the best GPU for a specific dollar amount, look at current market conditions and supply/demand factors.

Factors Determining Deep Learning Speed

• The number of CUDA cores determines how fast the GPU can do deep learning.

• Clock cycle is also important but fundamentally it's going to be determined by CUDA cores.

GPU Memory Size for Deep Learning

In this section, the speaker discusses the importance of memory size in GPUs for deep learning.

Memory Size Sweet Spot

• 12 gigabytes is considered the sweet spot for memory size in GPUs for deep learning.

• 8 gigabytes may be good to start with but can quickly become limiting.

• The RTX 3060 with 12 gigabytes is an interesting card to consider.

Choosing a GPU Based on Memory

• For the 40 series, anything above the 12 gigabyte sweet spot may be worth considering.

• Nvidia reports that both the 4090 and the 4080 beat anything in the 30 series speed-wise.

• Buying a GPU with more RAM will future-proof it better, but it's important to consider price point.

Final Thoughts

• The speaker recommends buying a GPU with as much RAM as possible while also considering price point.

• All opinions are subjective and open to discussion.