Moreover it requires a completely different skill set and expertise, with the low level understanding of GPU architecture (or several architectures). Maybe it is achievable, but I had not seen any DL developer wanting to spend time on such hardcore optimizations instead of working with the neural networks themselves. It’s because to achieve the peak performance you have to heavily optimize your calculations, keeping all parts of the processing pipeline optimally loaded, avoiding bottlenecks and so on.

More correctly to say, the real performance can be far behind the peak performance (and you’ll see it below). Important: Peak performance can be very far from the performance on the real tasks. But anyway, FP32 is a good common ground, because you’ll see that there are many caveats with others. So, you may see other charts with larger numbers. This is not the only option to measure, you’ll learn about FP16/FP64/INT8 soon. Important: This is FP32, a single-precision float, performance.

There are some activities and I’ll return to AMD at the end of the post. For the inference they are good as well, but here may play other factors (like size, power consumption, price, etc) depending on the target system you are developing a neural network (NN) for.Īmong GPUs the NVIDIA ones are beyond comparison, because almost every DL framework supports NVIDIA GPUs while have no support of AMD GPUs. The training is much more calculation intensive process than the inference, and GPUs are especially important for the training mode. The most modern DL systems are a mix of CPU and GPU, where the GPU does the heavy lifting, and CPU is responsible for loading the data into/from the memory of a graphics card and orchestrating the calculations. Matrix multiplications, the core of DL right now, are among these.

Modern GPUs contain a lot of simple processors (cores) and are highly parallel, which makes them very effective in running some algorithms. GPUs, Graphics Processing Units, are specialized processors originally created for computer graphics tasks.

Titan rtx fp64