Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Post History
It turns out there is a distinction between the API (i.e. the CUDA docs) and the actual implementation. In the Hopper whitepaper that is linked in the question (h100 SXM5), the implementation deta...
#1: Initial revision
by
mr Tsjolder
·
2026-04-07T12:55:42Z (5 months ago)
It turns out there is a distinction between the API (i.e. the CUDA docs) and the actual implementation.
In the Hopper whitepaper that is linked in the question (h100 SXM5), the implementation details are communicated by means of figures 8-11.
Concretely, each figure shows 4 clusters for each architecture, corresponding to the 4 tensor cores per SM.
The gray cubes apparently correspond to the number of performed MACs and the dimensions of one such a block of cubes are then the matrix dimensions that can be handled by a single tensor core per cycle.
For the example with the FP32/TF32 throughput of the tensor cores, this means that a single core computes the product of a 4x8 matrix with a 8x8 matrix, corresponding to $2 * 4 * 8 * 8 = 512\,\text{FLOPs}$ (according to figure 11).
As a result, we obtain the peak tensor core performance reported in the document: $528\,\text{cores} * 1\,830\,\text{MHz} * 512 \,\frac{\text{FLOPs}}{\text{core}} = 494.714 \,\frac{\text{TFLOPs}}{\text{s}}.$
I find it a bit weird that this information is only communicated implicitly by means of a figure, but this seems to resolve the issues for most non-consumer architectures that I looked into.