── pod
# NVIDIA RTX 2000 Ada Generation | SMs=22 | torch 2.14.1+cu130 cuda 13.0   (power limit 70 W, max SM clock 3105 MHz)
FP32 matmul (IEEE, no TF32)              6.00 TFLOPS |  1710 MHz |  69.3 W |  62.3% of full rate @ that clock
TF32 matmul                             17.14 TFLOPS |  1635 MHz |  69.4 W |  93.1% of full rate @ that clock
FP16 matmul, FP32 accum, n=8192         35.10 TFLOPS |  1695 MHz |  70.0 W |  91.9% of full rate @ that clock
FP16 matmul, FP16 accum, n=8192         37.30 TFLOPS |  1725 MHz |  68.8 W |  96.0% of full rate @ that clock
BF16 matmul (FP32 accum), n=8192        35.81 TFLOPS |  1725 MHz |  68.9 W |  92.1% of full rate @ that clock
FP16 matmul, FP32 accum, n=16384        35.46 TFLOPS |  1665 MHz |  69.4 W |  94.5% of full rate @ that clock
FP16 matmul, FP16 accum, n=16384        38.49 TFLOPS |  1755 MHz |  69.3 W |  97.3% of full rate @ that clock
BF16 matmul (FP32 accum), n=16384       36.40 TFLOPS |  1710 MHz |  69.6 W |  94.5% of full rate @ that clock
FP8 e4m3 scaled_mm, n=8192              67.45 TFLOPS |  1635 MHz |  69.5 W |  91.6% of full rate @ that clock
FP8 e4m3 scaled_mm fast-acc, n=8192     79.21 TFLOPS |  1800 MHz |  68.8 W |  97.7% of full rate @ that clock
FP8 e4m3 scaled_mm, n=16384             73.76 TFLOPS |  1740 MHz |  69.5 W |  94.1% of full rate @ that clock
FP8 e4m3 scaled_mm fast-acc, n=16384    77.78 TFLOPS |  1740 MHz |  68.5 W |  99.2% of full rate @ that clock
INT8 int_mm, n=8192                     17.84 TOPS   |  1740 MHz |  69.2 W |  22.8% of full rate @ that clock
INT8 int_mm (B col-major), n=8192       83.85 TOPS   |  1980 MHz |  68.4 W |  94.0% of full rate @ that clock
INT8 int_mm, n=16384                    17.85 TOPS   |  1740 MHz |  69.1 W |  22.8% of full rate @ that clock
INT8 int_mm (B col-major), n=16384      78.06 TOPS   |  1785 MHz |  69.5 W |  97.1% of full rate @ that clock
FP64 matmul                              0.19 TFLOPS |  2415 MHz |  28.4 W |  87.2% of full rate @ that clock
DRAM copy bandwidth (1 GiB)            206.57 GB/s   |  2415 MHz |  45.6 W |

── pod
# NVIDIA RTX A4000 | SMs=48 | torch 2.14.1+cu130 cuda 13.0   (power limit 140 W, max SM clock 2100 MHz)
FP32 matmul (IEEE, no TF32)             12.44 TFLOPS |  1282 MHz | 109.2 W |  78.9% of full rate @ that clock
TF32 matmul                             31.35 TFLOPS |  1320 MHz | 139.8 W |  96.6% of full rate @ that clock
FP16 matmul, FP32 accum, n=8192         61.81 TFLOPS |  1290 MHz | 130.8 W |  97.5% of full rate @ that clock
FP16 matmul, FP16 accum, n=8192         54.02 TFLOPS |  1185 MHz | 139.3 W |  92.7% of full rate @ that clock
BF16 matmul (FP32 accum), n=8192        60.49 TFLOPS |  1290 MHz | 132.8 W |  95.4% of full rate @ that clock
FP16 matmul, FP32 accum, n=16384        59.62 TFLOPS |  1230 MHz | 139.8 W |  98.6% of full rate @ that clock
FP16 matmul, FP16 accum, n=16384        62.91 TFLOPS |  1320 MHz | 139.8 W |  97.0% of full rate @ that clock
BF16 matmul (FP32 accum), n=16384       62.33 TFLOPS |  1275 MHz | 139.8 W |  99.5% of full rate @ that clock
FP8: no hardware support (needs sm_89+), skipped
INT8 int_mm, n=8192                     25.76 TOPS   |  1155 MHz | 139.8 W |  22.7% of full rate @ that clock
INT8 int_mm (B col-major), n=8192      116.78 TOPS   |  1290 MHz | 139.8 W |  92.1% of full rate @ that clock
INT8 int_mm, n=16384                    25.92 TOPS   |  1155 MHz | 139.9 W |  22.8% of full rate @ that clock
INT8 int_mm (B col-major), n=16384     116.94 TOPS   |  1268 MHz | 139.7 W |  93.9% of full rate @ that clock
FP64 matmul                              0.32 TFLOPS |  1950 MHz | 102.6 W |  85.1% of full rate @ that clock
DRAM copy bandwidth (1 GiB)            390.97 GB/s   |  1845 MHz | 128.5 W |