I am trying to find some advantage of tensor cores by comparing the performance of cublasGemmEx and cublasDgemm on A100. As the doc describes, cublasGemmEx on cuda11.2+A100 support FP64, and I think cublasDgemm adapts the old algorithm(CUBLAS_GEMM_DEFAULT?), cublasGemmEx should be faster than cublasDgemm. But my experiment show that both have the same performance. does cublasDgemm already adapt tensor cores? btw, for m=n=k=5440, both have about 10 TFLOPS.
1 Answers
For kernels such as those used by cublas, using a profiler you can identify whether tensorcore is being used, generally speaking, just from the kernel name. For arbitrary kernels, the linked article shows a metric that can be used for this purpose, in nsight compute.
In the case of A100, the published data indicates that for non-tensorcore usage, the peak FP64 rate is 9.7TF. If you are witnessing a throughput higher than 9.7TF (for FP64) its a safe bet that tensorcore is being used. (Really any throughput higher than something like 90% of 9.7TF should be sufficient for this observation, as the quoted numbers are peak theoretical, not real-world-achievable).
The CUBLAS developers want to give the library users the best experience. I can't think of a sensible reason to use a slow path for an operation that can be done using tensorcore. You should expect such libraries to use tensorcore when possible.
In your case, since you are witnessing the same throughput for both operations, and that throughput exceeds 9.7TF, it's a safe bet, even without profiling, that tensorcore is being used for both.
Related