The following code shows inference time of the network. However, I am not sure what the unit of time is. Read through the documentation here, but still confused
####This code is for measuring performance such as inference time, FLOPs, Parameters etc...
dummy_input = torch.randn(1, 3, 256, 256).cuda()
macs, params = profile(model, inputs=(dummy_input,), verbose=0)
macs, params = clever_format([macs, params], "%.3f")
name = "SegDepthWithTwoDecoders"
print("<" * 50, name)
print("Flops:", macs)
print("Parameters:", params)
starter, ender = torch.cuda.Event(enable_timing=True), torch.cuda.Event(
enable_timing=True
)
repetitions = 300
timings = np.zeros((repetitions, 1))
for _ in range(10):
_ = model(dummy_input)
# MEASURE PERFORMANCE
with torch.no_grad():
for rep in range(repetitions):
starter.record()
_ = model(dummy_input)
ender.record()
# WAIT FOR GPU SYNC
torch.cuda.synchronize()
curr_time = starter.elapsed_time(ender)
timings[rep] = curr_time
print("time :", np.average(timings))
###This is where the code ends for measuring the performance
This is what the output looks like
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< SegDepthWithTwoDecoders
Flops: 1.924G
Parameters: 3.986M
time : 12.67153577486674