Can I push file to multiple devices using one adb command?

You can do something like this using awk:

awk 'NR>1 && $2 == "device" {system("adb -s " $1 " shell date")}' < <(adb devices)

this example runs date on all connected devices, you can replace it with your push.

xargs can also be used but previous command is simpler.