dpkg-deb fails with "dpkg-deb: error: tar -cf subprocess returned error exit status 1"

I'm trying to package a custom-built version of OpenCV into a .deb file. I've done this several times with other projects, but this time, dpkg-deb fails unrecoverable with

root@ml:/.../# dpkg-deb --build testpackage/
dpkg-deb: building package 'testpackage' in 'testpackage.deb'.
... just normal logging from tar as the 3 lines below ...
tar: ./include/opencv4/opencv2/ximgproc.hpp: File shrank by 5246 bytes; padding with zeros
tar: ./include/opencv4/opencv2/xphoto/bm3d_image_denoising.hpp: File shrank by 5022 bytes; padding with zeros
tar: ./include/opencv4/opencv2/xphoto/white_balance.hpp: File shrank by 2714 bytes; padding with zeros
dpkg-deb: error: tar -cf subprocess returned error exit status 1

It packages everything up until the last file and then crashes... This is the only output I get, no matter the parameters and seemingly independent of the files I want to package...

What I've tried so far

• Use different files for packaging (package other files than OpenCV)
• Run inside Docker (using debian:bullseye)
• Enable debug outut for dpkg-deb with --verbose and --debug flags (doesn't log more somehow)
• Change the compression type and compression level using -Z and -z

Here another exampel with different files and different directories:

root@ml:/.../# dpkg-deb --build --debug testpackage/
dpkg-deb: building package 'testpackage' in 'testpackage.deb'.
tar: ./opt/test/aruco.hpp: File shrank by 32041 bytes; padding with zeros
tar: ./opt/test/calib3d.hpp: File shrank by 212292 bytes; padding with zeros
tar: ./opt/test/core.hpp: File shrank by 149841 bytes; padding with zeros
tar: ./opt/test/core_detect.hpp: File shrank by 1423 bytes; padding with zeros
tar: ./opt/test/hfs.hpp: File shrank by 1058 bytes; padding with zeros
dpkg-deb: error: tar -cf subprocess returned error exit status 1
root@ml:/.../# 

If you need more logs/output/testing, just write a comment and I'll update the question

Thanks in advance :)