git push process takes a long time

Viewed 49

I'm doing a git push to my github repository and everything is going fine until it hits writing objects

writing objects: 22% (8/35), 13.4Mib / 21.00Kib/s the internet that the process uses is very insignificant

I have contracted a 50Mb upload service and I have already done a test in the browser and everything is going well,and I don't know why it only uses 20 Kib if I have 50 Mb of upload

I'm waiting an hour and very little progress I have a binary file an mp4 of 60Mb I am doing the push with ssh key

1 Answers

When Git pushes or fetches data, it doesn't just send the raw data. It first deltifies it, which causes similar objects to be written as differences against a base object, decreasing size, and then compresses the data.

When you're pushing large video files, they are already compressed. Thus, in many cases, the deltification and compression are ineffective and the compression can actually serve to make the data larger, but they are both still required. Attempting to compress already compressed data is also very slow because the compression algorithm will expend a lot of effort trying to make things better but won't be able to.

Normally, Git does this step during the phase where it says, "Compressing objects". However, in some cases, the delta information is too large to fit all into memory, and thus data may need to be redeltified during the "Writing objects" phase. In any event, the compression needs to be done at this point.

For the reasons I've mentioned, Git is not a good fit for large compressed files. Your video file needs to be compressed, which is a single-threaded operation by Git, even though this will expand the data slightly, and this is likely why your push is very slow: your computer is busy compressing the data it's sending over the pipe.

If your video is a build product, those don't belong in a repository, and you should store it elsewhere. Similarly, if this video is not a part of your project but just a how-to or some such, it also doesn't belong in your project.

If it's really part of your project, you may consider using Git LFS to store it instead of storing it in your repo. Note that if the video file is still in your history, then you still will push it; you'd have to remove the video file from your history entirely for this not to be a problem.

Related