I'm currently programming a video sharing site. For the uploads, I'm using PHP. I noticed that when I upload a video, S3 takes an MD5 hash of the file. I'm wondering if S3 does any deduplication. I uploaded several of the same file and didn't see anything in there indicating S3 knew they were the same or that it was doing anything about it at least.
Should I implement this on my own? I have a MySQL database where I'm storing all the video info. I could take a hash of all the videos and serve up previously uploaded files if they're the same. I could simply do md5(tmp-file-here). This would seem appropriate since S3 is already using md5. However, md5 is slow when compared to something optimized for such things such as BLAKE2. Should I do this and what would be the best approach?