What is the most suitable hash function for file integrity checking (checksums) to detect corruption?
I need to consider the following:
Wide range of file size (1 kb to 10GB+)
Lots of different file types
Large collection of files (+/-100 TB and growing)
Do larger files require higher digest sizes (SHA-1 vs SHA 512)?
I see that the SHA-family is referred to as cryptographic hash functions. Are they ill-suited for "general purpose" use such as detecting file corruption? Will something like MD5 or Tiger be better?
If malicious tampering is also a concern, will your answer change w.r.t the most suitable hash function?
External libraries are not an option, only whats available on Win XP SP3+.
Naturally performance is also of concern.
(Please excuse my terminology if it is incorrect, my knowledge on this subject is very limited).