Compute a hash from a stream of unknown length in C#

jjxtra_56079 0
Viewed 23625

What is the best solution in C# for computing an "on the fly" md5 like hash of a stream of unknown length? Specifically, I want to compute a hash from data received over the network. I know I am done receiving data when the sender terminates the connection, so I don't know the length in advance.

[EDIT] - Right now I am using md5 and am doing a second pass over the data after it's been saved and written to disk. I'd rather hash it in place as it comes in from the network.

cryptography hash stream c#
6 Answers

This seems like a perfect use case for CryptoStream (docs).

I've used CryptoStream for processing unknown-length streams of database results that need to be gzipped and then transferred across the network along with a hash of the compressed file. Inserting a CryptoStream between the compressor and the file writer allows you to compute the hash on the fly so that it's ready as soon as the file is written.

The basic approach looks like this:

var hasher = MD5.Create();
using (FileStream outFile = File.Create(filePath))
using (CryptoStream crypto = new CryptoStream(outFile, hasher, CryptoStreamMode.Write))
using (GZipStream compress = new GZipStream(crypto, CompressionMode.Compress))
using (StreamWriter writer = new StreamWriter(compress))
{
    foreach (string line in GetLines())
        writer.WriteLine(line);
}
// at this point the streams are closed so the hash is ready
string hash = BitConverter.ToString(hasher.Hash).Replace("-", "").ToLowerInvariant();
user_2781850

Necromancing.

Two possibilitites in C# .NET Core:

private static System.Security.Cryptography.HashAlgorithm GetHashAlgorithm(System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
        return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.MD5.Create();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
        return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA1.Create();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
        return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA256.Create();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
        return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA384.Create();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
        return (System.Security.Cryptography.HashAlgorithm) System.Security.Cryptography.SHA512.Create();

    throw new System.Security.Cryptography.CryptographicException($"Unknown hash algorithm \"{hashAlgorithmName.Name}\".");
}


protected override byte[] HashData(System.IO.Stream data,
    System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
    using (System.Security.Cryptography.HashAlgorithm hashAlgorithm1 = 
    GetHashAlgorithm(hashAlgorithm))
    return hashAlgorithm1.ComputeHash(data);
}

or with BouncyCastle:

private static Org.BouncyCastle.Crypto.IDigest GetBouncyAlgorithm(
    System.Security.Cryptography.HashAlgorithmName hashAlgorithmName)
{
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.MD5)
        return new Org.BouncyCastle.Crypto.Digests.MD5Digest();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA1)
        return new Org.BouncyCastle.Crypto.Digests.Sha1Digest();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA256)
        return new Org.BouncyCastle.Crypto.Digests.Sha256Digest();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA384)
        return new Org.BouncyCastle.Crypto.Digests.Sha384Digest();
    if (hashAlgorithmName == System.Security.Cryptography.HashAlgorithmName.SHA512)
        return new Org.BouncyCastle.Crypto.Digests.Sha512Digest();

    throw new System.Security.Cryptography.CryptographicException(
        $"Unknown hash algorithm \"{hashAlgorithmName.Name}\"."
    );
} // End Function GetBouncyAlgorithm  



protected override byte[] HashData(System.IO.Stream data,
    System.Security.Cryptography.HashAlgorithmName hashAlgorithm)
{
    Org.BouncyCastle.Crypto.IDigest digest = GetBouncyAlgorithm(hashAlgorithm);

    byte[] buffer = new byte[4096];
    int cbSize;
    while ((cbSize = data.Read(buffer, 0, buffer.Length)) > 0)
        digest.BlockUpdate(buffer, 0, cbSize);

    byte[] hash = new byte[digest.GetDigestSize()];
    digest.DoFinal(hash, 0);
    return hash;
}
user_1550770

Another option could be to use the System.Security.Cryptography.IncrementalHash class instead.

byte[] DataBrick;
var IncMD5 = IncrementalHash.CreateHash(HashAlgorithmName.MD5);

then you can: accumulate data in the hasher

IncMD5.AppendData(DataBrick,0,DataBrick.Length);

,check the hash value for the data accumulated so far

byte[] hash = IncMD5.GetCurrentHash();
bytesReceived = netStream.Read(DataBrick,0,DataBrick.Length);
IncMD5.AppendData(DataBrick,0,bytesReceived);

,or stop and reset to start accumulating a new hash value

byte[] hash = IncMD5.GetHashAndReset();

Note: it implements iDisposable

IncMD5.Dispose(); // when done, or using(IncMD5){..} if that makes more sense in your scope
user_44965600
Related