Compression Performance Difference between swift and Managed(c#)

user_17495922 0
Viewed 119

I'm implementing LZF compress in managed memory environment and decompress from ios environment.

So this is my code implement lzf decompress like this in c#

private static int LZFDecompress(byte[] input, byte[] output)
{
    int inputLength = input.Length;
    int outputLength = output.Length;

    uint iidx = 0;
    uint oidx = 0;

    do
    {
        uint ctrl = input[iidx++];

        if (ctrl < (1 << 5)) /* literal run */
        {
            ctrl++;

            if (oidx + ctrl > outputLength)
            {
                //SET_ERRNO (E2BIG);
                return 0;
            }

            do
                output[oidx++] = input[iidx++];
            while ((--ctrl) != 0);
        }
        else /* back reference */
        {
            uint len = ctrl >> 5;

            int reference = (int)(oidx - ((ctrl & 0x1f) << 8) - 1);

            if (len == 7)
                len += input[iidx++];

            reference -= input[iidx++];

            if (oidx + len + 2 > outputLength)
            {
                //SET_ERRNO (E2BIG);
                return 0;
            }

            if (reference < 0)
            {
                //SET_ERRNO (EINVAL);
                return 0;
            }

            output[oidx++] = output[reference++];
            output[oidx++] = output[reference++];

            do
                output[oidx++] = output[reference++];
            while ((--len) != 0);
        }
    }
    while (iidx < inputLength);

    return (int)oidx;
}

and porting to swift like this

    private static func LZFDecompress(input: [UInt8],
                                      output: inout [UInt8])
        -> Int
    {
        let inputLength = input.count
        let outputLength = output.count
        
        var iidx = 0
        var oidx = 0
        
        repeat
        {
            var ctrl = Int(input[iidx])
            iidx += 1
            
            if ctrl < (1 << 5)
            {
                ctrl += 1
                
                if oidx + ctrl > outputLength
                {
                    return 0
                }
                
                repeat
                {
                    output[oidx] = input[iidx]
                    oidx += 1
                    iidx += 1
                    ctrl -= 1
                }
                while ctrl != 0
            }
            else
            {
                var len = ctrl >> 5
                var reference = oidx - ((ctrl & 0x1f) << 8) - 1
                
                if len == 7
                {
                    len += Int(input[iidx])
                    iidx += 1
                }
                
                reference -= Int(input[iidx])
                iidx += 1
                
                if oidx + len + 2 > outputLength
                {
                    return 0
                }
                
                if reference < 0
                {
                    return 0
                }
                
                output[oidx] = output[reference]
                oidx += 1
                reference += 1
                output[oidx] = output[reference]
                oidx += 1
                reference += 1
                
                repeat
                {
                    output[oidx] = output[reference]
                    oidx += 1
                    reference += 1
                    len -= 1
                }
                while len != 0
            }
        }
        while iidx < inputLength
        
        return oidx
    }

But I have a problem, it is a performance difference. It costs 2-3 seconds in c# but costs 9-10 seconds in swift to decompress same files... I can't understand this situation.

I tested c# from console in windows. And I tested swift from playground or project in mac.

swift c#
1 Answers

It was not efficient code unconsidered SIMD and CPUs. So I used decompress method (lz4, zlib) is provided by apple. It's so faster than before. It costs below 1-second decompressing 200Mb file.

But In a managed environment (c#) it's slower than unmanaged. If you want more performance, implement native.

I use these lzib managed codes.

It costs 6-7 seconds for decompressing and 30 seconds for compressing at the same file.

And then you should know this code to be compatible to lzip in apple. It includes adding the header for compressed data.

    public static byte[] Compress(byte[] inputData)
    {
        var zlib = new ZlibCodec(CompressionMode.Compress);
        zlib.CompressLevel = Zlib.CompressionLevel.AppleSupported; // Level5
        zlib.InputBuffer = inputData;
        zlib.OutputBuffer = new byte[inputData.Length];
        zlib.NextIn = 0;
        zlib.AvailableBytesIn = inputData.Length;
        zlib.NextOut = 0;
        zlib.AvailableBytesOut = inputData.Length;
        zlib.InitializeDeflate(Zlib.CompressionLevel.AppleSupported, false); 
        // 'false' means it's 1951(deflate) version not 1950(lzib) version
        zlib.Deflate(FlushType.Finish);
        var output = new byte[zlib.TotalBytesOut];
        Array.Copy(zlib.OutputBuffer, output, (int)zlib.TotalBytesOut);
        return output;
    }

    public static byte[] Decompress(byte[] inputData, int outputSize)
    {
        var zlib = new ZlibCodec(CompressionMode.Decompress);
        zlib.CompressLevel = Zlib.CompressionLevel.AppleSupported;
        zlib.InputBuffer = inputData;
        zlib.OutputBuffer = new byte[outputSize];
        zlib.NextIn = 0;
        zlib.AvailableBytesIn = inputData.Length;
        zlib.NextOut = 0;
        zlib.AvailableBytesOut = outputSize;
        zlib.InitializeInflate(false);
        zlib.Inflate(FlushType.Finish);
        var output = new byte[zlib.TotalBytesOut];
        Array.Copy(zlib.OutputBuffer, output, (int)zlib.TotalBytesOut);
        return output;
    }

I wish to help the same person like me who implement multi-platform compressing.

user_174959220
Related