C# Combine Archive Divided Into One File

user_14195664 0
Viewed 65

Code:

public void mergeFiles(string dir)
{
    for (int i = 0; i < parts; i++)
    {
        if (!File.Exists(dir))
        {
            File.Create(dir).Close();
        }
        var output = File.Open(dir, FileMode.Open);
        var input = File.Open(dir + ".part" + (i + 1), FileMode.Open);
        input.CopyTo(output);
        output.Close();
        input.Close();
        File.Delete(dir + ".part" + (i + 1));
    }
}

dir variable is for example /path/file.txt.gz

I have a file packed into a .gz archive. This archive is divided into e.g. 8 parts and I want to get this file.

The problem is that I don't know how to combine these files "file.gz.part1..." to extract them later.

When I use the above function, the archive is corrupted.
I have been struggling with it for a week, looking on the Internet, but this is the best solution I have found and it does not work.

Anyone have any advice on how to combine archive parts into one file?

divide gzip file c#
1 Answers

Your code has a few problems. If you look at the documentation for System.IO.Stream.Close you will see the following remark (emphasis mine):

Closes the current stream and releases any resources (such as sockets and file handles) associated with the current stream. Instead of calling this method, ensure that the stream is properly disposed.

So, per the docs, you want to dispose your streams rather than calling close directly (I'll come back to that in a second). Ignoring that, your main problem lies here:

var output = File.Open(dir, FileMode.Open);

You're using FileMode.Open for your output file. Again from the docs:

Specifies that the operating system should open an existing file. The ability to open the file is dependent on the value specified by the FileAccess enumeration. A FileNotFoundException exception is thrown if the file does not exist.

That's opening a stream at the beginning of the file. So, you're writing each partial file over the beginning of your output file repeatedly. I'm sure you noticed that your combined file size was only as large as the largest partial file. Take a look at FileMode.Append on the other hand:

Opens the file if it exists and seeks to the end of the file, or creates a new file. This requires Append permission. FileMode.Append can be used only in conjunction with FileAccess.Write. Trying to seek to a position before the end of the file throws an IOException exception, and any attempt to read fails and throws a NotSupportedException exception.

OK - but backing up even a step further, this:

if (!File.Exists(dir))
{
    File.Create(dir).Close();
}
var output = File.Open(dir, FileMode.Open);

... is ineffecient. Why would we check for the file existing n number of times, then open/close it n number of times? We can just create the file as the first step, and leave that output stream open until we have appended all of our data to it.

So, how would we refactor your code to use IDisposable while fixing your bug? Check out the using statement. Putting all of this together, your code might look like this:

public void mergeFiles(string dir)
{
    using (FileStream combinedFile = File.Create(dir))
    {
        for (int i = 0; i < parts; i++)
        {
            // Since this string is referenced more than once, capture as a
            // variable to lower risk of copy/paste errors.
            var splitFileName = dir + ".part" + (i + 1);
            using (FileStream filePart = File.Open(splitFileName, FileMode.Open))
            {
                filePart.CopyTo(combinedFile);
            }
            // Note that it's safe to delete the file now, because our filePart
            // stream has been disposed as it is out of scope.
            File.Delete(splitFileName);
        }
    }
}

Give that a try. And here's an entire working program with a contrived example that you can past into a new console app and run:

using System.IO;
using System.Text;

namespace temp_test
{
    class Program
    {
        static int parts = 10;
        static void Main(string[] args)
        {
            // First we will generate some dummy files.
            generateFiles();
            // Next, open files and combine.
            combineFiles();
        }

        /// <summary>
        /// A contived example to generate some files.
        /// </summary>
        static void generateFiles()
        {
            for (int i = 0; i < parts; i++)
            {
                using (FileStream newFile = File.Create("splitfile.part" + i))
                {
                    byte[] info = new UTF8Encoding(true).GetBytes($"This is File # ${i.ToString()}");
                    newFile.Write(info);
                }
            }
        }

        /// <summary>
        /// A contived example to combine our files.
        /// </summary>
        static void combineFiles()
        {
            using (FileStream combinedFile = File.Create("combined"))
            {
                for (int i = 0; i < parts; i++)
                {
                    var splitFileName = "splitfile.part" + i;
                    using (FileStream filePart = File.Open(splitFileName, FileMode.Open))
                    {
                        filePart.CopyTo(combinedFile);
                    }
                    // Note that it's safe to delete the file now, because our filePart
                    // stream has been disposed as it is out of scope.
                    File.Delete(splitFileName);
                }
            }
        }
    }
}

Good luck and welcome to StackOverflow!

user_37182460
Related