How to efficiently write a large text file in C#?

jessegavin_5651 0
Viewed 48242

I am creating a method in C# which generates a text file for a Google Product Feed. The feed will contain upwards of 30,000 records and the text file currently weighs in at ~7Mb.

Here's the code I am currently using (some lines removed for brevity's sake).

public static void GenerateTextFile(string filePath) {

  var sb = new StringBuilder(1000);
  sb.Append("availability").Append("\t");
  sb.Append("condition").Append("\t");
  sb.Append("description").Append("\t");
  // repetitive code hidden for brevity ...
  sb.Append(Environment.NewLine);

  var items = inventoryRepo.GetItemsForSale();

  foreach (var p in items) {
    sb.Append("in stock").Append("\t");
    sb.Append("used").Append("\t");
    sb.Append(p.Description).Append("\t");
    // repetitive code hidden for brevity ...
    sb.AppendLine();
  }

  using (StreamWriter outfile = new StreamWriter(filePath)) {
      result.Append("Writing text file to disk.").AppendLine();
      outfile.Write(sb.ToString());
  }
}

I am wondering if StringBuilder is the right tool for the job. Would there be performance gains if I used a TextWriter instead?

I don't know a ton about IO performance so any help or general improvements would be appreciated. Thanks.

text-files io c#
4 Answers

This might be old but I had a file to write with about 17 million lines so I ended up batching the writes every 10k lines similar to these lines

for (i6 = 1; i6 <= ball; i6++) 
{ //this is middle of 6 deep nest ..
  counter++;
  // modus to get a value at every so often 10k lines
  divtrue = counter % 10000; // remainder operator % for 10k
  //  build the string of fields with \n at the end 
  lineout = lineout + whatever 
  // the magic 10k block here
  if (divtrue.Equals(0))  
  {
     using (StreamWriter outFile = new StreamWriter(@filepath, true))
     { 
         //  write the 10k lines with .write NOT writeline..
         outFile.Write(lineout); 
     } 
     // reset the string so we dont do silly like memory overflow
     lineout = ""; 
  }
}

In my case it was MUCH faster then one line at a time.

user_97647830
Related