Parallel For Each Over Data Tables in a Data Set

user_3829855 0
Viewed 73

In VB.NET, can a parallel loop be performed on the tables inside a data set? This has been answered.

My project is this:

  1. Read a CSV file into a data table in memory.
  2. Group the rows by sorting the data.
  3. Process the data in synchronous loop detecting each group by a change in key values.

There are many separate processes that read these CSV files which each have their own layout and processing specifics.

My goal is to process these "groups" of data in parallel without using Entity Framework.

Is the Data Set/Data Table approach the best?

parallel-processing datatable vb.net
1 Answers

Yes, you can. A DataSet and a DataTable are in memory collections. So if you can process them independently it's a good candidate for a parallel For Each.

Parallel.ForEach(
    dataSet.Tables.Cast(Of DataTable),
    Sub(table)
        ProcessTable(table)
    End Sub
)
user_2842400
Related