Advice on asynchronous programming... VB .NET

Viewed 66

I have a desktop application where users can select a number of rows displayed in a form (customs declarations actually) and send them via web service to get a response with the information of their current state, that needs to be updated in the database.

Since now this process was made synchronously. Get the first row, consume the web service, wait for the response, update the state in database and then move to the next row. But the number of rows processed each time is growing (4000-5000 at a time) and its taking a while to complete.

So what I did is parallelize the consumption of the web service for each row more or less like this:

'I can't parallelize the transactions with the database (business logic reasons), 
'so I use a queue for storing the responses as they arrive 
Private ResponseQueue as ConcurrentQueue(Of WebServiceResponseDataType)

Private NumberOfRowsToSend as Integer
Private CounterResponsesProcessed as Integer

Public Sub Send(ByVal dataTableRowsToSend As DataTable) 
  
    NumberOfRowsToSend = dataTableRowstoSend.Rows.Count()
    
    'I use a separate Thread to launch the method that will be updating the database 
    'as the queue is filled with responses
    Dim responseProcessorThread as Thread
    responseProcessorThread = New Thread(AddressOf ResponseProcessor)
    responseProcessorThread.Start()

    For Each row In dataTableRowsToSend.Rows
         Dim dataToSend() as Object
        'Do some previous validations and data formatting here.
        'I need to get some IDs from another tables in the database 
        'and increment them before consuming the web servcie. 
        'To avoid duplicate IDs I keep this part synchronous.
         
        'If the validations are passed I use the ThreadPool to queue the consumption of the WebService
         ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf WebServiceConsumption)), dataToSend)
    Next
 
    'Wait for all responses to be processed
    'PS. I'm controlling if there is an exception during the webservice consumption,
    'generating a fake response in that case so the user can see what went wrong
     responseProcessorThread.Join()

End Sub

Private Sub ResponseProcessor()
        Dim WebServiceResponse As WebServiceResponseDataType

        While CounterResponsesProcessed < NumberOfRowsToSend 
             If ResponseQueue.Count > 0 AndAlso ResponseQueue.TryDequeue(WebServiceResponse) Then
                 UpdateRowStateInDataBase(WebServiceResponse)
                 CounterResponsesProcessed += 1
             End If
        End While
End Sub

Private Sub WebServiceConsumption (ByVal dataToSend() as Object)
     'This function consumes the web service and waits for the response. 
     'When the response arrives it goes to the queue:
      ResponseQueue.Enqueue(responseData)
End Sub

With this code the process is much faster since I don't have to wait for the response of the previous row to start with the next one.

But I'm not used to parallel programming and I'm not sure if this is a good approach or if I did something considered bad practice.

I.e. I dont feel comfortable using a counter to control when all the responses are updated in the database to know when the process is finished. Also after writing this code I have been reading that I should avoid using Threads directly and use Tasks instead.

0 Answers
Related