Azure CosmosDB: StackOverflowException After Calling UpsertDocumentAsync

Viewed 41

I have a durable orchestration that calls several activity triggers in a Monitor pattern.

After the "monitor" part of the code, and the function has received the required data, I am writing to Azure Cosmos DB.

I use a combination of StoredProcedures and Upsert to organize the data into two distinct collections.

The issue is lately after the UpsertDocumentAsync method (DocumentClient) is called, I get a StackOverflowException very often.

Weird thing is this didn't happen previously. This code was working with no issues.

Here's a representation of the code below:

public async Task UploadData(JobStatusObject input, DocumentClient client)
    {
        var parents = new List<Parent>();
        var parentCollectionLink = UriFactory.CreateDocumentCollectionUri("database", "Parents");
        
        foreach (var fileId in input.files)
        {
            // Fetch parent
            var zipFile = await _myClient.DownloadFile(input.JobId, fileId);

            var unzippedFiles = _zipper.UnzipBytesArrayToBytesFile(zipFile);

            foreach (var file in unzippedFiles)
            {
                var fileData = Encoding.UTF8.GetString(file, 0, file.Length);
                parents.AddRange(JsonConvert.DeserializeObject<List<Parent>>(fileData));
            }
        }

        var bulkDeleteSpLink = await CreateSpIfNotExists(client, "database", "child", "bulkDeleteSproc");
        var bulkInsertSpLink = await CreateSpIfNotExists(client, "database", "child", "bulkImport");

        var batchSize = 100;

        var count = 0;

        // Upsert parents           
        foreach (var parent in parents)
        {
            var requestOptions = new RequestOptions
            {
                PartitionKey = new PartitionKey(parent.id),
                EnableScriptLogging = true
            };

            _logger.LogDebug($"HttpParentPull: Deleting all lines for ParentNumber={parent.id}");
            bool continuation;
            do
            {
                var resp = await client.ExecuteStoredProcedureAsync<BulkDeleteResponse>(bulkDeleteSpLink, requestOptions, new dynamic[] { $"select c._self FROM c" });
                continuation = resp.Response.Continuation;
            }
            while (continuation);

            _logger.LogDebug($"HttpParentPull: Inserting lines for ParentNumber={parent.id}");
            var numberOfBatches = (parent.ChildrenItems.Count + batchSize - 1) / batchSize;

            parent.ChildrenItems.ForEach(l => l.ParentId = parent.id);

            for (int i = 0; i < numberOfBatches; i++)
            {
                await client.ExecuteStoredProcedureAsync<int>(bulkInsertSpLink, requestOptions, new dynamic[] { parent.ChildrenItems.Skip(i * batchSize).Take(batchSize).ToArray() });
            }

            _logger.LogDebug($"HttpParentPull: Upserting ParentNumber={parent.id}");
            parent.ChildrenItems = null;
            await client.UpsertDocumentAsync(parentCollectionLink, parent, disableAutomaticIdGeneration: true);
        }
        
    }

Is my code overflowing when calling a Microsoft side function??? To my understanding the design of my function does not use recursion or is in any kind of infinite loop..

1 Answers

I found the issue. It was in the definition of the "parent" class. I had accidentally added recursion logic to the ttl field, something like:

// DON'T DO THIS    
public int ttl get { if (condition) { return 30 days; }else {return ttl;} }

So when I would try to upsert, there would be an infinite loop scenario where the parent had a ttl value that didn't match the condition.

I fixed it by not returning ttl into itself:

// DO THIS
public int ttl get { if(condition) {return 30 days;}else{ return -1;} }
Related