Sometimes Getting ingestion failures in Kusto, with error -" Stream_WrongNumberOfFields"

user_12677136 0
Viewed 270

I am ingesting streaming data into Kusto. So first the record is ingested as a single string in the source table, then parsed and inserted into another target table.

My every file is having about 10000 records. Some records get ingested into the source table fine but sometimes I get this error:

"Stream_WrongNumberOfFields: HResult: 0x80DA0008, source: (CSV error: record 7 (line: 8, byte: 2767): found record with 1 fields, but the previous record has 2 fields)".

I get this error while monitoring using .show ingestion failures.

So some data is lost on every ingestion. The file is perfectly formatted. Any idea on what is going on?

The ingestion mapping is

.create table tempTable ingestion csv mapping 'MappingCSV' '[{"column":"Record","Ordinal":"0"}]'

bigdata kql azure-data-explorer
1 Answers

It looks like some rows have an extra comma (or missing a comma) in some of the lines, in the error message above it seems that this is the case with line 7.

If you are trying to ingest each row into the same column, set the ingestion data format to "TXT" instead of CSV, this will ingest each row in the source file to the first column of your table.

user_101679140
Related