Streaming with fetch API

user_8721981 0
Viewed 586

I have created .NET Core API and in it I expose a POST endpoint that streams the response over multiple chunks where each chunk contains a JSON object, and I have created Angular client app that queries that endpoint through the fetch API:

const response = await fetch(environment.apiUrl + `/Vehicle/ParseVehiclesData`,
{
  method: 'post',
  body: file,
});

const reader = response.body.getReader();
while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  let res = JSON.parse(new TextDecoder("utf-8").decode(value));
  console.log(res);
}

My goal here is to feed a progress bar with pieces of information about the progress which they already returned from the server in each chunk, now the problem I have is that a behavior occurs when users with weak connection speed tries it, their speed of consumption of the each chunk is less than speed of the server feed, which causes the browser to merge chunks together when there is a backpressure so my attempt to parse the decoded string fails because i will be trying to parse something like

{"name":"abc","progress":50}{"name":"ab1","progress":100"}

multiple json object will be attached together with no delimiters not even break lines, so what I'm looking for is solution that make prevents the merge of the chunks when there is a backpressure or in the worst case a way to parse JSON objects with no delimiters and feed them to a custom stream or an observable, any help about the subject would be appreciated.

these are some articles i have stumpled upon while searching, InternetSpeed

fetch-api stream javascript c#
1 Answers

You could define a pre-parser to split the msgs by }{ separator (and handle first/last entries):

let preParser = (a)=>a.split('}{').map((item,index) => {
       if(a.split('}{').length>1){
          if (index== 0){return item+'}'} // first msg
          else if (index<a.split('}{').length-1){return '{'+item+'}'} // middle msgs
          else {return '{'+item}} // last msg
       else {return a} // no need to do anything if single msg
})

a='{"name":"abc","progress":50}'

b='{"name":"abc","progress":50}{"name":"ab1","progress":100}{"name":"ab2","progress":150}'


console.log('example 1')
preParser(a).forEach(v => console.log(JSON.parse(v)))

console.log('example 2')
preParser(b).forEach(v => console.log(JSON.parse(v)))

user_14618500
Related