How to pass large amounts of data to process.stdin.write in Deno?

user_1540350 0
Viewed 197

Passing large amounts of data to stdin will fail. If you run this script under unix, you will get only a portion of the output of the website in the terminal:

const cat = Deno.run({
    cmd: ["cat"],
    stdin: "piped"
});
await cat.stdin.write(new Uint8Array(
    await (
            await fetch("https://languagelog.ldc.upenn.edu/nll/?feed=atom")
    ).arrayBuffer()
));
cat.stdin.close();
await cat.status();

The sample feed ends with </feed>, but the pipe will swallow in the middle:

Screen Shot 2021-07-13 at 00 22 38

Is there a way to circumvent this issue or did I spot a bug?

deno javascript
1 Answers

No other than Ryan Dahl himself answered me:

stdin.write is just one syscall, it returns the number of bytes written. If you use writeAll, I think it would work.

That said, ideally you'd stream large data, rather than buffer it up.

import { readerFromStreamReader } from "https://deno.land/std@0.100.0/io/streams.ts";

const cat = Deno.run({
  cmd: ["cat"],
  stdin: "piped",
});
const res = await fetch("https://languagelog.ldc.upenn.edu/nll/?feed=atom");
let r = readerFromStreamReader(res.body.getReader());
await Deno.copy(r, cat.stdin);
cat.stdin.close();
await cat.status();
user_15403500
Related