reading a file in js is corupting the file

user_735897 0
Viewed 95

in js i'm running this code

let str = new TextDecoder("utf8").decode(new Uint8Array(reader.result));

let data = {
    name:file.name,
    size: args.loaded,
    contentsize:str.length,
    type:file.type,
    content:str
};
let res = await $.post({
    url:"/api/files",
    data:JSON.stringify(data),
    processData: false,
    contentType: "application/json",
})

reader.result is the output of

reader.readAsArrayBuffer(file);

which is then being picked up in php

$new["actual"] =strlen($new["content"]);
$new["content"] = new MongoDB\BSON\Binary($new["content"] ,MongoDB\BSON\Binary::TYPE_GENERIC);
$result = IO::$db->files->insertOne($new);

which then results in a db document of

{ 
    "name" : "test.pdf", 
    "size" : NumberInt(128454), 
    "contentsize" : NumberInt(122693), 
    "type" : "application/pdf", 
    "actual" : NumberInt(215693)
}

however when i remove the content of the file back to the file system the file is corrupted and wont open. i've run the file comparer that comes with VSCode over it and says there are 0 differences between the DB content and the original, so i can only think the issues is with the 3kb difference in the file size and the read content, but what or why escapes me

note only seems to be Binary files

pdf php ajax javascript
2 Answers

Your example is a PDF file. PDF files are not text files and should not be handled as text files. If you change line endings, this will change the offset of the different objects in the file, which will corrupt the PDF file. Even worse, if there are binary streams in the PDF file (such as for fonts, images etc...), it will break the contents of the streams and likely make it impossible to decode them.

To solve this, make sure you read and write PDF files as binary files.

user_16829240

OK the UTF-8 specification does not include characters for all bit codes in the byte data range, other languages such as PHP get around this by adding special non-UTF-8 characters that complete the byte code mapping, but this is not the case in JavaScript

So this is a limitation of JavaScript and that instead of telling you that it has hit a byte code it can't de-serialise which would highlight that an issue has occurred it just skips that byte an moves on to the next, corrupting your data and hiding why that corruption occurred.

this means that you can't use the UTF-8 encoder in JavaScript to encode anything includes one of the none supported byte codes, to get around this one option would be to convert the byte to is HEX string and then use that to de-serialise on the server side, this causes a degree of data bloating as you are using 2 bytes to send one byte (though this is much less than trying to encode an array of bytes) but as a 2 letter hex string maps all byte range values means there is no corruption of the binary data and can be safely used in a text based data serialisation like JSON via JavaScript

user_7358970
Related