My application is a documentation website. And the documentation is stored on github. I am using the github graphQL API to copy the files from the content repository to the site repository in the build. Because then everything is local and I don't have to keep making requests.
What I did is working in parts. My problem is that some of the files are being returned incomplete. Simple example to undestant:
I have:
{
text: "est simplement du faux texte employé dans la composition et la mise en page avant
impression. Le Lorem Ipsum est le faux texte standard de l'imprimerie depuis les années
1500, quand un imprimeur anonyme assembla ensemble des morceaux de texte pour réaliser un
livre spécimen de polices de texte. Il n'a pas fait que survivre cinq siècles, mais s'est
aussi adapté à la bureautique informatique, sans que son contenu n'en soit modifié"
}
And the request return me(even though the text is cut off the final key appears haha):
{
text: "est simplement du faux texte employé dans la composition et la mise en page avant
impression. Le Lorem Ipsum est le faux texte standard de l'imprimerie depuis les années
1500, quand un imprimeur anonyme assembla
}
I don't know if it could be a file size limit. The very large files that came incomplete. The one I tested that was broken was a 16855 line json. Please someone help me :(
If I have forgotten any important detail, just let me know and I will correct it.
Now, the code:
import * as fs from 'fs'
import * as path from 'path'
const dir = path.join('public', 'teste')
const openSource = {
githubConvertedToken: 'xxxx',
}
import fetch from 'node-fetch'
const body = {
query: `
query {
repository(owner: "xxxx", name: "xxxx") {
object(expression: "HEAD:") {
... on Tree {
entries {
name
object {
... on Blob {
text
}
}
}
}
}
}
}`,
}
const baseUrl = 'https://api.github.com/graphql'
const headers = {
'Content-Type': 'application/json',
Authorization: 'bearer ' + openSource.githubConvertedToken,
}
const assertDirExists = (dir_path) => {
if (!fs.existsSync(dir_path)) {
fs.mkdirSync(dir_path)
}
}
fetch(baseUrl, {
method: 'POST',
headers: headers,
body: JSON.stringify(body),
})
.then((res) => res.json())
.then((res) => {
let content = res
assertDirExists(dir)
const pathdir = path.join(dir, 'aaa')
let count = 0
content.data.repository.object.entries.forEach((element) => {
if (element.name.includes('json')) {
try {
const test = JSON.parse(element.object.text)
fs.writeFileSync(`${pathdir}${count}.json`, JSON.stringify(test))
count = count + 1
} catch (err) {
console.log('error', err)
}
}
})
})
.catch((error) => console.log('erroooooooooo', error))