NextJS Using a memory cache in dev does not work

user_968988 0
Viewed 1409

I want to make a simple static blog site, and when I export my pages, everything is good and fast.

But in dev mode, my post generation is slow and I want to cache it, so that it is done only once. This is done in a post.ts file and my cache is const posts: Post[] = [] .

What I expect is that after typing yarn dev, the post.ts file is loaded once and I can fill and reuse my cached post array, but strangely this ts module is loaded many times.

import fs from 'fs'
import path from 'path'
// ...
​
console.log('????? reloading page')
const posts: Post[] = []  // cached posts, hope to generate once
let postsGenerated = false
​
export async function getSortedPostsData(): Promise<Post[]> {
  // Get file names under /posts
  const pendings: Promise<any>[] = []
  if (postsGenerated) {
    console.log('+++++ Using cache ')
    return sortPostsByDate(posts)
  } else {
    console.log('==== Calculating all')
    postsGenerated = true
  }
  let i = 0
  traverseDir('content/blog', (path) => {
    if (path.includes('.md')) { ... })
  }
      
  await Promise.all(pendings)
  return sortPostsByDate(posts)
}

The result is that sometimes cache is used, sometimes not. Even when reloading the same page, cache is not always called. Why ? And how to improve that ?

enter image description here

typescript next.js caching
2 Answers

It's probably how Fast Refresh works. You can't be sure that the server won't reload a .ts file once you start dev. Not sure how much it would improve, but you could try using a file as a cache instead of a bare const to persist the parsed data among reloads.

const devPostsCache = 'devPostsCache.json';
export async function getSortedPostsData(): Promise<Post[]> {
  // Get file names under /posts
  const pendings: Promise<any>[] = []
  const postGenerated = fs.existsSync(devPostsCache);
  if (postsGenerated) {
    const posts = JSON.parse(fs.readFileSync(devPostsCache));
    return sortPostsByDate(posts)
  } else {
    console.log('==== Calculating all')
    postsGenerated = true
  }
  let i = 0
  traverseDir('content/blog', (path) => {
    if (path.includes('.md')) { ... })
  }
  
  // not sure what pending promises exactly do
  // I'll assume you get an array of post
  const parsedPosts = await Promise.all(pendings);
  fs.writeFileSync(devPostsCache, JSON.stringify(parsedPosts));
  return sortPostsByDate(posts)
}

and delete it every time you start dev with a predev script

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "predev": "rm devPostsCache.json"
  },

UPDATE

The other approach I found to avoid recompiling the code too often is changing the onDemandEntries config.

module.exports = {
  onDemandEntries: {
    // period (in ms) where the server will keep pages in the buffer
    maxInactiveAge: 360 * 1000,
    // number of pages that should be kept simultaneously without being disposed
    pagesBufferLength: 10,
  },
}

The default is 60 * 1000 for maxInactiveAge and 2 for pagesBufferLength.

I also tried changing some webpack configs, but it didn't work because nextjs recompile the whole thing per request.

user_18680080
user_59340600
Related