Gatsby Large Site

You're throwing out a bunch of different stack questions at once, so let's break this down a bit.

Our site is currently on Drupal7. It is approaching 400k pages. Articles and product listing. We would like to move to a more modern platform and would like to decouple our CMS from our frontend.

That's a large site and as EliteRaceElephant suggested, Gatsby's build times will be unacceptable for a site that large. My experience suggests once you're over about 300-400 pages, build times get laggy (at least on Netlify). Gatsby is highly opinionated in how it goes about building your application, take a look for yourself. Don't get me wrong, I love Gatsby just doesn't sound like a great fit.

But you have a fairly static-sounding site, if it's just product listings and articles: I'd give Next.js a good look. It's still gives you many of the SSR benefits Gatsby does but with much more architectural flexibility. Their version 9 sniffs your codebase for which pages it can make static vs SSR.

I'm leaning towards prismic.io as an editorial CMS our products are in a seperate DB. My understanding is that Gatsby can help unify different data sources so that they are treated the same in the front end.

I've used Prismic and Contentful, and they get pricey quickly. It might just be better to leave everything in Drupal and run headless, plus your editorial is already in place. You might want to check out the Drupal-GraphQL module. You'd still have a migration from D7 up to D8 but that's probably going to be much more cost effective than the CMS-as-a-service options given how many pages you have.

I've heard react isn't the best for SEO, but have read on Gatsby's site that this isn't the case.

You're definitely going to want to look for an SSR solution if you jump into React, so you have some stable pages Google can query. React Helmet is a nice solution for SEO in React. It allows you a lot of customization, including doing the Google Structured Markup right.

Best of luck.

The Gatsby founder confirmed that they are working on incremental builds which would solve your problem. There is no indication how long that might take. They know this is a big problem issue so I would assume that it has high priority. See this github issue on the official github repository.

They also received additional funding ($15million) a few weeks ago so things are looking good for further improvements.

Here is blog post how a developer increased build speed.

One possibility is to create two projects that create two distinct pages. This does not solve the problem of slow build time but only divides it into two smaller parts:

• www.mysite.com: for frequent changes, build this site often
• www.artciles.mysite.com: for all the 100,000+ articles with infrequent changes, build this site only occasionally. But that would effect SEO negatively since all the artciles move to a new place.

At the moment, there is not that much you can do to improve build time. In the end you must make the decision:

• Can I handle the 30 minute build speed at the moment and wait until the gatsby team improves build time?

  • If yes: Use Gatsby.

  • If no: Use something else.

I am having the same problem very soon so I would be interested how you decided. Thanks. =)

It seems that Gatsby team fixed the problem with the performance for large web sites in Gatsby 2.9.0 version. https://www.gatsbyjs.org/blog/2019-06-12-performance-improvements-for-large-sites/