Including static files in App Router RSCs

Created: Jan 27, 2024

Updated: Jan 27, 2024

5 min read

So next.js is supposed to be for building static websites, so including static files in the build should be easy, right?

Unfortunately, it's actually quite hard.

Andrew Lisowski

@HipsterSmoothie

3 months ago

Just did some crazy things with a webpack loader

What is Supported?

Next.js is great for generating static websites and doing it with either the Pages Router or the App Router is pretty simple. For the sake of this article, we'll be using the App Router.

Most docs and articles will tell you to do something along the lines of:

app/post.tsx

import glob from "fast-glob";
 
export default async function Page() {
  const posts = await glob("./posts/*.mdx");
  const posts = await Promise.all(
    posts.map(async (post) => {
      // do some stuff
    })
  );
 
  return (
    <div>
      {/* render the stuff */}
    </div>
  );
}

This works great and you'll be able to generate static pages really easily!

Now lets say you want to build and Server Component that uses a static file at runtime. When this is deployed the source files aren't deployed, it's deployed as build output. This means none of the source files are going to be where you expect them to be!

In the above example we use fast-glob to find all the files in the ./posts directory. That directory doesn't exist in the build output, so when you go to run the code on your website it will fail.

Where does Next.js say to put static files?

All the the docs and answers on StackOverflow will tell you to put your static files in the public directory.

You could put them there but:

They'll actually be public and hosted on your website
If you're using @next/mdx you have to put the files in specific directories, none of which are public

Given point 2 the public directory is out of the question. We would have to represent the files in the public directory in the posts directory.

Maybe you could write some code to copy the files into the public directory, but that adds a lot of complexity to the code and build process.

What can we do?

The key to solving this is realizing that everything we're doing above is leaning into node to load files. When you're building a next.js app you're building on top of a bundler, and a bundler's main job is including files in your website!

In webpack imports have a lot of power. Most people only use the import keyword to import a single file, but webpack has a feature that lets you gather a list of files and then import them dynamically as needed.

This API is called require.context and it's pretty powerful.

app/lib/posts.js

const files = require.context(
  // Look in the ./posts directory
  "./posts", 
  // Include subdirectories
  true, 
  // Only include files that end in .mdx
  /\.js$/
);
 
const data = files.keys().map((key) => {
  // Dynamically import the file when you need it!
  // NOTE: In this example we are using the default export
  //       but you can use any export you want!
  const post = files(key).default;
  // do some stuff
});

Now since we're using bundler APIs to include the files they will get included in the bundle that webpack generates.

Getting the Raw File Content

In the above example we're importing a .js file. When we do imports like this all the loaders in our webpack config will be applied to the file. Sometimes you might want this, like if you're actually importing some code you want transformed, but in our case we just want the raw file content.

We could configure webpack somehow to do something special for .mdx files in this case, but that would be a lot of work and prone to breaking things. Instead we can can lean into another lesser known webpack feature called "Inline Loaders".

Modifying the above example we can use the raw-loader to get the raw file content.

app/lib/posts.js

const files = require.context(
  "!!raw-loader!./posts", true, /\.js$/
);

Injecting More Data at Build Time

For my use case I also wanted to inject some git data about the file at build time. This is the exact same problem, when deployed the git data won't be available.

To solve this we can replace the raw-loader with a custom loader that will inject the git data at build time and include the source.

app/lib/my-custom-loader.js

const { execSync } = require("child_process");
const { urlToRequest } = require("loader-utils");
 
module.exports = function gitLoader(source) {
  // Get whatever data you want here
  const filepath = urlToRequest(this.resourcePath).replace("./", "");
  const { stdout } = execSync(
    `git log --diff-filter=A --format=%aI ${filepath}`
  );
 
  // Export the data as a default export
  return `export default ${JSON.stringify({
    source,
    creationDate: stdout,
  })}`;
};

All we have to do is update the inline loader with a relative path to our custom loader.

app/lib/posts.js

const files = require.context(
  "!!./my-custom-loader.js!./posts", true, /\.js$/
);
 
const data = files.keys().map((key) => {
  // Dynamically import the file when you need it!
  // NOTE: In this example we are using the default export
  //       but you can use any export you want!
  const { creationDate, source } = files(key).default;
  // do some stuff
});