How to load content from a remote API with an Astro loader?

To load content from a remote API with an Astro loader, write a function that returns a loader object with a name, a schema and an async load() method, then use it as the loader of a collection in src/content.config.ts. Inside load(), fetch the API with the global fetch(), validate each item with parseData(), and save it with store.set(), optionally adding a digest so unchanged entries are skipped and a rendered value so Markdown bodies can be displayed with <Content />. Astro runs the loader at build time, stores the results, and your pages query them with getCollection() and getEntry() exactly like local Markdown.

For a quick one-off fetch you can get away with an inline function loader, but a real API usually brings pagination, authentication, errors and content that changes between builds. In this article you'll build a complete loader for a paginated REST API step by step: the schema, fetching every page, storing and rendering entries, skipping unchanged items, removing deleted ones, passing secrets safely, and using the collection in pages. Everything was tested with Astro 7.4 against the public JSONPlaceholder API, and the logs shown are from those runs.

When You Need a Custom Loader

Astro has two built-in loaders, glob() and file(), and both read local files. For anything remote you have three options:

• A community loader: many CMSs and services publish one. Check the Astro integrations directory before writing your own.
• An inline function loader: an async function returning an array of objects with id fields. Good for small, simple data.
• A loader object: full control through the loader context, with validation, Markdown rendering, digests, persistent metadata and logging.

The background on these options is in how to use the Content Layer API in Astro. This article goes deep on the third option, because that's what a production API needs.

The Example API

JSONPlaceholder exposes 100 fake blog posts at /posts. It supports _page and _limit query parameters, and returns the total count in an X-Total-Count header:

curl -sI "https://jsonplaceholder.typicode.com/posts?_page=1&_limit=20" | grep -i x-total-count

x-total-count: 100

Each post looks like this:

{
  "userId": 1,
  "id": 1,
  "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
  "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita..."
}

Your own API will differ, but the shape of the problem is the same: several pages of JSON, each item with a stable ID.

Step 1: Define the Schema

Start with a Zod schema describing one item. The loader uses it as the collection's default schema, which gives you validation and generated types without the user having to write a schema in their config:

// src/loaders/rest-loader.ts
import type { Loader } from "astro/loaders";
import { z } from "astro/zod";

const postSchema = z.object({
  id: z.number(),
  userId: z.number(),
  title: z.string(),
  body: z.string(),
});

type ApiPost = z.infer<typeof postSchema>;

z.infer turns the schema into a TypeScript type, so the fetch code below is typed too. If the collection in src/content.config.ts defines its own schema, that one overrides the loader's, so users can extend or reshape your data.

Step 2: Fetch Every Page

Keep the HTTP logic in its own function. It requests pages until it has every item, and throws on any non-2xx response:

// src/loaders/rest-loader.ts (continued)
interface RestLoaderOptions {
  /** Base URL of the API, e.g. https://api.example.com */
  baseUrl: string;
  /** Optional bearer token for private APIs */
  token?: string;
  /** Items per request */
  pageSize?: number;
}

async function fetchAllPosts(options: RestLoaderOptions): Promise<ApiPost[]> {
  const { baseUrl, token, pageSize = 20 } = options;
  const headers: Record<string, string> = token ? { Authorization: `Bearer ${token}` } : {};
  const all: ApiPost[] = [];

  for (let page = 1; ; page++) {
    const url = `${baseUrl}/posts?_page=${page}&_limit=${pageSize}`;
    const response = await fetch(url, { headers });

    if (!response.ok) {
      throw new Error(`GET ${url} failed: ${response.status} ${response.statusText}`);
    }

    const items: ApiPost[] = await response.json();
    all.push(...items);

    const total = Number(response.headers.get("x-total-count") ?? all.length);
    if (items.length < pageSize || all.length >= total) break;
  }

  return all;
}

Two stopping conditions keep this safe: a short page means there's nothing left, and reaching the total count means the same. Adapt them to whatever your API uses, whether that's a next cursor, a Link header or a hasMore flag.

Why Throwing Is the Right Default

If the API is down, you usually want the build to fail rather than deploy a site with an empty blog. Throwing inside a loader does exactly that. When I pointed a loader at a URL that returned 404, the build stopped with the loader's message and exited with code 1:

[content] Syncing content
Request to https://jsonplaceholder.typicode.com/nope failed with 404

Your previous deployment stays live, and you can retry once the API recovers. If some content is optional, catch the error for that part only and log a warning instead.

Step 3: Store, Validate and Render

Now the loader itself. It returns an object that satisfies the Loader type:

// src/loaders/rest-loader.ts (continued)
export function restLoader(options: RestLoaderOptions) {
  return {
    name: "rest-loader",
    schema: postSchema,
    load: async ({ store, meta, logger, parseData, generateDigest, renderMarkdown }) => {
      const lastSynced = meta.get("lastSynced");
      logger.info(`Syncing posts (last synced: ${lastSynced ?? "never"})`);

      const items = await fetchAllPosts(options);
      const seen = new Set<string>();
      let updated = 0;

      for (const item of items) {
        const id = String(item.id);
        seen.add(id);

        const data = await parseData({ id, data: item });
        const changed = store.set({
          id,
          data,
          body: item.body,
          digest: generateDigest(data),
          rendered: await renderMarkdown(item.body),
        });
        if (changed) updated++;
      }

      // Remove entries that were deleted upstream
      let removed = 0;
      for (const id of store.keys()) {
        if (!seen.has(id)) {
          store.delete(id);
          removed++;
        }
      }

      meta.set("lastSynced", new Date().toISOString());
      logger.info(`${items.length} posts, ${updated} updated, ${removed} removed`);
    },
  } satisfies Loader;
}

Here's what each part does:

• String IDs: entry IDs must be strings, and they end up in URLs. Use the API's permanent ID, never an array index.
• parseData(): validates the item against the schema (the collection's, if it has one, otherwise the loader's) and returns the parsed data. The store does no validation of its own.
• renderMarkdown(): renders the body with Astro's Markdown processor, which is Sätteri by default in Astro 7. Storing the result in rendered is what makes render(entry) work. If your API returns HTML, use rendered: { html: item.html } instead.
• digest: a hash of the data. store.set() returns false and skips the update when the digest matches the stored entry.
• Deleting stale entries: because the store persists between builds and this loader doesn't call store.clear(), it removes IDs that no longer exist upstream. Without this step, a deleted post would linger.
• meta: a small persistent key-value store. Here it just records the sync time, but it's also where you'd keep a sync token or updatedSince value.

Seeing the Digest Work

The first sync stored everything, and the second found nothing to update:

[rest-loader] Syncing posts (last synced: never)
[rest-loader] 100 posts, 100 updated, 0 removed
[rest-loader] Syncing posts (last synced: 2026-10-24T05:12:23.717Z)
[rest-loader] 100 posts, 0 updated, 0 removed

Notice that meta remembered the previous sync time. Be aware that any change to the content config clears the entire store and its metadata, so treat a missing lastSynced as "load everything".

Going Further: Incremental Fetching

The loader above still downloads every page on every build; the digest only saves work after the download. If your API supports a filter such as ?updated_since=, you can read meta.get("lastSynced"), request only changed items, and skip the stale-entry cleanup (or use a separate endpoint that lists deleted IDs). Whether that's possible depends entirely on your API, so check its documentation.

Step 4: Register the Collection

Use the loader in src/content.config.ts, passing options and reading any secret from the environment:

// src/content.config.ts
import { defineCollection } from "astro:content";
import { restLoader } from "./loaders/rest-loader";

const posts = defineCollection({
  loader: restLoader({
    baseUrl: "https://jsonplaceholder.typicode.com",
    token: import.meta.env.CMS_TOKEN,
  }),
});

export const collections = { posts };

With CMS_TOKEN=abc123 in a .env file, I confirmed that import.meta.env.CMS_TOKEN is available inside src/content.config.ts, while process.env.CMS_TOKEN was undefined. Use import.meta.env here. The token is only used at build time and never reaches the browser, but keep it out of version control all the same. More on this in how to use environment variables in Astro.

There's no collection schema here, so the loader's postSchema provides both validation and types. Running npx astro check on the project reported 0 errors.

Step 5: Use the Collection in Pages

Pages don't know or care that the data came from an API. This route builds one page per post and looks up its author in a second remote collection:

---
// src/pages/posts/[id].astro
import { getCollection, getEntry, render } from "astro:content";

export async function getStaticPaths() {
  const posts = await getCollection("posts");
  return posts.map((post) => ({
    params: { id: post.id },
    props: { post },
  }));
}

const { post } = Astro.props;
const author = await getEntry("users", String(post.data.userId));
const { Content } = await render(post);
---
<article>
  <h1>{post.data.title}</h1>
  <p>By {author?.data.name ?? "Unknown author"}</p>
  <Content />
</article>

The users collection here is a simple inline function loader that fetches /users and maps each user to { id: String(user.id), name, email }. The build produced all 100 pages, and /posts/42/ rendered with its title, the author name "Chelsey Dietrich" and the Markdown body as HTML paragraphs.

Looking up the author by ID like this is the simplest option for API data. For relationships declared in the schema with reference(), see how to reference one collection from another.

Keeping Remote Content Fresh

A build-time loader only fetches when the site builds. To pick up changes from the API:

1. Rebuild on publish: most CMSs can call a deploy hook URL on your host whenever content changes. This is the usual approach.
2. Schedule builds: a daily or hourly scheduled build works for APIs without webhooks.
3. Reload in development: press s then Enter in the astro dev terminal to re-run content sync.
4. Use a live collection: if content must be fresh on every request, a live loader in src/live.config.ts fetches at request time instead. It needs an adapter and can't render MDX or optimise images.

For the bigger picture on pulling CMS content into Astro, see how to use a headless CMS with Astro.

Packaging the Loader

Because a loader is just a function, you can move it into its own package and reuse it across projects. Export the factory function, accept configuration as options, document the schema it provides, and add the withastro and astro-loader keywords to package.json so it shows up in the integrations directory.

Troubleshooting

• Types are missing or stale: run npx astro sync to re-run the loader and regenerate types.
• "Cannot find name 'node:...'" in astro check: if your loader imports Node built-ins, install @types/node as a dev dependency.
• Entries disappear after a config change: expected. The store is cleared when the content config changes, and the next sync reloads everything.
• render() shows nothing: the loader didn't set rendered. Call renderMarkdown() or provide { html }.
• Slow builds: request larger pages, run independent requests with Promise.all(), and use digests so unchanged entries skip processing.

---

FAQ: Remote Content Loaders in Astro

---

Conclusion

A remote content loader is a small, focused piece of code: fetch every page of the API, validate each item with parseData(), and store it with a stable string ID, a digest and rendered content. Add stale-entry cleanup and persistent metadata, read secrets from import.meta.env, and throw on bad responses so a broken API never produces a broken site.

Once that's in place, the API becomes an ordinary content collection. Pages use getCollection(), getEntry() and render() just as they would with local Markdown, and the content stays current through deploy hooks or scheduled builds. When a build-time snapshot isn't fresh enough, the same ideas carry over to live loaders for request-time data.