Pagination (take five: success!)

❗ This is not a tutorial ❗

Reading https://www.wpgraphql.com/2020/03/26/forward-and-backward-pagination-with-wpgraphql, I realized the problem I’m having with pagination is that I’m trying to implement page based pagination but WPGraphQL has been optimized for cursor based pagination. The best work around I’ve found is to store the cursor data in a PaginationCursors table in the app’s database. Note, I haven’t implemented this:

Table: PaginationCursors
- id (Primary Key)
- archiveType (e.g., 'all', 'category', 'tag')
- archiveId (Nullable, can store categoryId or tagId depending on the archiveType)
- pageNumber (The page number in the pagination sequence)
- cursor (The actual cursor value)
- createdAt (Timestamp to track when the cursor was stored)
- updatedAt (Timestamp to track when the cursor was last updated)

This would technically work, but it’s overly complex. The article I linked to makes a good case against page based pagination…

I’m going to try adding cursor based “Previous” and “Next” buttons to the app’s archive page.

Cursor based pagination (take two)

I’ll start this on a new branch:

$ git checkout -b cursor_based_pagination

The app’s existing routes, with the pagination UI rendering on the blog.archive.tsx route and the pagination results rendering on blog.archive._index.tsx, should be able to stay as they are. Paginator is also a good enough name for the updated pagination component. I’ll make copies of all these files for later reference:

$ cp app/routes/blog.archive.tsx app/routes/blog.archiveBak.tsx
$ cp app/routes/blog.archive._index.tsx app/routes/blog.archiveBak._index.tsx
$ cp app/components/Paginator.tsx app/components/PaginatorBak.tsx

Copying files in this way isn’t a “best practice.”

Get cursor data and post details in a single query

This query from the WPGraphQL pagination blog post is useful:

query GET_PAGINATED_POSTS(
    $first: Int
    $last: Int
    $after: String
    $before: String
  ) {
    posts(first: $first, last: $last, after: $after, before: $before) {
      pageInfo {
        hasNextPage
        hasPreviousPage
        startCursor
        endCursor
      }
      edges {
        cursor
        node {
          id
          postId
          title
        }
      }
    }
  }

Loading the query into the GraphiQL IDE and hovering over the variable names reveals descriptions of what each variable does:

first: the number of items to return after the referenced “after” cursor
last: the number of items to return after the referenced “before” cursor
after: cursor used along with the “first” argument to reference where in the dataset to get data
before: cursor used along with the “last” argument to reference where in the dataset to get data

The IDE also provides details about each property that’s returned by the query. For pagination, I’m interested in:

hasNextPage: when paginating forward, are there more items?
hasPreviousPage: when paginating backwards, are there more items?
startCursor: when paginating backwards, the cursor to continue (to use as the value of the next query’s before argument.)
endCursor: when paginating forwards, the cursor to continue (to use as the value of the next query’s after argument.)

Note that an individual post’s cursor can be returned in the RootQueryToPostConnectionEdge results:

but for creating the “Previous” and “Next” UI elements, it seems that pageInfo.startCursor and pageInfo.endCursor will provide all the necessary information.

Also note that postId is deprecated in favour of databaseId.

The GraphiQL IDE is super useful. Thanks to whoever made it!

This query gets the details required to create a pagination “page” and a page’s pagination UI with a single request:

// app/models/wp_queries.ts

...
// todo: consider renaming this:
export const ARCHIVE_QUERY = gql(`
query ArchiveQuery (
    $first: Int
    $last: Int
    $after: String
    $before: String
  ) {
    posts(first: $first, last: $last, after: $after, before: $before) {
      pageInfo {
        hasNextPage
        hasPreviousPage
        startCursor
        endCursor
      }
      edges {
        cursor
        node {
          id
          databaseId
          title
          date
          slug
          excerpt
          author {
            node {
              name
            }
          }
          featuredImage {
            node {
              altText
              sourceUrl
            }
          }
        }
      }
    }
  }
`);
...

Pull in data with the `ARCHIVE_QUERY`

Querying for the first three posts:

// app/routes/blog.archive.tsx

import { json, LoaderFunctionArgs } from "@remix-run/node";
import { Outlet, useLoaderData, useRouteError } from "@remix-run/react";

import { createApolloClient } from "lib/createApolloClient";
import { ARCHIVE_QUERY } from "~/models/wp_queries";

export const loader = async ({ request }: LoaderFunctionArgs) => {
  const { searchParams } = new URL(request.url);
  const cursor = searchParams.get("cursor") || null;
  const client = createApolloClient();

  const response = await client.query({
    query: ARCHIVE_QUERY,
    variables: {
      first: 3,
      after: cursor,
    },
  });

  // note: the case of `response.data.posts.edges` being empty should be handled by the UI
  if (response.errors || !response?.data?.posts?.pageInfo) {
    throw new Error("An error was returned loading the posts.");
  }

  // data for the Paginator component:
  const pageInfo = response.data.posts.pageInfo;
  // data for the PostExcerptCard components:
  // if no RootQueryToPostConnectionEdge objects are returned, deal with it in the UI?
  const postConnectionEdges = response.data.posts?.edges || [];

  return json({ pageInfo: pageInfo, postConnectionEdges: postConnectionEdges });
};

export default function Archive() {
  const { pageInfo, postConnectionEdges } = useLoaderData<typeof loader>();
  console.log(`pageInfo: ${JSON.stringify(pageInfo, null, 2)}`);
  console.log(
    `postConnectionEdges: ${JSON.stringify(postConnectionEdges, null, 2)}`
  );

  return (
    <div className="px-6 mx-auto max-w-screen-lg">
      <h2 className="text-3xl py-3">Post Archive</h2>
      <Outlet />
    </div>
  );
}

export function ErrorBoundary() {
  const error = useRouteError();

  const errorMessage = error instanceof Error ? error.message : "Unknown error";
  return (
    <div className="mx-auto max-w-3xl px-5 py-4 my-10 bg-red-200 border-2 border-red-700 rounded break-all">
      <h1>App Error</h1>
      <pre>{errorMessage}</pre>
    </div>
  );
}

Here’s the value of pageInfo:

pageInfo: {
  "__typename": "RootQueryToPostConnectionPageInfo",
  "hasNextPage": true,
  "hasPreviousPage": false,
  "startCursor": "YXJyYXljb25uZWN0aW9uOjE0Ng==",
  "endCursor": "YXJyYXljb25uZWN0aW9uOjEyNA=="
}

That data can be used to tell the Paginator component to create a clickable “Next” link ("hasNextPage": true) with a cursor (or endCursor) URL param set to "YXJyYXljb25uZWN0aW9uOjEyNA==" ("endCursor": "YXJyYXljb25uZWN0aW9uOjEyNA==".) It can also tell the component to not create a clickable “Previous” link ("hasPreviousPage": false.)

Here’s where the updated Paginator component is at:

// app/components/Paginator.tsx

import type { RootQueryToPostConnectionPageInfo } from "~/graphql/__generated__/graphql";
// todo: left over from previous implementation
// delete this file if it doesn't get used:
//import { Page } from "~/types/Page";

interface PaginatorProps {
  pageInfo: RootQueryToPostConnectionPageInfo;
}

export default function Paginator({ pageInfo }: PaginatorProps) {
  const hasNextPage = pageInfo.hasNextPage;
  const hasPreviousPage = pageInfo.hasPreviousPage;
  const startCursor = pageInfo.startCursor;
  const endCursor = pageInfo.endCursor;

  console.log(`hasNextPage: ${hasNextPage}`);
  console.log(`hasPreviousPage: ${hasPreviousPage}`);
  console.log(`startCursor: ${startCursor}`);
  console.log(`endCursor: ${endCursor}`);

  return (
    <div>
      <p className="text-3xl">Paginator Component</p>
    </div>
  );
}

It’s included on blog.archive.tsx like this:

// app/routes/blog.archive.tsx

...
export default function Archive() {
  const { pageInfo, postConnectionEdges } = useLoaderData<typeof loader>();

  return (
    <div className="px-6 mx-auto max-w-screen-lg">
      <h2 className="text-3xl py-3">Post Archive</h2>
      <Outlet />
      <Paginator pageInfo={pageInfo} />
    </div>
  );
}
...

With the current route structure, visiting /blog/archive runs the blog.archive loader function, renders the Paginator component, then renders the blog.archive._index.tsx component in the blog.archive.tsx outlet. This might be wrong, but I’ll deal with that later.

Using the RootQueryToPostConnectionPageInfo prop in the Paginator component:

// app/components/Paginator.tsx

import { Link } from "@remix-run/react";
import type { RootQueryToPostConnectionPageInfo } from "~/graphql/__generated__/graphql";
// todo: left over from previous implementation
// delete this file if it doesn't get used:
//import { Page } from "~/types/Page";

interface PaginatorProps {
  pageInfo: RootQueryToPostConnectionPageInfo;
}

export default function Paginator({ pageInfo }: PaginatorProps) {
  const hasNextPage = pageInfo.hasNextPage;
  const hasPreviousPage = pageInfo.hasPreviousPage;
  const startCursor = pageInfo.startCursor;
  const endCursor = pageInfo.endCursor;

  return (
    <div>
      {hasPreviousPage && startCursor ? (
        <Link to={`?hasPreviousPage=true&startCursor=${startCursor}`}>
          Previous
        </Link>
      ) : (
        ""
      )}
      {hasNextPage && endCursor ? (
        <Link to={`?hasNextPage=true&endCursor=${endCursor}`}>Next</Link>
      ) : (
        ""
      )}
    </div>
  );
}

I like the way things are getting simplified. The next trick is to figure out how to render the posts. Maybe the blog.archive._index.tsx route can be removed?

This is getting silly, but it’s for a quick test:

$ mv app/routes/blog.archive._index.tsx app/routes/blog.archiveBakTmp._index.tsx

With the _index route removed, I’ll try getting the Paginator search params in the blog.archive.tsx loader:

And just like that it works! This seems like a big improvement from yesterday’s code:

// app/routes/blog.archive.tsx

import { json, LoaderFunctionArgs } from "@remix-run/node";
import { useLoaderData, useRouteError } from "@remix-run/react";
import { createApolloClient } from "lib/createApolloClient";
import { ARCHIVE_QUERY } from "~/models/wp_queries";
import Paginator from "~/components/Paginator";
import { Maybe } from "graphql/jsutils/Maybe";
import type { PostConnectionEdge } from "~/graphql/__generated__/graphql";
import PostExcerptCard from "~/components/PostExcerptCard";

interface ArchiveQueryVariables {
  first: Maybe<number>;
  last: Maybe<number>;
  after: Maybe<string>;
  before: Maybe<string>;
}

export const loader = async ({ request }: LoaderFunctionArgs) => {
  const { searchParams } = new URL(request.url);
  const startCursor = searchParams.get("startCursor") ?? null;
  const endCursor = searchParams.get("endCursor") ?? null;
  const chunkSize = Number(process.env?.ARCHIVE_CHUNK_SIZE) || 15;

  let queryVariables: ArchiveQueryVariables = {
    first: null,
    last: null,
    after: null,
    before: null,
  };
  if (endCursor) {
    queryVariables.first = chunkSize;
    queryVariables.after = endCursor;
  } else if (startCursor) {
    queryVariables.last = chunkSize;
    queryVariables.before = startCursor;
  } else {
    queryVariables.first = chunkSize;
  }

  const client = createApolloClient();
  const response = await client.query({
    query: ARCHIVE_QUERY,
    variables: queryVariables,
  });

  // note: the case of `response.data.posts.edges` being empty should be handled by the UI
  if (response.errors || !response?.data?.posts?.pageInfo) {
    throw new Error("An error was returned loading the posts.");
  }

  // data for the Paginator component:
  const pageInfo = response.data.posts.pageInfo;
  // data for the PostExcerptCard components:
  // if no RootQueryToPostConnectionEdge objects are returned, deal with it in the UI?
  const postConnectionEdges = response.data.posts?.edges || [];

  return json({ pageInfo: pageInfo, postConnectionEdges: postConnectionEdges });
};

export default function Archive() {
  const { pageInfo, postConnectionEdges } = useLoaderData<typeof loader>();

  return (
    <div className="px-6 mx-auto max-w-screen-lg">
      <h2 className="text-3xl py-3">Post Archive</h2>
      <div className="px-6 mx-auto max-w-screen-lg">
        <div className="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-6">
          {postConnectionEdges.map((edge: PostConnectionEdge) => (
            <PostExcerptCard
              title={edge.node?.title}
              date={edge.node?.date}
              featuredImage={edge.node?.featuredImage?.node?.sourceUrl}
              excerpt={edge.node?.excerpt}
              authorName={edge.node?.author?.node?.name}
              slug={edge.node?.slug}
              key={edge.node.id}
            />
          ))}
        </div>
      </div>
      <div className="my-3 flex justify-center items-center h-full ">
        <Paginator pageInfo={pageInfo} />
      </div>
    </div>
  );
}

export function ErrorBoundary() {
  const error = useRouteError();

  const errorMessage = error instanceof Error ? error.message : "Unknown error";
  return (
    <div className="mx-auto max-w-3xl px-5 py-4 my-10 bg-red-200 border-2 border-red-700 rounded break-all">
      <h1>App Error</h1>
      <pre>{errorMessage}</pre>
    </div>
  );
}

Note that this is simplified from the version I posted above. There’s no need to set hasPreviousPage=true or hasNextPage=true URL params on the links:

// app/components/Paginator.tsx

import { Link } from "@remix-run/react";
import type { RootQueryToPostConnectionPageInfo } from "~/graphql/__generated__/graphql";

interface PaginatorProps {
  pageInfo: RootQueryToPostConnectionPageInfo;
}

export default function Paginator({ pageInfo }: PaginatorProps) {
  const { hasNextPage, hasPreviousPage, startCursor, endCursor } = pageInfo;

  return (
    <div>
      {hasPreviousPage && startCursor ? (
        <Link to={`?startCursor=${startCursor}`}>Previous</Link>
      ) : (
        ""
      )}
      {hasNextPage && endCursor ? (
        <Link to={`?endCursor=${endCursor}`}>Next</Link>
      ) : (
        ""
      )}
    </div>
  );
}

It took five days, but I’m glad I stuck with it. All the previous versions worked, but felt off in one way or another.

$ git add .
$ git commit -m "it works!"
$ git checkout main
$ git merge cursor_based_pagination

I cleaned up the styles a bit and pushed the changes to the live site: https://hello.zalgorithm.com/blog/archive.

Next on the list is to display “featured posts” instead of “latest posts” in the top section of https://hello.zalgorithm.com/blog. Then add archive pages for each of the WordPress site’s categories. I think that can be accomplished by adding an optional $category segment to the blog.archive route (blog.($category).archive.tsx): https://remix.run/docs/en/main/file-conventions/routes#optional-segments.

Cursor based pagination (take two)

Get cursor data and post details in a single query

Pull in data with the ARCHIVE_QUERY

Pull in data with the `ARCHIVE_QUERY`