X API reference
Pagination tokens, and the state you should keep instead
next_token is the opaque cursor X returns in a response’s meta block, which you send back as pagination_token to get the next page — and because a bookmark list changes underneath it, it is state for the duration of one walk and nothing longer.
The mechanics
| Piece | Where it lives | What it is |
|---|---|---|
meta.next_token | Response | Present when more results exist. Absent on the last page — that absence is how you know you are done. |
meta.previous_token | Response | Walks back toward the newest results. Rarely what you want if you are walking from the top already. |
pagination_token | Request | Where you put the token you were given. A base32hex string; opaque, and not to be parsed or constructed. |
meta.result_count | Response | How many resources came back — which is also how many you were billed for. |
GET /2/users/:id/bookmarks?max_results=5 → data[0..4], meta.next_token = "7140dibdnow9c7btw482..." GET /2/users/:id/bookmarks?max_results=5&pagination_token=7140dibdnow9c7btw482... → data[0..4], meta.next_token = "7140dibdnow9c7btw483..." # …until a response arrives with no next_token, or until you recognise an ID.
Why a stored token is a bug
A cursor describes a position in a list. The bookmarks list is mutable: every post the user saves pushes everything down by one, and every unbookmark pulls everything up. The list is also ordered by when each post was bookmarked, not by post ID, so the order itself can change in ways an ID-based cursor would not.
Resuming from yesterday’s token therefore resumes into a list that has shifted underneath the cursor. The failure mode is not an error — you get results, they look plausible, and you have skipped or double-counted an unknowable number of posts. That is the worst kind of bug in a paid API: it is silent, and every duplicate you fetch is charged.
This works because the list is ordered newest-bookmarked-first, which is also what makes a single-result probe a complete answer to “is there anything new?”. The full contract, including the ordering guarantee it depends on, is in the endpoint contract.
Paginating is free; the pages are not
Requests are not billed. Resources are. Within the per-user limit of 180 requests every 15 minutes there is no cost to making more calls, so the only thing page size buys you is fewer round trips — and the only thing it costs you is over-reading past the point where you recognised an ID.
Write it down and the trade-off is exact. Walking pages of p to collect k new posts bills (floor(k/p) + 1) × p resources, so the waste is p − (k mod p): bounded by page size, and independent of how many posts are waiting.
| New posts waiting | Pages of 100 | Pages of 5 |
|---|---|---|
| 3 | 100 read · $0.500 | 5 read · $0.025 |
| 7 | 100 read · $0.500 | 10 read · $0.050 |
| 48 | 100 read · $0.500 | 50 read · $0.250 |
| 230 | 300 read · $1.500 | 235 read · $1.175 |
The one thing small pages cost is request count on a large catch-up, and that is a real limit rather than a theoretical one: a fixed page of 5 against a 2,400-post backlog needs nearly 500 requests, which exceeds the 15-minute window and starts returning 429s. A schedule that holds small and then grows — say twelve pages of 5, then doubling to a cap of 100 — covers the everyday case at minimum waste and still reaches 410 posts inside 18 requests.
What people get wrong
Parsing the token. It is base32hex and it is opaque. It encodes nothing you are entitled to rely on, and constructing one is not a supported operation.
Assuming pagination costs money. It is the single most common reason people reach for max_results=100. Under per-resource billing that instinct is backwards, and the pricing model is what makes it backwards.
Paginating to the end of history on every run. There is no reason to walk past the first ID you recognise: post content is immutable, so anything below that point is already stored, and re-reading it is charged again the following day even if a same-day repeat would have been free.
We built the thing this describes
Sift reads your bookmarks once, sorts them into categories, and emails you what you saved. The rules on this page are the reason it costs what it costs — a one-result probe instead of a full page, one paid pass per UTC day, and never an expansion parameter.
Connect X