Both are free.
1. Load the snapshot
Page through the snapshot withpageToken until nextPageToken is null. Load companies first, then documents.
- Keep the
cursorof the first page you load. It is the feed position your copy starts from, and it is the same on every page of a snapshot. - Changes made while you load come through the feed again in step 2, so nothing is missed.
companyIds.
2. Follow the changes
Call/changes with after set to your saved cursor, apply the changes in order, then save the response’s nextCursor and use it as after next time.
- When to call again: poll every few minutes. Changes are published about a minute after they are made.
- Catching up: while
hasMoreistrue, call again straight away.
type:
Apply every change as an upsert or a delete, so a change you already have can come through again without harm. For example, a change made while you loaded the snapshot.
Files and IDs
A change never contains file links, because they expire.- New version of a file:
changedFieldsnames the link:originalFileUrl,markdownFileUrl(new Markdown, for example after OCR) orlayoutJsonFileUrl. Get fresh links fromGET /api/v2/documents/{id}. - New file for the document: the document gets a new
id,changedFieldsincludesid, andprevious.idis the old one. Remove the old ID from your copy.
Example
Good to know
- What’s in the feed. A document is in the feed when
/documents/listlists it. A document that gains a category or a listable file comes through asdocument.created; one that loses them comes through asdocument.deleted. - Future dates. A document dated in the future is in the feed and the snapshot from the moment it is added, although
/documents/listonly shows it from its date. - Company names on documents.
companyNameandcompanyTickerson documents are the company’s current values. When a company changes, you get onecompany.updated, not one change per document. - Moved documents. A document moved to another company comes through as
document.updatedwithcompanyIdinchangedFields. With acompanyIdsfilter, you get it for both companies. - Filters.
types,companyIdsanddocumentCategoriesnarrow the feed.nextCursormoves past changes they skip, so a page can be empty while the feed goes on. - Webhooks or the feed? Webhooks push recently published new documents to you as they are added. The feed covers every change, including edits and removals, and you read it when you want.