Skip to main content
To keep your own copy of Captide’s documents and companies, load it once, then follow the changes feed. The feed lists every change to what the API shows, in order: new documents, edited metadata, new versions of files, documents removed from the API, and the same for companies. You don’t have to re-read the lists to find out what changed. Both are free.

1. Load the snapshot

Page through the snapshot with pageToken until nextPageToken is null. Load companies first, then documents.
  • Keep the cursor of the first page you load. It is the feed position your copy starts from, and it is the same on every page of a snapshot.
  • Changes made while you load come through the feed again in step 2, so nothing is missed.
To keep only some companies, pass companyIds.

2. Follow the changes

Call /changes with after set to your saved cursor, apply the changes in order, then save the response’s nextCursor and use it as after next time.
  • When to call again: poll every few minutes. Changes are published about a minute after they are made.
  • Catching up: while hasMore is true, call again straight away.
Apply each change by its type: Apply every change as an upsert or a delete, so a change you already have can come through again without harm. For example, a change made while you loaded the snapshot.

Files and IDs

A change never contains file links, because they expire.
  • New version of a file: changedFields names the link: originalFileUrl, markdownFileUrl (new Markdown, for example after OCR) or layoutJsonFileUrl. Get fresh links from GET /api/v2/documents/{id}.
  • New file for the document: the document gets a new id, changedFields includes id, and previous.id is the old one. Remove the old ID from your copy.

Example

Store the cursor together with your data, in the same transaction if you can. After a restart you then carry on from the last change you applied.

Good to know

  • What’s in the feed. A document is in the feed when /documents/list lists it. A document that gains a category or a listable file comes through as document.created; one that loses them comes through as document.deleted.
  • Future dates. A document dated in the future is in the feed and the snapshot from the moment it is added, although /documents/list only shows it from its date.
  • Company names on documents. companyName and companyTickers on documents are the company’s current values. When a company changes, you get one company.updated, not one change per document.
  • Moved documents. A document moved to another company comes through as document.updated with companyId in changedFields. With a companyIds filter, you get it for both companies.
  • Filters. types, companyIds and documentCategories narrow the feed. nextCursor moves past changes they skip, so a page can be empty while the feed goes on.
  • Webhooks or the feed? Webhooks push recently published new documents to you as they are added. The feed covers every change, including edits and removals, and you read it when you want.