# Integrations (experimental) Source: https://docs.littlebigbrain.com/guides/integrations/ This opt-in feature runs on [workflows](https://docs.littlebigbrain.com/guides/workflows/). An integration keeps one source system in step with one graph. You connect a source with its credential or a developer-owned app. LBB fits the source to your ontology and runs every sync. A sync reads the source page by page and writes records under stable keys. Depending on the connector, later syncs read incremental changes or repeat bounded source passes. ## What you get - **Your classes.** Each stream of the source maps to a class of your ontology when one fits. A match is by name, by a common synonym, or by the connector's class hint. HubSpot contacts and Linear users both land in `Person`. One SPARQL query over `Person` finds both. - **[A starter ontology](https://docs.littlebigbrain.com/guides/ontology-starters/).** Each connector maps onto the starter of its domain: HubSpot onto CRM, Linear and GitHub onto Work management, SharePoint, Google Drive and Notion onto Documents. What your ontology lacks is proposed under the starter's names, such as `Deal`, `annual_revenue` and `WORKS_AT`, with the starter's parent classes and relation rules. Two CRMs land in the same terms. - **Noise set aside.** A field the source hides, computes or keeps for itself (HubSpot's `hs_` properties), a field that repeats the record's name or a link, and a field no record fills get no property. A suggestion counts them: "It sets 23 fields aside: 12 internal, 8 calculated and 3 hidden." - **Fields and references.** Each field maps to a property of the same value type. Each reference maps to a relation. - **One suggestion per class.** What does not fit becomes a suggestion: "Add class Deal", or "Add 3 properties to Organization". The starter's properties come first, then the source's own fields, the best filled first. Each suggestion shows sample records and the fill rate of the new fields. It shows the record count when the source reports one. LBB counts the fill rates itself; no record value leaves it for that. - **Data that waits.** Streams that fit sync at once. A stream without a class waits, and so does a field without a property. After you accept a suggestion, the next sync reads the waiting records in full. Each record is written once. - **New fields on synced records.** Accept a suggestion that adds fields to a stream that already synced. The next sync reads only those fields from the source. It adds them to the records it wrote and keeps their other values. Any other change to what a stream writes reads the stream again in full. - **Stable keys.** A record is keyed by connection, stream and source id. A re-read updates the same entity. Source relationship and property clearing is not fully reconciled for the new provider-app readers. - **Deletes.** Supported explicit deletion signals retract source entities. Coverage and retention bounds depend on the connector; missing records in incomplete listings do not establish deletion. See the [provider-app limits](https://docs.littlebigbrain.com/guides/developer-owned-integrations/#data-and-access-limits). - **Only changes are written.** LBB keeps a short hash of what each record wrote. A record that writes the same again is left out. A sync with no change at the source writes nothing to the graph. The changed records of one sync step go in one write. - **Crash safety.** Each step reads one page of every stream still to read, as one checkpointed workflow step. A crash repeats at most the unfinished step. On an empty graph, the first integration's suggestions are the first draft of your ontology. Accept HubSpot's, and the graph has `Organization`, `Person`, `User`, `Deal`, `Team`, `Pipeline` and `Stage`. Linear then maps its teams into `Team` and its workflow states into `Stage`. It suggests only what is new, such as `Issue`, `Project` and `Comment`. ## Connectors Notion, GitHub and Slack use your company's registered provider apps in hosted LBB. This flow is separately opt-in and sends customers through provider consent and explicit resource selection. Open **Integrations → Provider apps** to set up an app, then create customer links from your backend. See [Offer Notion, GitHub and Slack in your product](https://docs.littlebigbrain.com/guides/developer-owned-integrations/) for setup, scopes, readers and current limits. Provider approval and live app acceptance are not established by LBB's fixture tests. The following connectors use credential entry: | Connector | Streams | Starter classes | Credential | | --- | --- | --- | --- | | Linear | teams, users, projects, issues, comments, workflows and their states, labels, cycles, project milestones | Work management: Team, User, Project, Issue, Comment, Pipeline, Stage, Tag, Iteration, Milestone | A personal API key (`lin_api_…`), from Linear under Settings, Security & access | | HubSpot | companies, contacts, deals, owners and their teams, deal pipelines and stages; tickets when you turn them on | CRM: Organization, Person, Deal, User, Team, Pipeline, Stage, Ticket | A private app token with read scopes for companies, contacts, deals and owners, plus tickets for tickets, from HubSpot under Settings, Integrations, Private apps | | SharePoint | sites, document libraries, documents with text from Office and text files | Documents: Site, Library, Document, User | The tenant id, client id and a client secret of a Microsoft Entra app registration with `Sites.Selected` and read access to each site | | Google Drive | My Drive and shared drives, folders, files with text from Google Docs, Sheets, Slides, Office and text files, and who each file is shared with | Documents: Library, Folder, Document, User | An OAuth client id, client secret and refresh token with the `drive.readonly` scope, or the JSON key of a service account that the folders are shared with | A deal links to its stage, a stage to its pipeline, an issue to its state, labels, cycle and milestone, and a document to its library. A HubSpot token that may not read the deal pipelines syncs the deals without their stages. SharePoint also reads each library's columns and content types as a description of the documents. They show in the fit, and their values are not synced. LBB stores a credential sealed, bound to its environment, graph and connection. No answer from LBB shows it again, the console's included. Google Drive links each file to its folder and drive. It writes `acl` from the file's sharing. Each entry is `user:`, `group:`, `domain:` or `anyone`. A file the account may not share has no `acl`. The **Folders** setting limits a sync to some folders or shared drives. A file moved out of them is removed from the graph at the next sync. Google Docs and Slides sync as plain text. Sheets sync as CSV of the first sheet. Google exports at most 10 MB of a file. PDF and image files keep their metadata without text. SharePoint and Google Drive read the text of files up to a size limit. The limit is 20 MB by default and 50 MB at most; a higher setting counts as 50 MB. Files larger than the limit are kept without their text, with `text_status` set to `too_large`. An Office file whose contents unpack to more than four times the limit (64 MiB at most) is kept the same way. A document keeps the first 100,000 characters of its text. ## Upload a file **File upload** is the first card on **Sources → Integrations**. It works on every plan, also where the connectors are off. Choose **Upload**, then paste or choose a Turtle, N-Triples or NDJSON file. It goes into the selected graph, and a graph that does not exist yet is created. For files in code, see [Load data](https://docs.littlebigbrain.com/start/load-data/). ## Connect a source Open **Sources → Integrations** in the console and choose **Connect** on a credential-based connector. Notion, GitHub and Slack cards instead open provider app setup; their customers use the hosted consent flow. The credential dialog has three steps. 1. **Credentials.** Paste the credential. The help under each field says where to create it. Choose the connector settings (for example the HubSpot objects or the SharePoint sites) and a schedule. 2. **Ontology.** The step shows at once which starter the connector maps onto. For example: "SharePoint maps onto the Documents starter: sites, libraries, documents and people." For each starter class, choose: - **Use your class**, for example `Customer` for HubSpot companies. - **Add class**, under the starter's name or one you type. - **Skip**, to leave the stream out. The starter properties and relations show under each class. In the background, LBB checks the fit against your ontology with its models. When that check answers, the step updates in place. A match then shows its score, or "model · 88%" when the fit model picked it. You do not need to wait for it. If the check waits behind another connection, the step says which one. For example: "The fit check starts after HubSpot's fit (about 8 min)." If the check fails, for example on a rejected credential, the step shows the error with **Retry** and **Update credentials**. A check with no progress for 3 minutes shows **Retry**. 3. **Start.** You can select **Start** as soon as the credentials step is done. LBB queues the first sync and runs every scheduled one. Anything new that the fit finds arrives as suggestions. If you close the dialog instead, the connection stays. Start its first sync from its card. You can connect several sources one after another. Nothing in the dialog waits for another connection. By default, new classes and properties wait for your review in **Review**. The streams that fit sync at once. Choose **Apply the ontology now** in the Ontology step to add them before the first sync. The card of a connection shows its state, and a line on what runs now. For example: "Claude is drafting new terms (4 of 12) · about 8 min left". | State | Meaning | | --- | --- | | Queued | The next turn starts after another connection's turn. The card names it, for example "Starts after HubSpot's fit (about 8 min)". | | Starting… | LBB is picking up the next sync or check. | | Syncing | A sync runs, with its progress. | | Up to date | The last sync finished. | | Waiting for review | A stream waits for an ontology change you have not accepted. | | Needs attention | A turn failed, for example on a rejected credential. **Retry** or **Skip** it. | | Storage full | The graph refused an import because the Free plan's 30 GiB is used. Delete data or upgrade to Cloud, then **Retry** the turn. | | Delayed | LBB has not picked the turn up for five minutes. It retries by itself. | | Paused | Nothing syncs until you resume. | The card also counts what the connection wrote: - **Written**: the records written over all syncs, the running sync included. A record counts again each time it changes. - **Last sync**: when the last sync finished, and its records. While a sync runs, it shows **Running** and the records written so far. - **Streams**: the same counts for each stream. They add up to **Written**. During the first sync, the three numbers are the same. A sync that starts while another one runs joins it. It continues from where that sync is. A full resync starts again from the first record. ## Manage a connection The menu on a connection's card has: - **Sync now** and **Full resync**. A full resync reads every record again and writes it, changed or not. - **Edit schedule…**: manual, hourly, daily, or every N minutes. - **Ontology mode**: review changes first, or apply them automatically. - **Update credentials…**: paste a new credential. LBB checks it against the source. - **Edit settings…**: the connector settings you chose on connect. - **Pause** and **Resume**. - **Remove…**: type the connection id to confirm. LBB stops syncing and deletes the connection with its stored credential. The records it wrote stay in the graph. A change to the schedule, the mode, the settings or the credential applies within a minute. Open a connection to see its **Fit** tab: each stream against the class it maps to, each field against a property, and what waits for review. Change a class, a property or a relation there, and LBB checks the fit again. Query the result with SPARQL: ```sparql PREFIX p: PREFIX r: PREFIX rdfs: SELECT ?issue ?team WHERE { ?i p:priority_label "Urgent" ; r:in_team ?t ; rdfs:label ?issue . ?t rdfs:label ?team . } ``` ## Review ontology changes By default, an integration files its suggestions and syncs what already fits. Open **Review** in the console. There you read a change, discuss it, and edit it. Then you accept or dismiss it. Your edits carry over. Rename a class or a property, and the integration uses your name. Remove a property, and the integration leaves that field out. An accept queues a sync. Until it runs, the stream shows **accepted · syncs next**. A failed sync holds everything after it. Retry or skip it first. A dismissed suggestion keeps its stream waiting. The graph refuses a change that would narrow or remove what it holds. A source can only add to your ontology. ## Link the same thing across sources HubSpot contacts and Linear users both land in `Person`, so Ada can exist twice: once from each source. After a sync, the integration compares its records with those of the other connections in the same class. It matches on an email property, or on `website`, `domain` or `url`. Emails compare lowercase and trimmed. Web addresses also drop the scheme, `www.` and a trailing `/`. The matches arrive as one suggestion per class and pair of connections, for example "12 people appear in both HubSpot and Linear". In **Knowledge graph → Suggestions** you see each group of records with its sources and the value they share. **Edit, then accept** lets you leave out a group, or a record of a larger group. Accept, and the graph links each group with `SAME_AS` edges. Follow them in SPARQL to read a person's records together. The comparison runs as SPARQL inside your graph. No record value leaves it. ## Map a source yourself A connection's mapping binds streams, fields and references to terms you choose. It holds only what differs from the match: ```json { "type": "configure", "mapping": { "companies": { "class": "Organization", "fields": { "domain": "website", "hs_lead_status": null } }, "owners": { "class": null } } } ``` `"class": null` ignores a stream. A field or reference set to `null` is left out. A class that does not exist yet is suggested under the name you give. Send the mapping as a `configure` message to the connection's workflow. The Ontology step and the **Fit** tab send the same message. ## Let a model match the rest Names do not match everything. HubSpot's `domain` field means the same as a `website` property, and no rule of names says so. TypeSafe's Jev model answers what the names leave open: the class of a stream, and the property of a field. It never answers for a stream or field you mapped yourself. LBB runs the fit model and pays for it. You choose no model and bring no key. Where the environment has no TypeSafe key, fit uses names and synonyms only. The console's **Models** page lists Jev as a decision model. It shows how many decisions Jev made this month. Only schema text leaves LBB. That is the source label, the stream and field names, labels and value types, and your ontology's class and property names. Record values never leave it. An answer at the confidence threshold or above maps at once. The threshold is 0.7. The **Fit** tab shows it as "model · 87%". A lower answer shows as "Model suggests website · 62%", with a **Use** button. Each question is asked once; the answer is kept with the integration. ## Let a model propose what your ontology lacks Your own fields often have no place in the starter. A renewal date or a customer tier is one. In the hosted console, Claude Sonnet 5.5 proposes the smallest set of terms that holds them. Jev maps the fields it can to your terms. It sets internal fields aside and checks each proposed change. The fit model reads the source before the first sync. It reads it again when the source's fields change. A run takes a few minutes. Meanwhile the **Fit** tab shows the step it is on, for example "Claude is drafting new terms (4 of 12)". A fit check you start then runs after the fit model. Its changes arrive in **Review**, one per class. An example is "Extend Organization: 9 properties, 1 relation". A new class gets its own suggestion. In automatic mode, a change applies by itself only when Jev supports it well and a dry run passes. Every other change waits for your review. Accepted changes start syncing the fields they hold. The **Fit** tab shows each class's changes and where they stand. It also lists the fields the model left out, with the reason for each. A field the model left out is not synced. The models see the schema and statistics LBB computes: fill rate, distinct values and the longest value. They never see a record value. ## Run the worker yourself The self-run worker is for self-hosted servers, where LBB does not hold source credentials. You run it from a checkout of the repository, with the credentials in its environment. The console lists the connections it serves. Start a data plane with workflows enabled (see the workflows guide). Then build the worker in the checkout: ```sh npm ci npm run build -w @littlebigbrain/client -w @littlebigbrain/integrations ``` Set the variables and run it: ```sh export LBB_URL=http://127.0.0.1:7400 LBB_API_KEY=lbb_sk_test_local LBB_GRAPH=main export LINEAR_API_KEY=lin_api_... node sdk/integrations/dist/cli.js preview linear # how the source fits your ontology; writes nothing node sdk/integrations/dist/cli.js add linear-main linear --every 1h node sdk/integrations/dist/cli.js worker # runs syncs until you stop it node sdk/integrations/dist/cli.js fit linear-main # each stream against its class ``` The worker reads these variables: | Connector | Variables | | --- | --- | | Linear | `LINEAR_API_KEY` | | HubSpot | `HUBSPOT_TOKEN` | | SharePoint | `SHAREPOINT_TENANT_ID`, `SHAREPOINT_CLIENT_ID`, `SHAREPOINT_CLIENT_SECRET` | | Google Drive | `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_REFRESH_TOKEN`, or `GOOGLE_SERVICE_ACCOUNT_KEY` with an optional `GOOGLE_DELEGATED_USER` | A connection-scoped variable (`_`) wins over the plain one. One worker can then serve two accounts of the same system. Set `JEV_API_KEY` for the fit model, `LBB_FIT_MODEL=off` to turn it off, and `LBB_FIT_MODEL_THRESHOLD` to change the 0.7. The worker runs the work of up to four connections at once. A new connection's fit does not wait for another connection's sync. The steps of one connection always run in order. A connection's workflow type is `sdk.integration` (`lbb.integration` before 2026-10-13). The worker claims both names until no connection of the old name is left. Build it from a checkout of 2026-10-13 or later before your server moves its connections (`database/scripts/workflow-retype.py`): an older worker claims only the old name and stops finding work. A self-hosted server moves them, then sets `LBB_RESERVED_WORKFLOW_TYPES=all`, as in the [workflows guide](https://docs.littlebigbrain.com/guides/workflows/). `fit` prints one row per stream. A row shows the stream's records and its class with the match score, or "new class Issue". It counts the fields that map, are new, or are ignored. The last column says if the stream syncs or waits. Review suggestions from the command line: ```sh node sdk/integrations/dist/cli.js suggestions hubspot-main # this integration's suggestions node sdk/integrations/dist/cli.js suggestions accept --sync # accept one, then sync node sdk/integrations/dist/cli.js suggestions dismiss --reason "We track deals elsewhere" node sdk/integrations/dist/cli.js apply hubspot-main # accept them all, then sync node sdk/integrations/dist/cli.js resolve hubspot-main # compare records across sources now ``` To apply every change without review, add `--auto`: ```sh node sdk/integrations/dist/cli.js add hubspot-main hubspot --auto ``` ## Experimental PostgreSQL CDC controls PostgreSQL CDC uses a separate workflow and a dedicated source reader. It is still under development and is disabled by default. Authenticated discovery, mapping approval and the gated console setup flow are implemented. Its console entry points share the `integrations` feature flag with the other connectors; backend CDC capability gates still apply independently. Open **PostgreSQL CDC** in the sidebar, next to **Integrations**. Saved PostgreSQL connections also appear at the top of **Integrations**; choose **View sync** to open a source and its customer graphs. These links remain available even when the selected graph's workflow status cannot load, including before a first destination graph exists. For an active sync, PostgreSQL streams committed changes from its write-ahead log. Each captured batch wakes the graph workflow; a 30-second recovery check repairs missed notifications. Query visibility follows publication, so click **Run** again to refresh existing query results. Operators must approve the truth-format rollout before activating a source; enabling API routes alone is insufficient. Managed Models now propose customer ownership and ontology fit from schema metadata. Source discovery and the guided console work without a destination graph. A reviewed source can feed multiple isolated customer graphs; adopting existing populated destinations is not supported. A model proposal or review does not activate replication. After a SaaS host restart, unfinished proposals resume automatically if their source, ontology, Models policy and decisions are unchanged. Failed attempts need an explicit retry. Recovery does not review a draft or start replication. For a disposable local end-to-end test, a source checkout includes `database/scripts/cdc-acceptance.py`. With the local server, SDKs, SaaS API and reader image built, run: ```sh python3 database/scripts/cdc-acceptance.py --supervised \ --image lbb-cdc-reader:local \ --interactive-ready-file /tmp/lbb-cdc-playground.json ``` The harness verifies real insert/update/delete publication, then stays running. The JSON file contains PostgreSQL connection settings, a ready-to-use `psql` command, and the LBB endpoint and query. Point a development console at that endpoint, edit `public.items`, and rerun **Query** to see changes. This maps `id`, `title` and `body` into one graph; customer routing and model-assisted setup are separate. No provider keys are needed. Ctrl-C cleans up this fixture and its temporary data. The one-second recovery check is for this local test. For a bounded bulk test against that running fixture: ```sh python3 database/scripts/cdc-playground-bulk.py \ --playground /tmp/lbb-cdc-playground.json --rows 500 --batch 50 \ --report /tmp/lbb-cdc-bulk-result.json ``` This inserts 500 new rows, updates 250 and deletes 100, leaving 400 for inspection. It also checks rollback isolation. Each phase verifies the exact published values; your pre-existing rows remain untouched. Click **Run** in Query to refresh results. In an enabled hosted environment, choose **Add connection → PostgreSQL** or **Connect** on its Integrations catalog card. Setup walks through **Connect → Tables → Review → Preview**. Enter credentials and an explicit list of `schema.table` names, then **Check source**. Port and private CA settings are under **Advanced connection settings**; publication and slot names are under **Replication settings**. The publication must match the table list. Choose an existing ontology template or start from the tables, then **Suggest setup**. Review the proposed customer boundary and expandable table mappings, answer any questions, and **Save reviewed draft**. Choose a graph prefix and customer limit, then **Preview customer graphs**. Preview reads the source to check ownership without creating destination graphs. **Start syncing** approves creation of the customer graphs with the reviewed ontology; subsequent customers get a graph up to that limit. The connection list is scoped to the environment, independent of the selected graph. A saved plan remembers its ontology template. Table mappings and Model details expand when needed; field exclusions remain visible for review. If a field's type cannot be fitted, open **Field exclusions** and explicitly select any fields to leave out, then **Revise proposal**. Exact PostgreSQL `numeric` values are currently one such type. Exclusions stay visible when you reopen the draft. Removing an exclusion requires another proposal; unsent edits must be resolved before you can save a review. No values are silently converted. **Graph syncs** retains the earlier single-target foundation controls for the selected graph. Its approval/start path remains separate and release-gated. The status card follows **PostgreSQL → Graph → Queries**, showing where changes are waiting. **Pause updates** keeps capturing source changes while graph updates are paused; **Resume updates** wakes that backlog. Capture controls and retirement are in **More actions**, and counters are under **Sync details**. Source risks and active alerts stay visible. For an already approved binding on a deployment that enables the CDC integration API, use a stack API key with the TypeScript or Python SDK: ```ts const status = await client.integrations.cdcStatus("postgres", { graph: "orders" }); const requested = await client.integrations.cdcControl("postgres", { graph: "orders", operationId: "maintenance-1", action: "pause_capture", }); ``` Python uses `cdc_status` and `cdc_control`, with `operation_id`. Status separates captured, applied and published progress. Only published data is ready for queries. A source-health sample expires after 60 seconds; an unknown source is not reported as streaming simply because its last captured work was applied. Controls are `pause_capture`, `resume_capture`, `pause_apply`, `resume_apply` and `retire`. Reuse the same operation id after a lost response. A `pending` result is durable intent; status reports when it becomes `applied` or `failed`. `applied` confirms the mode change; reader health separately reports shutdown. Retirement requires `confirm` equal to the connection id, preserves captured and graph data, and never drops a PostgreSQL replication slot. An interrupted initial snapshot cannot resume through this control; it requires an explicit rebuild. On a deployment with the CDC API enabled and an explicit source sealing key, `client.integrations.cdcDiscover` creates a sealed discovery job for an existing graph. Supply `jobId`, `expectedRevision: 0`, the source host/port/database/ publication/slot, an explicit list of tables, and credentials. Keep the same job id, settings and credentials after a lost response. `cdcDiscovery` reads progress and the verified catalog; `cdcCancelDiscovery` requires the job id, graph epoch and revision from that response. Python exposes `cdc_discover`, `cdc_discovery` and `cdc_cancel_discovery` on both sync and async clients. The dedicated host connects to PostgreSQL over verified TLS. Source credentials and sealed values never appear in status. Discovery does not activate capture; it prepares the schema for mapping review. CDC connection and job ids accept letters, digits, underscores and hyphens. Developer creation limits apply, while an exact retry of an existing job spends no additional creation. The connection view separates captured, applied and published progress from source health. Fresh source checks show slot activity and retained/unconfirmed WAL bytes; these are byte distances, not elapsed lag or available disk space. Unknown values remain unknown. Per-table initial-snapshot counts describe rows durably captured and may be unavailable after reader restore. Paused and retired readers stop probing PostgreSQL, so continue monitoring the source slot directly. The optional reader resource view shows sampled process CPU time, container current/peak memory and JVM heap use. Unknown measurements remain unknown; lifetime counters reset when the reader restarts. These operating samples are not billing quantities. The connector uses existing storage and completed-work metering and does not add row pricing. The connection view also shows active capacity, source/WAL and application-stall conditions plus the latest 32 alert/recovery notices. Notices appear once per transition and survive host restart. Muting suppresses new notices while keeping active conditions visible; unmuting does not replay missed notices. A stale evaluation is explicitly marked, and missing source evidence is not recovery. External notification channels are not part of this experimental release. ```ts await client.integrations.cdcMuteAlerts("postgres", { graph: "orders", expectedRevision: status.alerts!.settings_revision, muted: true, }); ``` After a lost response, retry the same body. Python uses `cdc_mute_alerts` with `expected_revision` on both sync and async clients. A settings conflict requires reading the current revision before choosing a new preference. ## Limits - Polling connectors use the property-graph write model. A graph first written with RDF refuses the import route. The sync then fails with that reason. - A many-valued reference (HubSpot associations, project teams) only adds edges. An association removed at the source stays in the graph. - Properties are graph-wide. A field name can exist with another value type. The field is then suggested with the class as a prefix, as in `deal_amount`. - A suggestion holds at most 64 changes. A class that needs more gets the rest in its next suggestion, the least filled fields last.