Load RDF data
Most RDF already exists somewhere: a dump from another triple store, a public dataset, an export from a pipeline that emits triples. This guide takes that data and makes it queryable. Two loading paths exist. The right one depends on how much data you have and where it comes from.
| Your data | Use |
|---|---|
| An RDF document or stream (any size, over HTTP) | POST /v1/graph/import/rdf |
| Rows and records you shape yourself | NDJSON entity/edge import |
Both write the same graph. A corpus loaded one way can later be written another way.
Import RDF over HTTP
Section titled “Import RDF over HTTP”POST /v1/graph/import/rdf accepts N-Triples, Turtle, N-Quads, and
TriG. Pick the format with the matching Content-Type, or override it with
?format=ntriples|turtle|nquads|trig. The first import into a graph that does
not exist yet creates it.
BASE=https://0abc1def--production.db.eu.littlebigbrain.com
curl -sS -X POST "$BASE/v1/graph/import/rdf?graph=research" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H 'Content-Type: application/n-triples' \ --data-binary @dataset.ntTurtle and TriG documents with relative IRIs need a base:
curl -sS -X POST "$BASE/v1/graph/import/rdf?graph=research&base_iri=https://example.org/data/" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H 'Content-Type: text/turtle' \ --data-binary @dataset.ttlThe SDKs wrap the same route:
import { LbbClient } from "@littlebigbrain/client";
const lbb = new LbbClient({ baseUrl: "https://0abc1def--production.db.eu.littlebigbrain.com", apiKey: process.env.LBB_API_KEY,});
const graph = lbb.graph("research");const res = await graph.facts.importRdf(turtleText, { format: "turtle", baseIri: "https://example.org/data/", idempotencyKey: "research-load-v1",});console.log(res.triples_read, res.committed_commit_seq);import osfrom lbb import LbbClient
lbb = LbbClient( "https://0abc1def--production.db.eu.littlebigbrain.com", api_key=os.environ["LBB_API_KEY"],)
graph = lbb.graph("research")res = graph.facts.import_rdf( turtle_text, format="turtle", base_iri="https://example.org/data/", idempotency_key="research-load-v1",)print(res["triples_read"], res["committed_commit_seq"])What the importer stores
Section titled “What the importer stores”Every statement is stored on one fixed RDF_TRIPLE relation. Resource-object
triples connect keyed Resource entities; literal-object triples connect the
subject Resource to a deterministic RdfLiteral placeholder. The source
predicate IRI, the named-graph label, and each literal’s lexical form, datatype,
and language tag are kept as edge metadata, and the RDF/SPARQL projection expands
them back to the original RDF terms on read.
The fixed schema keeps a million-predicate corpus from becoming a million
ontology relations. Your SPARQL queries still use the source predicate
IRIs. The response returns the predicate map it recorded (uri, the internal
relation, lbb_predicate_iri, lbb_relation_iri) along with statement counts
and duplicate counts.
Useful query parameters:
?strict=trueaborts on the first parse error. The default skips the bad line and reports it inerrors.?graph_uri=https://…places a Turtle or N-Triples document into one named graph. N-Quads and TriG carry their own graph labels, so combining the two is rejected.?resource_type=renames the generated resource type (defaultResource).?edge_idempotency=skip_unchanged(the default) makes an import re-runnable.appendis faster for a one-shot load, where theIdempotency-Keyheader protects request retries.
Retain mutation_receipt_id and committed_commit_seq from the response. A
repeated request with the same Idempotency-Key returns idempotent_replay: true, the same receipt, and the original commit sequence, and graph head does
not advance.
Chunking a large stream
Section titled “Chunking a large stream”Two limits apply to a large load.
One commit per request. ?batch= sets how many triples go into each internal
commit, and it defaults to the 1,000,000-triple cap. A fully buffered request
therefore commits once. Pass a smaller ?batch= only when you want the load
committed in smaller pieces.
64 MiB per request. Requests above the import cap are rejected, so a large corpus is sent as several requests. N-Triples and N-Quads are line-oriented, so split them on line boundaries. Turtle and TriG are document grammars, so every chunk must be a complete parseable document.
Defer eager reconciliation until the last chunk. Ordinary writes immediately
advance one coalesced desired publication fence, so developers do not coordinate
index jobs. For an explicit large stream, send ?build=false on every chunk
except the last. If the bounded exact-query delta suffix crosses its soft
watermark, the server still starts one coalesced safety reconciliation so the
stream cannot wedge at the hard limit:
# every chunk but the lastcurl -sS -X POST "$BASE/v1/graph/import/rdf?graph=research&build=false" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H 'Content-Type: application/n-triples' \ -H "Idempotency-Key: research-chunk-0007" \ --data-binary @chunk-0007.nt
# the final chunk publishescurl -sS -X POST "$BASE/v1/graph/import/rdf?graph=research" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H 'Content-Type: application/n-triples' \ -H "Idempotency-Key: research-chunk-0008" \ --data-binary @chunk-0008.ntbuild is a query parameter on the HTTP route. SDK callers should use
facts.importRdfMany(...) / facts.import_rdf_many(...); the helper defers
every intermediate document, triggers one final publication, and returns its
final sequence.
NDJSON import for rows and records
Section titled “NDJSON import for rows and records”When the source is rows rather than triples, shape each record yourself and send newline-delimited JSON: one triplet or one entity-properties object per line. This path gives you named types, typed scalar properties, external keys, and per-entity control. Use it when you want a modeled schema.
curl -sS -X POST "$BASE/v1/graph/import-jobs?graph=research" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H "Idempotency-Key: rows-load-v1" \ -H 'Content-Type: application/x-ndjson' \ --data-binary @records.ndjsonPOST /v1/graph/import-jobs streams the body to object storage and returns a job
id: poll it with GET, cancel it with DELETE. Workers checkpoint bounded
grouped commits and enqueue one publication at the end. POST /v1/graph/import
is a synchronous route that runs the commit loop inside the request.
Unknown entity types are rejected at commit, so register your types, relations,
and typed property fields with POST /v1/ontology/evolve before the first
record. See Bulk NDJSON ingest for the
record shapes, external-key identity, and the merge-by-default property
semantics.
Wait for publication
Section titled “Wait for publication”Writes are durable the moment they commit, and queries read a published
generation, so a fresh load becomes queryable once maintenance publishes a
root covering it. GET /v1/graph/publication-status is safe to poll even before
the first root:
curl -sS "$BASE/v1/graph/publication-status?graph=research" \ -H "Authorization: Bearer $LBB_API_KEY"The response reports current, queued, planning, building, verifying,
publishing, or blocked, with head, target, and published sequences, lag,
current stage, last progress time, and retry guidance. Poll until state is
current and published_seq covers the committed_commit_seq your import
returned. SDK callers use graph.waitForPublished(...) /
graph.wait_for_published(...); each owns one explicit deadline and should be
called once after the final batch. The old index-lineage waiter remains a
deprecated alias.
Run your first query
Section titled “Run your first query”Query with the source predicate IRIs, since that is what the projection serves back:
curl -sS -X POST "$BASE/v1/query/sparql-text?graph=research" \ -H "Authorization: Bearer $LBB_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query":"SELECT ?s ?name WHERE { ?s <http://xmlns.com/foaf/0.1/name> ?name } LIMIT 10"}'const { vars, rows } = await lbb.sparqlRows({ query: `SELECT ?s ?name WHERE { ?s <http://xmlns.com/foaf/0.1/name> ?name } LIMIT 10`,});for (const row of rows) console.log(row.s, row.name);results = lbb.sparql(""" SELECT ?s ?name WHERE { ?s <http://xmlns.com/foaf/0.1/name> ?name } LIMIT 10""")for row in results: print(row["s"], row["name"])External SPARQL tools connect to the native /sparql endpoint instead. See
SPARQL, structured query & SHACL.
Why this shape
Section titled “Why this shape”- Facts are append-only. Re-running an import with the same idempotency key
is a no-op, and new evidence on an existing edge is recorded alongside the old
one. The same events form the temporal record: read
historyto see how a relationship changed. - Indexes are derived. The graph is the source of truth and the index families are rebuildable, so a tokenizer or model change needs only a rebuild.
- Consistency is explicit.
eventualserves one published watermark;strongrequires a published generation that already covers head.
Automatic published generations
Section titled “Automatic published generations”Every successful write advances durable maintenance demand. Dedicated workers
build the leveled RDF, RDF summary, and conformance artifacts off-path, validate
the complete set, and atomically publish one immutable root. Clients do not
start index builds, delta passes, or full-index jobs. Set
LBB_PUBLICATION_FAMILY_ROSTER=all only to opt into the dormant Base, BM25, and
ANN families.
Eventual reads pin the root’s common served_at_seq, so unpublished commits
change freshness lag rather than request cost. Strong reads return
strong_read_pending until the published generation covers head.
Maintenance compacts immutable sorted runs under bounded fanout. Readers never merge the live WAL, decode an unbounded L0 chain, or fall back to a different family watermark.
- SPARQL, structured query & SHACL: the query surfaces over the data you loaded.
- Reasoning & inference: answer with facts you never wrote.
- HTTP API & SPARQL protocol: the raw routes and parameters.
Full-text and vector search are not documented yet.