Skip to content
little big braindocs

Search by meaning

View Markdown

A SPARQL query finds what you can name: a label, a field value, a path. Search by meaning finds what you can only describe. An example is “which service decides if a card payment is fraudulent”.

You declare an embedding on a type of record. The platform builds one text per record from the fields you choose. It embeds the text with the graph’s model.

The platform keeps the vectors current as the graph changes. You run no embedding model and no vector store.

Search by meaning is on every plan, Free included. On Free, the vectors count toward the 30 GiB allowance.

A search embeds your question with the same model and finds the nearest records. It covers every searchable type, or the types you pick. Each hit is checked against one published commit of the graph. A deleted or re-typed record does not come back.

The running example is a service catalog. Each service is a record of the type Service. It has a name, a description and CALLS links to other services.

In this guide, a type is called a class, as in the API. A class has a global name, an IRI.

For a type you declare in JSON, the IRI is the name in lowercase after https://littlebigbrain.com/class/. So the Service class is https://littlebigbrain.com/class/service. Core concepts explains these names.

  1. Preview the text. A preview stores nothing and calls no model. It resolves the fields and shows the exact text of some records.

    Without from, the server picks the fields from a sample of the class. It takes the label, frequent text values and the labels of linked records. It leaves out ids, keys and URLs, such as source_id and source_url. It also leaves out a value that is the same on every record. Name such a field in from to embed it.

    candidates lists each fact of the class, chosen or not. It shows the share of records that have the fact, and example values.

    Terminal window
    curl -X POST "$LBB_URL/v1/embeddings/preview" \
      -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
      --data '{"class": "https://littlebigbrain.com/class/service",
               "from": ["label", "description", "calls/label"], "sample": 3}'

    The text of one record is one line per field:

    service: risk-service
    description: Scores card payments for fraud risk before the charge is captured.
    calls: payment-service, stripe-api

  2. Declare the embedding. The declaration stores the recipe: the class, the fields and the model. Then it starts the embed job. The name defaults to the last part of the class IRI, service.

    A second declaration with the same recipe does nothing. On an existing embedding, a declaration changes only what it names. Without from, the fields stay. Without model, the model stays.

    Terminal window
    curl -X PUT "$LBB_URL/v1/embeddings" \
      -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
      --data '{"class": "https://littlebigbrain.com/class/service",
               "from": ["label", "description", "calls/label"]}'

  3. Wait until it serves. The first version builds in the background. While it builds, the status shows a building version with its progress. Then it shows a serving version. That version reports its entries, the commit it covers (embedded_through_seq) and lag_commits.

    A search before the first version serves answers 409 embedding_not_ready with a Retry-After.

    Terminal window
    curl "$LBB_URL/v1/embeddings?name=service" -H "Authorization: Bearer $LBB_API_KEY"

  4. Search. The search covers every searchable class of the graph. Each hit names its class. Here, filter narrows the search to the services (see Filters). include: ["text"] returns the embedded text of each hit.

    Terminal window
    curl -X POST "$LBB_URL/v1/search" \
      -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
      --data '{"text": "card payments", "top_k": 5, "include": ["text"],
               "filter": [{"class": "https://littlebigbrain.com/class/service"}]}'

What a response tells you

Section titled “What a response tells you”

Each hit has the record’s iri, label, score and class. The score is the cosine similarity. The hit also has its text when you ask for it. The response also states how current it is:

Field Meaning
embeddings One entry per searched class: its version, model, watermark, lag, a building_version while a change of fields or model builds, and its hits in this response.
not_ready Classes in scope still building their first version; the search skipped them.
served_at_seq The published commit every hit was checked against.
head_seq The newest commit of the graph when the search ran.
embedded_through_seq The oldest commit the searched vectors cover.
lag_commits The most commits one searched class is behind head_seq. At 0, the search found every commit. The publication and then the embed job catch up by themselves.
timings Milliseconds per phase: embed_ms (the model call for the question), index_ms, rerank_ms (the exact vectors of the best candidates), check_ms, relevance_ms (the rerank model), and others.
rerank With a rerank: its status, model and the hits it read.

A repeated question is served from a cache and calls no model.

Search your own write

Section titled “Search your own write”

A commit becomes searchable in two steps. First it is published. Then the embed job embeds the changed records. This usually takes a few seconds. GET /v1/embeddings shows both steps:

Field Meaning
head_seq The newest commit of the graph.
published_seq The newest published commit.
serving.embedded_through_seq The newest commit the vectors cover.
lag_commits head_seq − serving.embedded_through_seq. At 0, a search finds every commit.

To search right after a write, pass the write’s commit_seq as min_commit_seq. The search waits up to 3 seconds for the vectors to cover that commit. If they do not, it answers 429 read_your_writes_pending with a Retry-After. Retry with the same value. A class still building its first version does not count.

Terminal window
curl -X POST "$LBB_URL/v1/search" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"text": "card payments", "min_commit_seq": 128}'

When the search answers, every hit was checked at that commit or a later one.

Rerank the best hits

Section titled “Rerank the best hits”

Similarity finds the records near your words. It does not read them. A record can be near the words and still miss the question.

The rerank reads the best hits with your words. It puts first the hits that answer them. The rerank model is Jev, a decision model from TypeSafe.

On BEIR SciFact, a public set of science claims, the rerank lifted nDCG@10 from 0.713 to 0.797 over 300 test queries. It adds about 0.3 s to a search.

Turn it on for a graph. Every search of the graph then reranks: the console, the API and your agents. In the console, the Rerank switch sits beside each search box.

Terminal window
curl -X PUT "$LBB_URL/v1/search/settings" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"rerank": true}'

One search can also decide for itself. rerank: true reranks it, and rerank: false keeps the similarity order.

Terminal window
curl -X POST "$LBB_URL/v1/search" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"text": "which service decides if a card payment is fraudulent",
           "top_k": 5, "rerank": true}'
  • The rerank reads the best 2 × top_k hits, at least 20 and at most 40, or the graph’s rerank_depth (20 to 80). It reads only hits that passed the graph check and your filters.
  • Each hit gets relevance: the probability that it answers your words. score stays the similarity.
  • The response’s rerank says how it went: applied, failed, unavailable or limited. It also names the model and the hits it read.
  • A rerank never fails a search. When the model does not answer within 2.5 s, the hits keep the similarity order.
  • A repeated search reuses the answers and calls no model.
  • GET /v1/search/settings shows the graph’s settings and the model.
  • Three more settings tune the order. rerank_depth (20 to 80) is how many hits the model reads. blend (0 to 1) mixes the two orders: blend × relevance + (1 − blend) × similarity; 1 is the rerank order and 0 the similarity order. probe_factor (1 to 4) widens the vector search over large indexes.
  • A PUT changes only the fields you send. null sets a field back to its default: {"blend": null}.
  • A tuning session (POST /v1/search/tuning) tries settings on the graph’s own searches and proposes the best. See Check model quality.
  • A SPARQL query reranks only when it says so: see Rerank inside a query.
  • Your app may show fewer hits than it gets, for example only hits with a relevance of 0.3 or more. Send that cut with the user’s words: "request": "triple stores", "shown": {"min_score": 0.3, "min_relevance": 0.3}. The eval trace then keeps each hit’s relevance and whether your app showed it, and a test of the search checks only the hits your app shows. The cut changes no hit.

Filters

Section titled “Filters”

filter is one list of conditions. Each hit must meet all of them. The graph checks each hit, so a hit that fails a condition does not come back. A change counts as soon as it is published.

Terminal window
curl -X POST "$LBB_URL/v1/search" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"text": "fraud checks",
           "filter": [{"class": "https://littlebigbrain.com/class/service"},
                      {"via": "calls", "to": "payment-service"}]}'
  • A class condition, {"class": …}, takes one IRI or a list. It keeps the records of those classes and of their subclasses (rdfs:subClassOf). A search takes one class condition. List several classes in it.
  • An embedding on a class covers its subclasses. A condition on a subclass reads the embedding of the class above it. It keeps only the subclass’s records.
  • A relationship condition, {"via": …, "to": …}, keeps the hits linked to a record. via names the link, and to names the record.
  • to is an IRI or a name. A name matches the labels of the records the link reaches. A small spelling slip still resolves when one record is clearly closest.
  • direction: "in" reverses the link. {"via": "impacted", "direction": "in", "to": "checkout outage"} keeps what the outage impacted. Several conditions must all hold. A list in to accepts any of its records.
  • The response’s filter shows each condition as resolved, in order, and how each name matched. explain: true shows the plan without running the search.
  • An unknown link or name answers 400 with the options. An agent that plans the search can then correct itself.
  • When the class tree changes, the embed job rebuilds the embedding. It reuses the vectors of texts that did not change.

Search inside a SPARQL query

Section titled “Search inside a SPARQL query”

A search can be one pattern of a SPARQL query. Then any pattern can filter the hits: a link, a value, a date, a path or a type. The rest of the query joins, groups and orders the hits like other rows.

The search predicates have the prefix search:, for https://littlebigbrain.com/search#. The syntax is standard SPARQL, so any SPARQL parser and client reads the query.

The meaning of these predicates is an extension of this engine. In the SPARQL standard, a pattern matches the facts stored in the graph. Another SPARQL engine runs the query and returns no rows. Other engines extend SPARQL in the same way, for example Jena’s text:query.

This query finds the services nearest to “card payments” among those that call payment-service:

PREFIX search: <https://littlebigbrain.com/search#>
PREFIX dclass: <https://littlebigbrain.com/class/>
PREFIX dr: <https://littlebigbrain.com/r/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>


SELECT ?service ?score ?name WHERE {
  ?service search:similarTo "card payments" ;
           search:score ?score .
  ?service a dclass:service ;
           rdfs:label ?name ;
           dr:calls ?callee .
  ?callee rdfs:label "payment-service" .
}
ORDER BY DESC(?score)
LIMIT 5

Send it to POST /v1/query/sparql-text or to the /sparql endpoint.

Pattern What it does
?x search:similarTo "text" Finds the records nearest to the text.
?x search:similarTo <iri> Finds the records nearest to that record. It calls no model.
?x search:similarTo "[0.1, …]"^^search:vector Finds the records nearest to your vector. It must have the model’s dimension.
?x search:score ?s Binds the score.
?x search:top 25 Sets the number of hits, 1 to 1,000. The default is the query’s LIMIT, then 10.
?x search:embedding "service" Searches one embedding only.
?x search:rerank true Lets the rerank model keep the hits that answer the text.
?x search:relevance ?r Binds the rerank model’s answer, 0 to 1.

Put the search in the main group of the query. A search inside OPTIONAL, UNION, MINUS or a subquery answers 400. The message names the place.

FILTER(?score > 0.8) keeps only close hits. The engine applies it after the search, so it can return fewer rows than the limit.

How the engine runs it

Section titled “How the engine runs it”

The engine picks a plan from the rest of the query. It reads the graph at the query’s commit.

  • Nearest. No other pattern uses ?x. This is the search of POST /v1/search, with the same scores.
  • Filter first. The engine reads the records that match the rest of the query. When at most 20,000 match, it scores only those records. A narrow filter keeps its recall.
  • Search first. The engine reads the nearest part of the index and keeps the records that match. It picks this plan for a broad filter: more matches than records in that part of the index. With more than 20,000 matches, it reads more of the index until it has enough hits, for at most 4 rounds.

The engine counts both costs from the index metadata before it reads any vectors. The response reports them in search.estimate.

A filter-first answer scores every match, so it misses none. A search-first answer has the recall of a search without a filter.

The response of POST /v1/query/sparql-text has a search object. It names the plan and its counts. It also gives the time of each phase. complete: false means a search-first plan found fewer hits than asked.

as_of_commit_seq reads the graph at an older commit. The filters and the rest of the query then see that commit. The vectors come from the current version, and lag_commits reports how far they trail.

Rerank inside a query

Section titled “Rerank inside a query”

search:rerank true runs the rerank inside the query. search:relevance binds the model’s answer for each hit.

PREFIX search: <https://littlebigbrain.com/search#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>


SELECT ?service ?name ?relevance WHERE {
  ?service search:similarTo "which service decides if a card payment is fraudulent" ;
           search:rerank true ;
           search:relevance ?relevance ;
           search:top 5 .
  ?service rdfs:label ?name .
}
ORDER BY DESC(?relevance)
  • The engine finds the best 2 × top hits that match the query, at least 20 and at most 40. The rerank model reads them with your text.
  • The query keeps the top hits with the highest relevance. ?score stays the similarity.
  • Order by ?relevance to show the model’s order. The rows follow the query’s ORDER BY.
  • The query decides. The graph’s rerank setting applies to POST /v1/search only. So the same query always runs the same search.
  • A rerank never fails the query. Without an answer, the query binds the most similar hits, and ?relevance stays unbound.
  • A rerank needs text. With a record or a vector as the query, it answers 400.
  • The response’s search.rerank says how it went, as on POST /v1/search.

Plan a search (for agents)

Section titled “Plan a search (for agents)”

An agent can plan a search with three reads. It needs no extra endpoint.

  1. List the searchable classes. Call GET /v1/embeddings, or MCP lbb_embeddings action=list. It gives each class and the facts of its text.

  2. Read the relationships of a class. The schema is data, so SPARQL shows it. This query reads a sample of 500 records:

    PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
    PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
    

    SELECT ?p (COUNT(DISTINCT ?s) AS ?n) (SAMPLE(?l) AS ?example) WHERE {
      { SELECT ?s WHERE { ?s a <CLASS> } LIMIT 500 }
      ?s ?p ?o .
      FILTER(isIRI(?o) && ?p != rdf:type)
      OPTIONAL { ?o rdfs:label ?l }
    } GROUP BY ?p ORDER BY DESC(?n)

    It lists each relationship with a count and one example name. For the links into the class, write ?o ?p ?s.

  3. Check the plan. Send the search with the filter and explain: true. The response shows the resolved relationship and entities. It also shows the allowed count and the mode. It calls no model. Then run the search.

If a name or a relationship is wrong, the error lists the options. The agent can correct the plan and try again.

One model per graph

Section titled “One model per graph”

All classes of a graph use one embedding model. One question then gets one vector, and scores compare across classes. The first embedding sets the model. The next ones take it. A declaration that names another model is refused.

PUT /v1/embeddings/model moves all classes to a new model. The SDKs call it lbb.embeddings.setModel and client.embeddings.set_model. MCP calls it lbb_embeddings_manage action=model.

Each class builds a new version while the old one serves. The graph switches to the new model when all classes are ready. A search never mixes two models.

How the vectors stay current

Section titled “How the vectors stay current”

After each published commit, the embed job reads what the commit changed. It embeds again only the records whose text changed. This includes records that link to a renamed record through a field such as calls/label. Records that leave the class are removed.

A change of fields or model builds a new version while the current one keeps serving. The new version replaces it when it covers the same commits. It reuses the vectors of texts that did not change.

The text of one record must fit the model’s input limit. For the OpenAI embedding models, the limit is 8,191 tokens. A longer text is shortened from its longest values first, so the short fields stay whole.

What it costs

Section titled “What it costs”

You pay only for storage. Vectors are stored data. They count toward your storage like the rest of your graph (see Usage & billing).

At 1,536 dimensions, two measured catalogs took 7.3 KB and 7.8 KB per record. That includes the text.

Older versions and merged runs stay stored until the graph’s index cleanup removes them. Billing lists the vectors of each environment by graph and embedding. It shows their monthly cost at your storage rate.

Model tokens are not billed. A monthly model budget per environment stops runaway use. When the budget is spent, requests that need the model answer 429 embedding_budget_exceeded. The error names the date the budget resets. After the first build, only changed texts are embedded again.

Errors

Section titled “Errors”
HTTP code Meaning
400 invalid_request The declaration or search is malformed. An unknown field name lists the candidates; an unknown body field, such as where, is refused.
400 parameter_missing A search without text.
404 not_found No embedding with that name, or none on the class.
409 embedding_not_ready The first version still builds. Retry after Retry-After.
429 embedding_budget_exceeded The monthly model budget is spent. The budget is a limit and adds no charge. Do not retry before the reset date.
429 server_busy Briefly contended. Retry after Retry-After.
503 embeddings_unavailable The server has no embedding model.

The name of each hit

Section titled “The name of each hit”

Each hit shows a name. By default it comes from the ontology.

  • skos:prefLabel names the hits when the records use it.
  • Next, a property your graph declares rdfs:subPropertyOf rdfs:label.
  • Next, rdfs:label, then schema:name, foaf:name or dcterms:title.

Name another fact with title, for example "title": "display_name".

Some labels are keys, like yc-job:94386. Then the preview suggests a fact that reads as a name. The response calls it title_suggestion.

For a new search, the console selects the suggestion in Name of each hit. For an existing search, it offers a Use button. You confirm when you save.

A new name reuses every vector, so it calls no model.

Terminal window
curl -X PUT "$LBB_URL/v1/embeddings" \
  -H "Authorization: Bearer $LBB_API_KEY" -H 'Content-Type: application/json' \
  --data '{"class": "https://littlebigbrain.com/class/jobposting",
           "title": "display_name"}'

Change or remove an embedding

Section titled “Change or remove an embedding”

Declare the new fields or model to build a new version. DELETE /v1/embeddings?name=service&confirm=service removes the embedding and its vectors. To search it again, declare it again and wait for the build. With MCP, lbb_embeddings_delete asks for the name twice.