Configuration - IMP Docs

The ExuluContext constructor takes a single options object. This page documents every option, verified against the current @exulu/backend source.

import { ExuluContext } from "@exulu/backend";

const context = new ExuluContext({
  id,              // string — required
  name,            // string — required
  description,     // string — required
  active,          // boolean — required
  fields,          // ExuluContextFieldDefinition[] — required
  sources,         // ExuluContextSource[] — required (pass [] if none)
  embedder,        // ExuluContextEmbedder
  chunker,         // ChunkerOperation
  processor,       // ExuluContextProcessor
  queryRewriter,   // (query: string) => Promise<string>
  resultReranker,  // (results) => Promise<results>
  configuration,   // retrieval configuration
  entities,        // ExuluEntitiesConfig
});

Identity

string

required

Unique identifier. Lowercased with spaces replaced by underscores, it becomes the Postgres table prefix: <id>_items, <id>_chunks, and (with the entity layer) <id>_entities. Must start with a letter or underscore, contain only letters, digits, and underscores, and be at most 80 characters — treat 5 as the practical minimum.

name

string

required

Human-readable name shown in the platform UI and logs.

description

string

required

What this context contains. Surfaced to users and agents choosing where to search.

active

boolean

required

Whether the context is active and available.

Never change id after the context has data — the tables are named after it, so a renamed context points at new, empty tables.

Fields

fields

ExuluContextFieldDefinition[]

required

The custom schema for items, on top of the built-in columns (id, name, description, tags, archived, external_id, created_by, ttl, rights_mode, embeddings_updated_at, last_processed_at, textlength, source, chunks_count, createdAt, updatedAt).

type ExuluContextFieldDefinition = {
  name: string;
  type: ExuluFieldTypes;
  editable?: boolean;
  unique?: boolean;
  required?: boolean;
  default?: any;
  calculated?: boolean;
  index?: boolean;
  enumValues?: string[];
  allowedFileTypes?: allFileTypes[];
};

fields[].name

string

required

Column name. Sanitized to lowercase with underscores. For file fields, the actual column is <name>_s3key.

fields[].type

ExuluFieldTypes

required

One of text, longText, shortText, number, boolean, code, json, enum, markdown, file, date, uuid. See the type table below for the Postgres column each maps to.

fields[].editable

boolean

Whether the field can be edited in the platform UI after creation.

fields[].unique

boolean

Adds a unique constraint on the column.

fields[].required

boolean

Whether a value is required when creating items through the platform API.

fields[].default

any

Default value declared for the field. Surfaced through the context’s table definition (API and admin UI); the Postgres column itself is created without a database-level default.

fields[].calculated

boolean

Marks the field as computed (for example, filled by a processor) rather than user-provided.

fields[].index

boolean

Whether to index the field for faster filtering.

fields[].enumValues

string[]

For enum fields: the allowed values.

fields[].allowedFileTypes

allFileTypes[]

For file fields: the accepted file types (for example ["pdf", "docx"]).

fields: [
  { name: "content", type: "longText", required: true },
  { name: "category", type: "enum", enumValues: ["guide", "reference"], index: true },
  { name: "priority", type: "number", default: 0 },
  { name: "metadata", type: "json" },
  { name: "document", type: "file", allowedFileTypes: ["pdf", "docx"] },
  { name: "published_at", type: "date" },
]

Field type reference

Type	Postgres column
`text`, `longText`, `code`, `markdown`, `enum`	`TEXT`
`shortText`	`VARCHAR(100)`
`number`	`FLOAT`
`boolean`	`BOOLEAN` (defaults to `false`)
`json`	`JSONB`
`date`	`TIMESTAMP`
`uuid`	`UUID`
`file`	`TEXT` — stored as `<name>_s3key` holding the S3 object key

Embedder

embedder

ExuluContextEmbedder

A reference to a LiteLLM embedding model, plus an optional queue. Without an embedder, vector and hybrid search are unavailable — the context still works for structured storage and full-text search.

type ExuluContextEmbedder = {
  model: string;                       // model_name from config.litellm.yaml
  queue?: Promise<ExuluQueueConfig>;   // run embedding jobs in the background
};

embedder.model

string

required

The model_name of an embedding model declared in config.litellm.yaml. The model’s model_info must declare its dimensionality — the chunks table’s vector(n) column is sized from it, and table creation throws if it is missing.

embedder.queue

Promise<ExuluQueueConfig>

When set, embedding generation is scheduled as a BullMQ job on this queue instead of running inline. Required in practice for bulk generation: without a queue, embeddings.generate.all() refuses to process more than 2,000 items.

import { ExuluContext, ExuluQueues } from "@exulu/backend";

const context = new ExuluContext({
  id: "product_docs",
  name: "Product documentation",
  description: "Guides and references.",
  active: true,
  fields: [{ name: "content", type: "longText", required: true }],
  sources: [],
  embedder: {
    model: "text-embedding-3-small",
    queue: ExuluQueues.register("embeddings", { worker: 5, queue: 5 }).use(),
  },
});

ExuluQueues.register(name, concurrency, ratelimit?, timeoutInSeconds?) registers a BullMQ queue and returns { use }; calling .use() yields the Promise<ExuluQueueConfig> the embedder, processor, and sources expect. Registering queues requires an Enterprise Edition license.

Chunker

chunker

ChunkerOperation

default:"defaultChunker"

Splits an item into embeddable chunks before the embedder runs. When omitted, the built-in defaultChunker (a sentence chunker) runs over the item’s content field — falling back to description — combined with name. Contexts with structured or file-backed content should supply their own.

type ChunkerOperation = (
  item: Item & { id: string },
  maxChunkSize: number,
  utils: { storage: ExuluStorage },
) => Promise<{
  item: Item & { id: string };
  chunks: {
    content: string;
    index: number;
    metadata?: Record<string, unknown>;
  }[];
}>;

maxChunkSize is the per-chunk token budget, derived from the embedding model. utils.storage gives file-backed chunkers access to object storage. Chunk metadata lands in the chunks table’s JSONB column, so any JSON-serializable value (for example a page number) is allowed.

const chunker: ChunkerOperation = async (item, maxChunkSize, { storage }) => {
  const sections = String(item.content ?? "").split("\n## ");
  return {
    item,
    chunks: sections.map((content, index) => ({
      content,
      index,
      metadata: { section: index },
    })),
  };
};

@exulu/backend also exports ready-made chunkers under ExuluChunkers (sentence, markdown, recursive) and the defaultChunker itself.

Sources

sources

ExuluContextSource[]

required

Data sources that ingest items from external systems. Pass an empty array if the context has none.

type ExuluContextSource = {
  id: string;
  name: string;
  description: string;
  config?: {
    schedule?: string;                  // cron expression
    queue?: Promise<ExuluQueueConfig>;
    retries?: number;                   // default 3
    backoff?: {
      type: "exponential" | "linear";
      delay: number;                    // milliseconds, default 2000 exponential
    };
    params?: {
      name: string;
      description: string;
      default?: string;
    }[];
  };
  execute: (inputs: { exuluConfig: ExuluConfig; [key: string]: any }) => Promise<Item[]>;
};

sources[].id

string

required

Unique source ID — also used as the BullMQ job scheduler key.

sources[].name

string

required

Human-readable name.

sources[].description

string

required

What the source ingests.

sources[].config.schedule

string

Cron expression, for example "0 */6 * * *" for every six hours. When the worker process starts (app.bullmq.workers.create()), it upserts a BullMQ job scheduler per source that has both a schedule and a queue. Sources without a queue log a warning and are skipped.

sources[].config.queue

Promise<ExuluQueueConfig>

The queue the source job runs on.

sources[].config.retries

number

default:"3"

BullMQ retry attempts for a failed run.

sources[].config.backoff

object

default:"{ type: \"exponential\", delay: 2000 }"

Retry backoff strategy.

sources[].config.params

object[]

Declared input parameters (name, description, optional default) for manual runs — surfaced in the platform UI when triggering the source by hand. Values arrive in execute’s inputs.

sources[].execute

function

required

Async function that fetches external data and returns an array of items (ExuluItem). Each returned item is written with createItem: items carrying an external_id (or id) are upserted — existing rows with the same external_id are updated — while items without one are inserted fresh. Processor and embedding triggers apply as configured.

import { ExuluQueues, type ExuluItem } from "@exulu/backend";

sources: [
  {
    id: "github_issues",
    name: "GitHub issues",
    description: "Syncs open issues from the tracker repository.",
    config: {
      schedule: "0 */6 * * *",
      queue: ExuluQueues.register("github_sync", { worker: 2, queue: 2 }).use(),
      retries: 3,
      backoff: { type: "exponential", delay: 2000 },
      params: [
        { name: "since", description: "Only fetch issues updated after this ISO date." },
      ],
    },
    execute: async ({ exuluConfig, since }) => {
      const issues = await fetchIssues({ since });
      return issues.map((issue): ExuluItem => ({
        external_id: String(issue.id),
        name: issue.title,
        content: issue.body,
      }));
    },
  },
]

Processor

processor

ExuluContextProcessor

Transforms items after they are written — extract text from an uploaded file, enrich a record, compute derived fields. The result is written back to the items table and can trigger embedding generation.

type ExuluContextProcessor = {
  name: string;
  description: string;
  filter?: (args: {
    item: Item;
    user?: number;
    role?: string;
    utils: { storage: ExuluStorage };
    exuluConfig: ExuluConfig;
  }) => Promise<Item | undefined | null>;
  execute: (args: {
    item: Item;
    user?: number;
    role?: string;
    utils: { storage: ExuluStorage };
    exuluConfig: ExuluConfig;
  }) => Promise<Item>;
  config?: {
    queue?: Promise<ExuluQueueConfig>;
    timeoutInSeconds?: number;          // default 600 in queue mode
    trigger: "manual" | "onUpdate" | "onInsert" | "always";
    generateEmbeddings?: boolean;
  };
};

processor.name

string

required

Processor name, used in job labels.

processor.description

string

required

What the processor does.

processor.filter

function

Optional predicate. Return a falsy value to skip processing for an item — for example, when no file is attached yet.

processor.execute

function

required

The transformation. Receives the item and utils.storage for reading files from object storage; must return the updated item, which is persisted with last_processed_at set.

processor.config.trigger

string

required

When the processor runs: manual (only explicit processField() calls), or automatically on writes. Both createItem() and updateItem() run the processor when the trigger is onInsert, onUpdate, or always — currently the three non-manual values behave identically on both write paths.

processor.config.queue

Promise<ExuluQueueConfig>

Run processing as a background job instead of inline.

processor.config.timeoutInSeconds

number

default:"600"

Job timeout in queue mode.

processor.config.generateEmbeddings

boolean

When true, embedding generation is triggered automatically after the processor finishes.

processor: {
  name: "pdf_text_extractor",
  description: "Extracts text from uploaded PDF files into the content field.",
  config: {
    trigger: "onInsert",
    generateEmbeddings: true,
    queue: ExuluQueues.register("doc_processing", { worker: 2, queue: 2 }).use(),
  },
  filter: async ({ item }) => (item.document_s3key ? item : undefined),
  execute: async ({ item, utils }) => {
    const url = await utils.storage.getPresignedUrl(item.document_s3key);
    const text = await extractPdfText(url);
    return { ...item, content: text, textlength: text.length };
  },
}

Retrieval hooks

queryRewriter

(query: string) => Promise<string>

Rewrites the natural-language query before retrieval — for example, expanding it with an LLM. Applied inside the search pipeline whenever a query is present.

queryRewriter: async (query) => {
  return await expandQueryWithSynonyms(query);
},

resultReranker

(results: chunk[]) => Promise<chunk[]>

Declared hook for reordering retrieved chunks after search. Each chunk carries chunk_content, chunk_index, chunk_id, chunk_source, chunk_metadata, chunk_created_at, chunk_updated_at, item_id, item_external_id, and item_name.

resultReranker is accepted and stored on the context, but the current search pipeline does not invoke it — the call site is disabled in the source. Don’t rely on it running until a release notes otherwise.

Retrieval configuration

configuration

object

Retrieval and write behavior. When you omit the entire object, the defaults below apply. When you pass a partial object, only your keys are set — the per-key defaults are not merged in, so pass everything you rely on.

configuration: {
  calculateVectors: "manual",
  languages: ["english"],
  defaultRightsMode: "private",
  maxRetrievalResults: 10,
  expand: { before: 0, after: 0 },
  cutoffs: { cosineDistance: 0.5, tsvector: 0.5, hybrid: 0.5 },
}

configuration.calculateVectors

"manual" | "onUpdate" | "onInsert" | "always"

default:"manual"

When embeddings are generated automatically: never (manual), when items are created (onInsert), updated (onUpdate), or both (always). With manual, call embeddings.generate.one() / .all() yourself or pass the generateEmbeddingsOverwrite flag on writes.

configuration.maxRetrievalResults

number

default:"10"

Fallback result limit used when a search() call passes no limit.

configuration.defaultRightsMode

"private" | "users" | "roles" | "teams" | "public"

default:"private"

Default rights_mode for new items — the items table column defaults to this value.

configuration.cutoffs

object

Minimum relevance scores per search method. Results scoring below the cutoff are dropped. Can be overridden per search() call.

configuration.expand

object

default:"{ before: 0, after: 0 }"

Number of neighboring chunks to include around each matched chunk, giving agents more surrounding context. Can be overridden per search() call.

configuration.languages

("german" | "english")[]

default:"[\"english\"]"

Languages for the generated full-text search columns. Each language adds a to_tsvector expression to the items and chunks tables, affecting stemming and stop words. Set before the tables are created — the columns are generated at table-creation time.

Entity layer

entities

ExuluEntitiesConfig

Opt-in entity extraction over chunks. When present (or when an admin has configured entity types for the context in the UI), IMP extracts typed entities from embedded content into <id>_entities, enabling entity filters and insights in search(). Absent → behavior is unchanged.

type ExuluEntitiesConfig = {
  types?: { name: string; description: string }[];
  model?: string;                 // extraction model; falls back to a platform default
  boostWeight?: number;           // shared-entity ranking boost, default 0.3
  confidenceThreshold?: number;   // drop mentions below this (0..1), default 0.5
  canonicalLanguage?: string;     // language for canonical names, default "english"
};

entities.types

object[]

Entity types declared in code, for example { name: "Person", description: "A natural person mentioned in the document." }. Merged (union) with types an admin declares in the UI.

entities.model

string

Model ID used for extraction. Falls back to a platform default when omitted.

entities.boostWeight

number

default:"0.3"

Weight of the shared-entity boost term in retrieval ranking.

entities.confidenceThreshold

number

default:"0.5"

Mentions below this extractor confidence are dropped.

entities.canonicalLanguage

string

default:"english"

Target language for canonical entity names.

See Entities for the admin-side view of the entity layer.

Complete example

import { ExuluContext, ExuluQueues, type ExuluItem } from "@exulu/backend";

export const productDocsContext = new ExuluContext({
  id: "product_docs",
  name: "Product documentation",
  description: "Guides, API references, and tutorials for the product.",
  active: true,

  fields: [
    { name: "content", type: "longText", required: true },
    { name: "category", type: "enum", enumValues: ["guide", "api", "tutorial"], index: true },
    { name: "url", type: "text", unique: true },
    { name: "attachment", type: "file", allowedFileTypes: ["pdf"] },
  ],

  embedder: {
    model: "text-embedding-3-small",
    queue: ExuluQueues.register("embeddings", { worker: 5, queue: 5 }).use(),
  },

  sources: [
    {
      id: "docs_repo_sync",
      name: "Docs repository sync",
      description: "Imports markdown pages from the documentation repository.",
      config: {
        schedule: "0 */4 * * *",
        queue: ExuluQueues.register("docs_sync", { worker: 1, queue: 1 }).use(),
        retries: 3,
        backoff: { type: "exponential", delay: 2000 },
      },
      execute: async ({ exuluConfig }) => {
        const pages = await fetchDocPages();
        return pages.map((page): ExuluItem => ({
          external_id: page.path,
          name: page.title,
          content: page.markdown,
          category: page.category,
          url: page.url,
        }));
      },
    },
  ],

  configuration: {
    calculateVectors: "onInsert",
    maxRetrievalResults: 15,
    defaultRightsMode: "public",
    cutoffs: { cosineDistance: 0.5, tsvector: 0.5, hybrid: 0.5 },
    expand: { before: 1, after: 2 },
    languages: ["english"],
  },

  entities: {
    types: [
      { name: "Feature", description: "A named product feature." },
      { name: "Company", description: "An organization mentioned in the docs." },
    ],
  },
});

Next steps

API reference

Search, item CRUD, embeddings, entity layer, and table methods.

​Identity

​Fields

​Field type reference

​Embedder

​Chunker

​Sources

​Processor

​Retrieval hooks

​Retrieval configuration

​Entity layer

​Complete example

​Next steps

API reference

ExuluApp

Identity

Fields

Field type reference

Embedder

Chunker

Sources

Processor

Retrieval hooks

Retrieval configuration

Entity layer

Complete example

Next steps