NEBU /ˈnɛ.buː/
ROOT / DOCS / MAN / NEBU(1)
NEBU(1) NEBU(1)

NAME

nebu — a toolkit for building Stellar indexers. Origin/transform/sink processors, composed with Unix pipes.

SYNOPSIS

nebu [OPTIONS] COMMAND [ARGS]...
nebu list
nebu install <processor>
nebu fetch [--mode rpc|archive] <start> <end>
<processor> --start-ledger N [--end-ledger N]

DESCRIPTION

nebu packages Stellar's modern indexing primitives — RPC-backed ledger access, the ingest SDK, XDR-native extraction — into a stable Go contract, standalone processor binaries, and Unix-composable pipelines.

The nebu binary is intentionally minimal. It discovers processors via a registry.yaml, installs them with go install, and can fetch raw ledger XDR. Actual event extraction happens in standalone processor binaries that speak newline-delimited JSON over stdout.

Pipelines are composed with the shell's own pipe. There is no daemon, no orchestrator, no YAML pipeline file.

COMMANDS

nebu list

Show all processors known to the embedded registry and the external processor registry. Lists name, type, location, and description.

$ nebu list
nebu install <processor>

Resolve the processor in the registry and install its binary to $GOPATH/bin via go install. Works for in-repo educational examples and external processors whose Go module or prebuilt release installs cleanly.

$ nebu install token-transfer
nebu fetch <start> <end>

Fetch raw LedgerCloseMeta XDR for a ledger range and write it to stdout. RPC and archive modes pass the backend's XDR bytes directly through without decoding and re-encoding each ledger. The output remains compatible with existing processors and saved .xdr files.

Supports two modes:

  • --mode rpc · (default) stream from a Stellar RPC endpoint
  • --mode archive · read from a GCS/S3 ledger archive (e.g. the public aws-public-blockchain bucket)
$ nebu fetch 60200000 60200100 > ledgers.xdr
$ cat ledgers.xdr | token-transfer | jq
<processor> --start-ledger N [--end-ledger N]

Run an installed processor. Origins read ledgers (from RPC or stdin) and emit typed events as NDJSON on stdout. Transforms and sinks read NDJSON from stdin.

$ token-transfer --start-ledger 60200000 --end-ledger 60200001
$ token-transfer --start-ledger 60200000
<processor> --describe-json

Emit a JSON-schema-valid manifest of the processor's flags, inputs, outputs, and schema IDs. The agent-legible contract — tooling and LLMs can discover capabilities without reading source.

$ token-transfer --describe-json | jq .outputs

OPTIONS

--rpc-url <url> Stellar RPC endpoint. Overrides NEBU_RPC_URL.
--network <id> Network: mainnet, testnet, or a full passphrase.
--start-ledger N Starting ledger sequence (inclusive).
--end-ledger N Ending ledger sequence (inclusive). Omit it for unbounded streaming.
--quiet Suppress informational output. Only NDJSON events on stdout.
--program-status <mode> OSC 7501 terminal status: auto, always, or never. Default: auto.
--status-file <path> Write stable nebu.status.v1 NDJSON lifecycle and progress records to a dedicated file.
--checkpoint <path> Persist resumable state for a bounded fetch. Use with file output and keep the two paths distinct.
--checkpoint-interval N Ledgers between durable checkpoint writes. Default: 1.
-v, --version Print version information and exit.
-h, --help Show help message and exit.

ENVIRONMENT

NEBU_RPC_URL — default Stellar RPC endpoint. Default: https://archive-rpc.lightsail.network.

NEBU_NETWORK — mainnet, testnet, or passphrase. Default: mainnet.

NEBU_RPC_AUTH — authorization header value for premium RPC endpoints (e.g. Api-Key YOUR_KEY).

NEBU_MODE — rpc or archive.

NEBU_DATASTORE_TYPE — GCS or S3 (archive mode).

NEBU_BUCKET_PATH — bucket path to ledger archive (archive mode).

NEBU_REGION — S3 region (archive mode, S3 only).

NEBU_BUFFER_SIZE — archive fetch cache size. Default: 100.

NEBU_NUM_WORKERS — archive parallel fetch workers. Default: 10.

NEBU_PROGRAM_STATUS — OSC 7501 reporting mode: auto, always, or never.

NEBU_STATUS_FILE — path for the dedicated nebu.status.v1 NDJSON stream.

FILES

registry.yaml — processor registry (repo root or embedded in the CLI).

description.yml — per-processor manifest consumed by nebu install.

$GOPATH/bin/ — install target for processor binaries.

EXIT STATUS

0 — success.

1 — general error (see stderr).

2 — invalid usage or configuration.

3 — requested processor or ledger range was not found or is ambiguous.

4 — RPC or archive source is unavailable.

130 — interrupted by SIGINT / Ctrl+C.

EXAMPLES

Extract token transfers
bash 
$ token-transfer --start-ledger 60200000 --end-ledger 60200100
USDC-only, deduped, to a file
bash 
$ token-transfer --start-ledger 60200000 \
    | jq -c 'select(.transfer.assetCode == "USDC")' \
    | dedup --key meta.txHash \
    | json-file-sink --out usdc.jsonl
SQL analytics with DuckDB, no database
bash 
$ token-transfer --start-ledger 60200000 --end-ledger 60200100 \
    | duckdb -c "
        SELECT
          json_extract_string(transfer, '\$.assetCode') AS asset,
          COUNT(*) AS transfers,
          SUM(CAST(json_extract_string(transfer, '\$.amount') AS DOUBLE)) AS volume
        FROM read_json('/dev/stdin')
        WHERE transfer IS NOT NULL
        GROUP BY asset
        ORDER BY volume DESC
      "
Fan-out: NATS + file + terminal
bash 
$ token-transfer --start-ledger 60200000 \
    | tee >(nats-sink --subject "stellar.live" --jetstream) \
    | tee >(json-file-sink --out archive.jsonl) \
    | jq -r '"Ledger \(.meta.ledgerSequence): \(.transfer.amount)"'
Archive-mode backfill from the AWS public bucket

No AWS credentials required — the SDK falls back to anonymous access. Pubnet only; testnet uses a different (non-Galexie) layout in this bucket.

bash 
$ nebu fetch --mode archive \
    --datastore-type S3 \
    --bucket-path "aws-public-blockchain/v1.1/stellar/ledgers/pubnet" \
    --region us-east-2 \
    62080000 62081000 \
  | gzip > historical.xdr.gz

SCHEMA

Every event carries two self-describing fields:

_schema — schema identifier (e.g. nebu.token_transfer.v1).

_nebu_version — nebu version that produced the event.

Breaking changes bump the schema version. Non-breaking additions (new fields, new event types) do not. Filter by _schema in your downstream queries to stay compatible across rolls.

SEE ALSO

nebu(7) — overview, architecture, quickstart.

token-transfer(1), contract-events(1), json-file-sink(1), nats-sink(1), postgres-sink(1)

flowctl(1) — production orchestrator for nebu-style pipelines.

jq(1), duckdb(1), tee(1), pipe(7)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AUTHOR

Built by OBSRVR as part of the Stellar ecosystem infrastructure.

nebu — /ˈnɛ.buː/ — noun — the vessel that carries you between worlds.

OBSRVR NEBU(1)