lucene-retrieval-usagelisted
Install: claude install-skill josix/agentic-retrieval
# Lucene Retrieval Usage (pi-serini)
Lucene BM25 retrieval via `PiSeriniRetriever` (`retrieval/retrievers.py`),
backed by Pyserini/Anserini — the reference lexical retriever from
[pi-serini](https://github.com/justram/pi-serini) ("Rethinking Agentic Search
with Pi-Serini: Is Lexical Retrieval Sufficient?").
`pi-serini` and `pyserini` are two different names, and neither is a typo
for the other: `pi-serini` is the retrieval *strategy* from the paper above
(and this plugin's registry key / `--retriever` value), while `pyserini` is
the Castorini Python library — and this repo's install extra — that the
strategy is implemented on top of. Wherever both appear on the same line
below, the difference is deliberate.
## What it is
`PiSeriniRetriever` builds an in-memory Lucene inverted index
(`pyserini.index.lucene.LuceneIndexer`) over the corpus and queries it with
`LuceneSearcher.set_bm25(k1, b)`. Pi-serini's thesis: a well-configured
lexical (BM25) retriever, given **sufficient retrieval depth**, can be
competitive with dense retrieval for agentic deep research — instead of
closing the vocabulary gap with embeddings, retrieve deeper and let a capable
LLM agent compensate by reading more candidates.
Pi-serini's BrowseComp-Plus-tuned defaults push BM25 far from its usual
settings: `k1=25` (vs. Lucene's usual ~0.9) nearly disables term-frequency
saturation, letting repeated query terms keep accumulating score, and `b=1`
applies full document-length normalization — tuned for long (