# JATS dev pilot

This document describes the planned dev contract for the `oa-jats-25` collection. It becomes available only after routes are deployed and verified in dev; it is not a claim that these routes are live today.

## REST

`GET /v1/articles` searches or browses scoped articles. `query` is optional: omit it to browse all permitted articles, or set `include_unmatched=true` to include permitted zero-score rows alongside query matches. Lexical search covers normalized title, abstract, body, and publisher keywords; bibliography and identifiable inline bibliographic citations are excluded. Unquoted input is term-based, quoted text remains a phrase unit, and raw `OR` is supported; a reference-only term cannot create a hit. Results default to BM25 (`rank=bm25`); `vector` is reserved and `hybrid` currently returns `501`. `order` accepts `desc` or `asc`. Filters are `collection_id`, comma-separated `journal_ids`, optional literal `author` and `keyword` filters (each max 128 characters), and inclusive `published_from`/`published_to` actual publication dates. Author matching is normalized literal substring/token phrase matching without stemming or identity guessing. `page_size` is 1-300 (default 100), independent of customer policy limits. `cursor` is opaque and max 2048 characters.

Default hits are thin: `id`, nullable `doi`, `title`, `updated`, nullable `published_at`, and `version_id`; query results also include `score`, while browse results omit it. Add `include=frontmatter` for the complete abstract, authors, keywords, journal, licence, source and version metadata flattened onto the same hits. Responses include `total` (complete logical result-set count), `matched_total` (query matches), `returned` (rows on this page), `next_cursor`, and `snapshot`.

Cursors are bound to customer scope, query, filters, index and release revision. A corpus/release change returns `409`; restart from the first page. There is no full-result offset ceiling.

The starting BM25 profile is SQLite FTS5 with `k1=1.2`, `b=.75`, and field weights title 5, abstract 2, body 1, keywords 3. Scores use positive SQLite BM25 values; stable article ID breaks ties in descending order. Index statistics include all indexed version rows, are corpus-global, and are not recalculated per customer or current scoped collection; compare scores only within the same index revision. Filtering occurs before pagination. These are resource safeguards and ranking choices, not a customer tariff. Lexical routes make no model calls.

There is no automatic typo correction or quorum relaxation on these lexical routes.

Author and keyword filters are normalized literal word-boundary substring/token phrase matches (maximum 128 characters), without stemming, fuzzy matching, synonyms, or identity guessing; `Smith` matches `Alice Smith`, not `Smithson`. Publication bounds use actual publication dates. Publisher metadata is core; externally sourced metrics, classifications, and additional provenance remain future sidecars. Search observability records D1 rows read and execution time, response bytes, item count, and latency. Content licence units approximate response size, not AI model token usage. Final MCP/search hit-count policy remains five pending measurement.

Identifiers are exposed as `identifiers.content_id`, `identifiers.doi`, `identifiers.publisher_id`, and `identifiers.publisher_url`, while the stable internal content ID remains retained. DOI lookup accepts exact DOI forms including `doi:prefix`, `https://doi.org/...`, the BMJ DOI lookup URL, and PMC publisher IDs/source URLs; it is an exact identifier lookup against the item's own metadata and never searches body or references. URL query values must be URL-encoded.

`GET /v1/frontmatter` accepts an optional comma-separated `ids` list of up to 50, preserves requested order, and omits unknown or out-of-scope IDs. Its collection response uses `items`, `total`, `next_cursor`, and `snapshot`. Without `ids`, it provides a paged browse ordered by publication date descending with the same filters and cursor rules. `GET /v1/frontmatter/{id}` reads one scoped metadata record.

`GET /v1/articles/{id}` returns the complete article. The default is `application/jats+xml`; `format=jats|json|markdown` is supported, with optional `version_id`. Full content is returned or the service responds `413`; it is never silently truncated.

The existing `POST /v1/articles/search` and `GET /v1/articles/{id}/content` routes remain available for compatibility. Existing vector and AI behaviour is unchanged; this pilot adds no vector calls.

## MCP

The new MCP surface exposes `search(query)` and `fetch(id)`. `search` returns structured `id`, `title`, and `url` results and includes the complete abstract in its content. `fetch` returns the whole article, references (`id`, `title`, `text`, `url`), and metadata. MCP is subject to the same customer scope and release rules and is available only after verified dev deployment.

The pilot uses a metadata batch projection for front matter; it does not introduce a separate canonical content object. A future `snapshot` request parameter remains unspecified pending deployment review.

Publisher metadata includes authors, DOI, title, dates, journal, abstract, keywords, licence, source, and version. Externally sourced metrics, classifications, and additional provenance are future sidecars. Search observability records D1 rows read and execution time, plus response bytes, item count, and latency. Content licence units approximate response size and are not AI model token usage; future model calls will have separate AI usage metrics. Final MCP/search hit-count policy remains five pending measurement.