From 243f717bc4201d7e1a626a97ea5e469a0a0f3f55 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Tue, 2 Dec 2025 23:46:53 +0100 Subject: [PATCH 1/4] IPIP-0526: historic Bitswap provider publishing API documents the legacy PUT /routing/v1/providers/ endpoint for signed Bitswap provider records, originally introduced for index-provider/IPNI integration. includes CSS styling for historic IPIP state. --- src/css/specs.css | 56 +++++++++ src/img/watermark-historic.svg | 8 ++ src/ipips/ipip-0526.md | 221 +++++++++++++++++++++++++++++++++ 3 files changed, 285 insertions(+) create mode 100644 src/img/watermark-historic.svg create mode 100644 src/ipips/ipip-0526.md diff --git a/src/css/specs.css b/src/css/specs.css index 4caa2fb12..7c081f4c3 100644 --- a/src/css/specs.css +++ b/src/css/specs.css @@ -30,6 +30,62 @@ body { background: url('/img/watermark-ratified.svg'); } +.ipip-historic .watermark { + display: block; + background: url('/img/watermark-historic.svg'); + background-color: #faf8f5; +} + +.ipip-historic { + --highlight-colour: #b08050; + --highlight-colour-faded: #c9a07a; + --standard-gradient: linear-gradient(to right, #5c4a3a, #b08050); +} + +.ipip-historic header { + border-bottom-color: #b08050; +} + +.ipip-historic a { + color: #8b6914; +} + +.ipip-historic a:hover { + color: #b08050; +} + +.ipip-historic h1, +.ipip-historic h2, +.ipip-historic h3, +.ipip-historic h4, +.ipip-historic h5, +.ipip-historic h6 { + color: #5c4a3a; +} + +.ipip-historic :is(h2, h3, h4, h5, h6) bdi.secno { + color: #9a8a7a; +} + +.ipip-historic pre { + background: #faf6f0; + border-color: #b08050; +} + +.ipip-historic div.note, +.ipip-historic div.warning, +.ipip-historic div.example { + background: #faf6f0; +} + +.ipip-historic div.note { + border-color: #b08050; +} + +.ipip-historic div.warning { + border-color: #c07030; +} + #ipseity-back-to-root { margin-bottom: 2rem; background: var(--standard-gradient); diff --git a/src/img/watermark-historic.svg b/src/img/watermark-historic.svg new file mode 100644 index 000000000..ab34bee09 --- /dev/null +++ b/src/img/watermark-historic.svg @@ -0,0 +1,8 @@ + + + + Historic + + + + diff --git a/src/ipips/ipip-0526.md b/src/ipips/ipip-0526.md new file mode 100644 index 000000000..aa0e4026f --- /dev/null +++ b/src/ipips/ipip-0526.md @@ -0,0 +1,221 @@ +--- +title: 'IPIP-0526: Historic Bitswap Provider Publishing API' +date: 2025-12-02 +ipip: historic +editors: + - name: Marcin Rataj + github: lidel + url: https://lidel.org/ + affiliation: + name: Shipyard + url: https://ipshipyard.com +thanks: + - name: Adin Schmahmann + github: aschmahmann + affiliation: + name: Shipyard + url: https://ipshipyard.com + - name: Andrew Gillis + github: gammazero + affiliation: + name: Shipyard + url: https://ipshipyard.com + - name: Henrique Dias + url: https://hacdias.com/ + github: hacdias +relatedIssues: + - https://github.com/ipni/index-provider/issues/403 + - https://discuss.ipfs.tech/t/19545/9 +order: 526 +tags: ['ipips'] +xref: + - http-routing-v1 +--- + +## Summary + +This IPIP documents the `PUT /routing/v1/providers/` HTTP endpoint for publishing signed Bitswap provider records, introduced in 2022 for [index-provider](https://github.com/ipni/index-provider) integration with IPNI. + +The API is deprecated because it was never formally specified, lacks conformance tests, has only partial implementation (see notes below), and cannot announce content available via HTTP ([Trustless Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)). + +:::warning + +This API was never formally specified, and community projects have been reverse-engineering it. This historic IPIP discourages further adoption and is not intended to be merged into :cite[http-routing-v1]. + +::: + +## Motivation + +The Delegated Routing V1 HTTP API (:cite[http-routing-v1]) was designed primarily for content discovery via `GET` endpoints. However, the ecosystem also needed a mechanism for peers to publish their provider records to routing servers without relying solely on DHT announcements. + +The [index-provider](https://github.com/ipni/index-provider) project (created in 2021) required a way for IPFS nodes (such as Kubo) to advertise their content to indexers (like cid.contact) alongside DHT. This created a parallel discovery mechanism where providers could reach indexing networks without relying exclusively on the distributed hash table. + +The provider publishing API was first added to the Reframe protocol in August 2022 ([ipfs/specs#285](https://github.com/ipfs/specs/pull/285), [go-delegated-routing#37](https://github.com/ipfs/go-delegated-routing/pull/37)). In November 2022, the code was refactored to use `/routing/v1` paths, and the `WriteBitswapRecord` schema was introduced. The code was later migrated to [go-libipfs](https://github.com/ipfs/go-libipfs) and then to [boxo](https://github.com/ipfs/boxo). + +When [IPIP-337](https://github.com/ipfs/specs/pull/337) introduced the Delegated Routing V1 HTTP API specification in February 2023, write operations were [explicitly excluded](https://github.com/ipfs/specs/pull/370) to reduce scope. [IPIP-378](https://github.com/ipfs/specs/pull/378) was opened to design a modernized, protocol-agnostic replacement that would support announcing multiple CIDs in a single request, but work on it was never completed before Protocol Labs restructuring. As a result, the Bitswap-specific `PUT /routing/v1/providers/` endpoint remained in boxo but was never formally documented. + +The API has not seen significant changes since January 2023, when the HTTP method was changed from POST to PUT. It remains in [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) but is marked as deprecated, pending replacement by a protocol-agnostic alternative. + +Despite being undocumented, community projects such as [Plebbit](https://plebbit.com) have been reverse engineering this API to implement provider publishing. Creating this historic IPIP discourages further adoption of this deprecated API and provides a reference for discussions about its deprecation and replacements. + +## Detailed design + +This IPIP documents the `PUT /routing/v1/providers/` endpoint as a historical artifact. This specification is self-contained and not intended to be merged into :cite[http-routing-v1]. + +### `PUT /routing/v1/providers/` + +This endpoint allows peers to publish signed provider records announcing which CIDs they can provide. + +#### Request Headers + +- `Content-Type`: SHOULD be `application/json`. + +#### Request Body + +```json +{ + "Providers": [ + { + "Schema": "bitswap", + "Protocol": "transport-bitswap", + "Signature": "", + "Payload": { + "Keys": ["", ...], + "Timestamp": , + "AdvisoryTTL": , + "ID": "", + "Addrs": ["", ...] + } + } + ] +} +``` + +Example with concrete values: + +```json +{ + "Providers": [ + { + "Schema": "bitswap", + "Protocol": "transport-bitswap", + "Signature": "mx5kamm5kzxuCnVJtX3K9DEj8gKlFqXil2x/M8zDTozvzowTY6W+HOALQ2LCkTZCEz4H5qizpnHxPM/rVQ7MNBg", + "Payload": { + "Keys": ["bafkreigur6gzxm3ykiol7ywou3iy3obruzs2q7boizj7oznznid34dzc3e"], + "Timestamp": 1725833163372, + "AdvisoryTTL": 86400000000000, + "ID": "12D3KooWEdCRaQTjjgbtBoSMhnguznp7GHhsin8eRDEtgEso6Z1B", + "Addrs": ["/ip4/198.51.100.1/tcp/4001", "/ip4/198.51.100.1/udp/4001/quic-v1"] + } + } + ] +} +``` + +Where: + +- `Schema`: MUST be `"bitswap"` for this record type. +- `Protocol`: SHOULD be `"transport-bitswap"`. +- `Signature`: Multibase-encoded (base64) signature of the SHA256 hash of the JSON-serialized `Payload`. +- `Payload`: The signed provider information: + - `Keys`: Array of CIDs (string-encoded CIDv1) that the peer provides. + - `Timestamp`: Unix timestamp in milliseconds when the record was created. + - `AdvisoryTTL`: Suggested time-to-live in nanoseconds (as JSON number). + - `ID`: The provider's Peer ID (string-encoded as Multihash in Base58btc or CIDv1 with libp2p-key codec). + - `Addrs`: Array of multiaddrs where the provider can be reached. + +:::note + +Implementation detail: At the time of writing this IPIP (2025Q4), [boxo v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) enforces `Schema` (must be `"bitswap"`, returns 400 otherwise) and signature verification (returns 403 on failure). However, `Protocol` is not validated - it is stored and echoed in the response. In [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0), `Timestamp` and `AdvisoryTTL` from the request are ignored: the server uses `time.Now()` for record timestamps and returns its own configured `cidTtl` in the response. Only `Keys`, `ID`, and `Addrs` are used for IPNI announcements, making parts of this specification effectively unused in practice. + +::: + +#### Signature Verification + +The server MUST verify the signature before accepting the record: + +1. Extract the public key from the `Payload.ID` peer ID. +2. Decode the `Signature` from multibase (base64). +3. Compute SHA256 hash of the `Payload` object serialized as JSON (the exact bytes as sent in the request). +4. Verify the signature against the hash using the extracted public key. + +If verification fails, the server MUST respond with `403 Forbidden`. + +#### Response Status Codes + +- `200` (OK): The provider record was accepted. +- `400` (Bad Request): The request body is malformed or missing required fields. +- `403` (Forbidden): Signature verification failed. +- `422` (Unprocessable Entity): Request does not conform to schema or semantic constraints. + +#### Response Body + +```json +{ + "ProvideResults": [ + { + "Schema": "bitswap", + "Protocol": "transport-bitswap", + "AdvisoryTTL": 86400000000000 + } + ] +} +``` + +Where: + +- `Schema`: Echoes `"bitswap"`. +- `Protocol`: Echoes `"transport-bitswap"`. +- `AdvisoryTTL`: Server-computed TTL in nanoseconds indicating how long the record will be cached. Clients SHOULD republish before this TTL expires. + +## Design rationale + +### Bitswap-Specific Schema + +The `bitswap` schema was designed specifically for the Bitswap transfer protocol. The schema name and `Protocol` field make it explicit that these records are for peers providing content over Bitswap. + +Historically, this made sense when Bitswap was the only means of retrieving data from IPFS peers. In modern times, many providers use HTTP [Trustless Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/) in addition to Bitswap, and sometimes instead of it. + +This tight coupling to Bitswap is one of the reasons this API is considered deprecated. Future iterations should support protocol-agnostic provider announcements. + +### User benefit + +This API provided a convenient way for [index-provider](https://github.com/ipni/index-provider) to enable Kubo users to announce content to IPNI indexers (like cid.contact) without relying solely on DHT. + +However, the API was never adopted outside of the IPNI ecosystem, limiting its utility and test coverage in practice. + +### Compatibility + +This is a legacy API. The existing client/server implementation in [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) serves as the historical reference. + +The related types (`WriteBitswapRecord`, `WriteBitswapRecordResponse`, `SchemaBitswap`) were marked as deprecated in boxo on August 24, 2023, when they were revived specifically to maintain backwards compatibility with index-provider. + +New implementations SHOULD NOT implement this endpoint. Existing deployments MAY continue using it but should plan migration to future protocol-agnostic alternatives. + +### Security + +- **Authentication**: Provider records are cryptographically signed to prevent unauthorized announcements. Only the peer that owns the private key corresponding to the announced Peer ID can publish records for that identity, preventing malicious actors from advertising content they do not possess. +- **Replay Protection**: The `Timestamp` field was intended to allow servers to reject stale records. +- **TTL Enforcement**: The `AdvisoryTTL` was intended to limit how long records persist. + +:::note + +As noted earlier, [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0) ignores `Timestamp` and `AdvisoryTTL`, providing no replay protection. This is acceptable for deployments where both client and server are trusted and run in a private network. + +::: + +### Alternatives + +[IPIP-378](https://github.com/ipfs/specs/pull/378) was an attempt to design a modernized, protocol-agnostic replacement that would support announcing multiple CIDs in a single request using signed DAG-CBOR and/or JSON payloads. [ipni/index-provider#403](https://github.com/ipni/index-provider/issues/403) was opened to migrate index-provider to IPIP-378 once ratified, but IPIP-378 was never completed. + +At the time of writing this IPIP (2025Q4), there is no vendor-agnostic alternative to this HTTP API. A future IPIP could address this gap by extending [:cite[http-routing-v1]](https://specs.ipfs.tech/routing/http-routing-v1/) with a modern specification for delegated publishing of provider information over HTTP. Such work should incorporate lessons from this historic API and support both Bitswap and HTTP ([Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) providers natively. + +## Test fixtures + +Not applicable. This IPIP documents existing implementation behavior. + +The lack of test fixtures and conformance tests is one of the contributing reasons why this API was never part of official `/routing/v1` specification. + +## Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From cde9a6269e7e86600a109b688d8fc5ef2b93694c Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Sat, 8 Aug 2026 02:42:41 +0200 Subject: [PATCH 2/4] fix: historic theme contrast and CSS var hooks Style historic IPIPs through ipseity's variable hooks instead of overriding selectors: the TOC keeps body-text color, links meet WCAG AA (4.8:1), and note/warning borders reuse --note-colour and --warning-colour. Add the missing pre box-shadow override that left a blue sliver next to the brown border, and drop the unused --highlight-colour-faded. --- src/css/specs.css | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/src/css/specs.css b/src/css/specs.css index 7c081f4c3..9a51c8f94 100644 --- a/src/css/specs.css +++ b/src/css/specs.css @@ -38,7 +38,9 @@ body { .ipip-historic { --highlight-colour: #b08050; - --highlight-colour-faded: #c9a07a; + --link-colour: #8b6914; + --note-colour: #b08050; + --warning-colour: #c07030; --standard-gradient: linear-gradient(to right, #5c4a3a, #b08050); } @@ -46,14 +48,6 @@ body { border-bottom-color: #b08050; } -.ipip-historic a { - color: #8b6914; -} - -.ipip-historic a:hover { - color: #b08050; -} - .ipip-historic h1, .ipip-historic h2, .ipip-historic h3, @@ -70,6 +64,7 @@ body { .ipip-historic pre { background: #faf6f0; border-color: #b08050; + box-shadow: -1px 0 0 0 #b08050, 0 0 0 1px #e5d8c8; } .ipip-historic div.note, @@ -78,14 +73,6 @@ body { background: #faf6f0; } -.ipip-historic div.note { - border-color: #b08050; -} - -.ipip-historic div.warning { - border-color: #c07030; -} - #ipseity-back-to-root { margin-bottom: 2rem; background: var(--standard-gradient); From 8ad18981ffa4b504b02677707193d1502b12ecf5 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Sat, 8 Aug 2026 02:42:48 +0200 Subject: [PATCH 3/4] IPIP-0526: match observed behavior, add fixtures Align the endpoint documentation with boxo v0.35.2 and known deployments, and add reproducible test vectors so existing users have an interoperability reference. - error codes: drop unused 422, add 500 delegate errors, document non-atomic processing and the crash on missing Timestamp/AdvisoryTTL - signature: exact signed bytes (SHA-256 of payload as sent), multibase encoding, identity-multihash key limitation, verification as server policy that some deployments skip - example replaced with a valid Ed25519 signature; signed and unsigned fixtures with reproducible key material in Test fixtures - known users: index-provider (single provider identity) and Bitsocial trackers (previously Plebbit), with pkc-js and pkc-http-router links - request headers, CIDv0 keys, Protocol echo-only, response TTL source --- src/ipips/ipip-0526.md | 180 ++++++++++++++++++++++++++++++++--------- 1 file changed, 140 insertions(+), 40 deletions(-) diff --git a/src/ipips/ipip-0526.md b/src/ipips/ipip-0526.md index aa0e4026f..e96809297 100644 --- a/src/ipips/ipip-0526.md +++ b/src/ipips/ipip-0526.md @@ -1,11 +1,10 @@ --- -title: 'IPIP-0526: Historic Bitswap Provider Publishing API' -date: 2025-12-02 +title: 'IPIP-0526: Historic Bitswap Provider Publishing HTTP API' +date: 2026-08-08 ipip: historic editors: - name: Marcin Rataj github: lidel - url: https://lidel.org/ affiliation: name: Shipyard url: https://ipshipyard.com @@ -23,6 +22,11 @@ thanks: - name: Henrique Dias url: https://hacdias.com/ github: hacdias + - name: Rinse + github: Rinse12 + affiliation: + name: Bitsocial + url: https://bitsocial.net relatedIssues: - https://github.com/ipni/index-provider/issues/403 - https://discuss.ipfs.tech/t/19545/9 @@ -36,11 +40,13 @@ xref: This IPIP documents the `PUT /routing/v1/providers/` HTTP endpoint for publishing signed Bitswap provider records, introduced in 2022 for [index-provider](https://github.com/ipni/index-provider) integration with IPNI. -The API is deprecated because it was never formally specified, lacks conformance tests, has only partial implementation (see notes below), and cannot announce content available via HTTP ([Trustless Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)). +The API is deprecated because it was never formally specified, lacks conformance tests, has only partial implementation (see notes below), and cannot announce content available via HTTP ([Trustless Gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) due to being naively hardcoded to the `bitswap` schema. + +This document describes the wire format and the observed behavior of the reference implementation, including reproducible test vectors, so that existing deployments keep an interoperability reference while a replacement is designed. :::warning -This API was never formally specified, and community projects have been reverse-engineering it. This historic IPIP discourages further adoption and is not intended to be merged into :cite[http-routing-v1]. +This API was never formally specified, and community projects have been reverse engineering it. This historic IPIP discourages further adoption and is not intended to be merged into :cite[http-routing-v1]. ::: @@ -54,9 +60,9 @@ The provider publishing API was first added to the Reframe protocol in August 20 When [IPIP-337](https://github.com/ipfs/specs/pull/337) introduced the Delegated Routing V1 HTTP API specification in February 2023, write operations were [explicitly excluded](https://github.com/ipfs/specs/pull/370) to reduce scope. [IPIP-378](https://github.com/ipfs/specs/pull/378) was opened to design a modernized, protocol-agnostic replacement that would support announcing multiple CIDs in a single request, but work on it was never completed before Protocol Labs restructuring. As a result, the Bitswap-specific `PUT /routing/v1/providers/` endpoint remained in boxo but was never formally documented. -The API has not seen significant changes since January 2023, when the HTTP method was changed from POST to PUT. It remains in [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) but is marked as deprecated, pending replacement by a protocol-agnostic alternative. +The wire format has not changed since January 2023, when the HTTP method was changed from POST to PUT. The implementation remains in [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) but is marked as deprecated, pending replacement by a protocol-agnostic alternative. -Despite being undocumented, community projects such as [Plebbit](https://plebbit.com) have been reverse engineering this API to implement provider publishing. Creating this historic IPIP discourages further adoption of this deprecated API and provides a reference for discussions about its deprecation and replacements. +Despite being undocumented, community projects have reverse engineered this API to implement provider publishing. [Bitsocial](https://bitsocial.net) (previously named [Plebbit](https://plebbit.com)), a peer-to-peer social media protocol, uses it so that nodes running apps such as [Seedit](https://seedit.app) can announce content to volunteer-run trackers instead of the public DHT (see Known users below). Creating this historic IPIP discourages further adoption of this deprecated API, while providing a reference for its existing users and for discussions about replacements. ## Detailed design @@ -66,9 +72,11 @@ This IPIP documents the `PUT /routing/v1/providers/` endpoint as a historical ar This endpoint allows peers to publish signed provider records announcing which CIDs they can provide. +In this document, "reference client" and "reference server" refer to the implementations in [boxo v0.35.2 `routing/http`](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http): the client is what Kubo uses to publish, and the server is the handler that index-provider mounts. + #### Request Headers -- `Content-Type`: SHOULD be `application/json`. +The reference client sets no `Content-Type` header on the request (only `User-Agent`), and the reference server never reads it: the body is decoded as JSON regardless of headers. Clients MAY send `Content-Type: application/json`; servers MUST NOT require it. #### Request Body @@ -91,7 +99,7 @@ This endpoint allows peers to publish signed provider records announcing which C } ``` -Example with concrete values: +Example with concrete values (a validly signed record; see [Test fixtures](#test-fixtures) for the key material): ```json { @@ -99,54 +107,81 @@ Example with concrete values: { "Schema": "bitswap", "Protocol": "transport-bitswap", - "Signature": "mx5kamm5kzxuCnVJtX3K9DEj8gKlFqXil2x/M8zDTozvzowTY6W+HOALQ2LCkTZCEz4H5qizpnHxPM/rVQ7MNBg", - "Payload": { - "Keys": ["bafkreigur6gzxm3ykiol7ywou3iy3obruzs2q7boizj7oznznid34dzc3e"], - "Timestamp": 1725833163372, - "AdvisoryTTL": 86400000000000, - "ID": "12D3KooWEdCRaQTjjgbtBoSMhnguznp7GHhsin8eRDEtgEso6Z1B", - "Addrs": ["/ip4/198.51.100.1/tcp/4001", "/ip4/198.51.100.1/udp/4001/quic-v1"] - } + "Signature": "mlGWqELZSoVjY22w6NxT7TuNUj5BmQRlrv/x27jQHo5pM3CYZRlJ834bi0UazFuXlH2SuMBxdwfELdXXwVkjZBQ", + "Payload": {"Keys":["bafkreigur6gzxm3ykiol7ywou3iy3obruzs2q7boizj7oznznid34dzc3e"],"Timestamp":1725833163372,"AdvisoryTTL":86400000000000,"ID":"12D3KooWKcTtoUYcQVfURdwUZtPsanm97SdEf7DkesyxTz7kkD2z","Addrs":["/ip4/198.51.100.1/tcp/4001","/ip4/198.51.100.1/udp/4001/quic-v1"]} } ] } ``` +:::note + +The `Signature` above is a real signature that verifies against this exact `Payload`. Because the signature covers the `Payload` bytes exactly as sent, the single-line `Payload` value must stay byte-identical: reformatting it (for example, pretty-printing) invalidates the signature. + +::: + Where: - `Schema`: MUST be `"bitswap"` for this record type. -- `Protocol`: SHOULD be `"transport-bitswap"`. -- `Signature`: Multibase-encoded (base64) signature of the SHA256 hash of the JSON-serialized `Payload`. +- `Protocol`: SHOULD be `"transport-bitswap"`. The reference server does not validate this field. +- `Signature`: Multibase base64 (prefix `m`, no padding) signature over the SHA-256 hash of the `Payload` bytes exactly as they appear in the request body (see Signature Creation below). - `Payload`: The signed provider information: - - `Keys`: Array of CIDs (string-encoded CIDv1) that the peer provides. + - `Keys`: Array of string-encoded CIDs that the peer provides. The reference server accepts both CIDv0 and CIDv1; new records SHOULD use CIDv1. - `Timestamp`: Unix timestamp in milliseconds when the record was created. - `AdvisoryTTL`: Suggested time-to-live in nanoseconds (as JSON number). - `ID`: The provider's Peer ID (string-encoded as Multihash in Base58btc or CIDv1 with libp2p-key codec). - `Addrs`: Array of multiaddrs where the provider can be reached. +`ID`, `Timestamp`, and `AdvisoryTTL` are required in practice: a record without `ID` fails signature verification (`403`), and the reference server assumes `Timestamp` and `AdvisoryTTL` are present and crashes on records without them (the client sees a dropped connection instead of an HTTP error). + :::note -Implementation detail: At the time of writing this IPIP (2025Q4), [boxo v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) enforces `Schema` (must be `"bitswap"`, returns 400 otherwise) and signature verification (returns 403 on failure). However, `Protocol` is not validated - it is stored and echoed in the response. In [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0), `Timestamp` and `AdvisoryTTL` from the request are ignored: the server uses `time.Now()` for record timestamps and returns its own configured `cidTtl` in the response. Only `Keys`, `ID`, and `Addrs` are used for IPNI announcements, making parts of this specification effectively unused in practice. +Implementation detail: At the time of writing this IPIP (2025Q4), [boxo v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) enforces `Schema` (must be `"bitswap"`, returns 400 otherwise) and signature verification (returns 403 on failure). However, `Protocol` is not validated: it is echoed in the response and never passed to the backing service. In [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0), `Timestamp` and `AdvisoryTTL` from the request are ignored: the server uses `time.Now()` for record timestamps and returns its own configured `cidTtl` in the response. Only `Keys`, `ID`, and `Addrs` are used for IPNI announcements, making parts of this specification effectively unused in practice. index-provider also accepts records from only one provider identity per instance (the configured `ProviderID`, or the first peer that publishes when none is configured); correctly signed records from any other peer are rejected with `500` and a `delegate error: provider ... isn't allowed` message. ::: +#### Signature Creation + +The signature covers the `Payload` bytes exactly as they appear in the request body: clients serialize the payload first, sign those bytes, and embed the identical bytes in the request. + +1. Serialize `Payload` to JSON. The reference client emits compact JSON with HTML escaping disabled (Go's `encoding/json`); any serialization works, as long as the transmitted `Payload` bytes are byte-identical to the signed ones. +2. Compute the SHA-256 hash of the payload bytes. +3. Sign the 32-byte digest with the libp2p private key corresponding to `Payload.ID` (for Ed25519 keys this is a plain Ed25519 signature over the digest). +4. Encode the result as multibase base64: prefix `m` followed by standard-alphabet base64 without padding. + #### Signature Verification -The server MUST verify the signature before accepting the record: +The reference server verifies every record and responds with `403 Forbidden` when any step fails: + +1. Reject records that have no `Signature` or no `Payload.ID`. +2. Extract the public key from the `Payload.ID` Peer ID. This only succeeds for Peer IDs that embed the public key as an identity multihash, which is the case for Ed25519 and Secp256k1 keys. RSA and ECDSA Peer IDs contain only a hash of the key, so their records always fail verification, even when correctly signed. +3. Decode the `Signature` from multibase. +4. Compute the SHA-256 hash of the received `Payload` bytes and verify the signature against it using the extracted public key. -1. Extract the public key from the `Payload.ID` peer ID. -2. Decode the `Signature` from multibase (base64). -3. Compute SHA256 hash of the `Payload` object serialized as JSON (the exact bytes as sent in the request). -4. Verify the signature against the hash using the extracted public key. +Verification is server policy, not a wire-format guarantee: -If verification fails, the server MUST respond with `403 Forbidden`. +- The reference server always verifies: [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) has no option to disable verification, and its client cannot produce unsigned records. index-provider inherits this behavior and performs no signature checks of its own. +- Bitsocial trackers do not verify signatures and treat the `Signature` field as optional; they bound abuse by requiring announced addresses to match the request IP instead. +- Skipping verification is reasonable only when the server does not accept announcements from untrusted parties, for example when client and server run on the same machine or in a private network. A publicly reachable server that skips verification allows anyone to publish records for any Peer ID and any CID. + +Unsigned records (no `Signature` field) interoperate only with servers that skip verification; the reference server rejects them with `403`. #### Response Status Codes -- `200` (OK): The provider record was accepted. -- `400` (Bad Request): The request body is malformed or missing required fields. -- `403` (Forbidden): Signature verification failed. -- `422` (Unprocessable Entity): Request does not conform to schema or semantic constraints. +Observed behavior of the reference server implementation: + +- `200` (OK): All provider records were accepted. +- `400` (Bad Request): The request body is not valid JSON or does not decode into the structure above, or a record's `Schema` is not `"bitswap"`. +- `403` (Forbidden): Signature verification failed. This includes a missing or invalid `Signature`, a missing `Payload.ID`, and Peer IDs from which a public key cannot be extracted (see Signature Verification above). +- `500` (Internal Server Error): The backing service rejected the records (`delegate error: ...`), for example due to index-provider's provider identity restriction, or another server-side failure occurred. + +:::warning + +Acceptance is not atomic. The reference server processes `Providers` sequentially and stops at the first failing record: records before it were already accepted and passed to the backing service, and there is no rollback. Clients MUST NOT treat an error response as proof that no records were accepted. + +A signed record without `Timestamp` or `AdvisoryTTL` crashes the reference server's request handler: the client sees a dropped connection with no HTTP status code at all. + +::: #### Response Body @@ -166,7 +201,7 @@ Where: - `Schema`: Echoes `"bitswap"`. - `Protocol`: Echoes `"transport-bitswap"`. -- `AdvisoryTTL`: Server-computed TTL in nanoseconds indicating how long the record will be cached. Clients SHOULD republish before this TTL expires. +- `AdvisoryTTL`: TTL in nanoseconds chosen by the server (not the value from the request), indicating how long the record will be cached. Clients SHOULD republish before this TTL expires. ## Design rationale @@ -182,25 +217,36 @@ This tight coupling to Bitswap is one of the reasons this API is considered depr This API provided a convenient way for [index-provider](https://github.com/ipni/index-provider) to enable Kubo users to announce content to IPNI indexers (like cid.contact) without relying solely on DHT. -However, the API was never adopted outside of the IPNI ecosystem, limiting its utility and test coverage in practice. +Although built for IPNI, the API was later picked up outside that ecosystem: Bitsocial reverse engineered it to let its apps announce content to community-run trackers (see Known users below). Documenting the endpoint, together with test vectors, gives these existing deployments an interoperability reference. The API's utility remains limited by its Bitswap-only scope and the lack of conformance tests. + +### Known users + +Two kinds of deployments are known to use this API: + +- **index-provider (IPNI)**: the deployment the API was created for. [index-provider](https://github.com/ipni/index-provider) mounts boxo's reference server handler and runs as a sidecar on the same machine as a Kubo node, which is configured with `Routing.Type` set to `custom` and an HTTP router pointing at the sidecar (usually over localhost) for the `provide` method. Signatures are generated by Kubo and verified by the sidecar. Each sidecar instance accepts announcements from a single provider identity only: the configured `ProviderID`, or the first peer that publishes when none is configured. +- **Bitsocial trackers**: [Bitsocial](https://bitsocial.net) (previously named [Plebbit](https://plebbit.com)) is a peer-to-peer social media protocol whose apps, such as [Seedit](https://seedit.app), bundle a Kubo node. These nodes run with the public DHT disabled and announce content over this API to volunteer-run trackers whose default endpoints are hardcoded in the [pkc-js client SDK](https://github.com/pkcprotocol/pkc-js) (PKC, "Public Key Communities", is the project's name for its protocol layer). The [pkc-http-router tracker server](https://github.com/pkcprotocol/pkc-http-router) stores only `Keys`, `ID`, and `Addrs`, does not verify signatures, and instead requires announced addresses to match the IP the request came from. On this path the `Signature` is usually not even valid: a local proxy on the client machine [rewrites `Payload.Addrs`](https://github.com/pkcprotocol/pkc-js/blob/49f6b9c5028a870dcaaae08a5a709e9b5b506600/src/runtime/node/addresses-rewriter-proxy-server.ts) after Kubo signs the record, without re-signing it. + +This deployment is described first-hand, under the Plebbit name, in the [discussion that led to this IPIP](https://discuss.ipfs.tech/t/only-peers-found-from-dht-seem-to-be-getting-used-as-relays-so-cant-use-http-routers/19545). ### Compatibility This is a legacy API. The existing client/server implementation in [boxo v0.35.2](https://github.com/ipfs/boxo/tree/v0.35.2/routing/http) serves as the historical reference. -The related types (`WriteBitswapRecord`, `WriteBitswapRecordResponse`, `SchemaBitswap`) were marked as deprecated in boxo on August 24, 2023, when they were revived specifically to maintain backwards compatibility with index-provider. +The related types (`WriteBitswapRecord`, `WriteBitswapRecordResponse`, `SchemaBitswap`) were marked as deprecated in boxo in August 2023, when they were revived specifically to maintain backwards compatibility with index-provider. + +Kubo remains capable of publishing via this API, but only when explicitly configured with `Routing.Type` set to `custom` and an HTTP router that handles the `provide` method ([delegated routing docs](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md)); the default configuration does not publish provider records over HTTP. New implementations SHOULD NOT implement this endpoint. Existing deployments MAY continue using it but should plan migration to future protocol-agnostic alternatives. ### Security -- **Authentication**: Provider records are cryptographically signed to prevent unauthorized announcements. Only the peer that owns the private key corresponding to the announced Peer ID can publish records for that identity, preventing malicious actors from advertising content they do not possess. +- **Authentication**: A valid signature proves the record was created by the holder of the private key for the announced Peer ID, so third parties cannot publish records under someone else's identity. It proves nothing about content possession: any peer can announce any CID under its own Peer ID, so signatures alone do not prevent index spam or announcements of content the peer cannot serve. - **Replay Protection**: The `Timestamp` field was intended to allow servers to reject stale records. - **TTL Enforcement**: The `AdvisoryTTL` was intended to limit how long records persist. :::note -As noted earlier, [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0) ignores `Timestamp` and `AdvisoryTTL`, providing no replay protection. This is acceptable for deployments where both client and server are trusted and run in a private network. +As noted earlier, [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0) ignores `Timestamp` and `AdvisoryTTL`, providing no replay protection, and Bitsocial trackers additionally skip signature verification. These relaxations are acceptable only where the server does not accept records from untrusted parties, for example when both client and server are operated by the same party on one machine or in a private network, or where abuse is bounded by other checks (see Signature Verification above). ::: @@ -208,14 +254,68 @@ As noted earlier, [index-provider v0.16.0](https://github.com/ipni/index-provide [IPIP-378](https://github.com/ipfs/specs/pull/378) was an attempt to design a modernized, protocol-agnostic replacement that would support announcing multiple CIDs in a single request using signed DAG-CBOR and/or JSON payloads. [ipni/index-provider#403](https://github.com/ipni/index-provider/issues/403) was opened to migrate index-provider to IPIP-378 once ratified, but IPIP-378 was never completed. -At the time of writing this IPIP (2025Q4), there is no vendor-agnostic alternative to this HTTP API. A future IPIP could address this gap by extending [:cite[http-routing-v1]](https://specs.ipfs.tech/routing/http-routing-v1/) with a modern specification for delegated publishing of provider information over HTTP. Such work should incorporate lessons from this historic API and support both Bitswap and HTTP ([Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) providers natively. +At the time of writing this IPIP (2025Q4), there is no vendor-agnostic alternative to this HTTP API. A future IPIP could address this gap by extending :cite[http-routing-v1] with a modern specification for delegated publishing of provider information over HTTP. Such work should incorporate lessons from this historic API and support both Bitswap and HTTP ([Trustless Gateway](https://specs.ipfs.tech/http-gateways/trustless-gateway/)) providers natively. ## Test fixtures -Not applicable. This IPIP documents existing implementation behavior. +The signed example in the Detailed design section is reproducible from the following key material (a throwaway Ed25519 key generated for this document): + +- Ed25519 seed (hex), derived as the SHA-256 of the ASCII string `ipip-0526-test-vector`: + + ``` + 829ecb5f7f2b24c936a194361c919249ae9d6b88701e33939daec2fe6f08d747 + ``` + +- Private key (libp2p protobuf encoding, base64): + + ``` + CAESQIKey19/KyTJNqGUNhyRkkmunWuIcB4zk52uwv5vCNdHkYhfe8CtcuHgLiYw9NjGH5MWhlZ5lPeWEFq1aEG7kos= + ``` + +- Peer ID: `12D3KooWKcTtoUYcQVfURdwUZtPsanm97SdEf7DkesyxTz7kkD2z` + +### Signed record + +The exact `Payload` bytes to sign (a single line, compact JSON): + +```json +{"Keys":["bafkreigur6gzxm3ykiol7ywou3iy3obruzs2q7boizj7oznznid34dzc3e"],"Timestamp":1725833163372,"AdvisoryTTL":86400000000000,"ID":"12D3KooWKcTtoUYcQVfURdwUZtPsanm97SdEf7DkesyxTz7kkD2z","Addrs":["/ip4/198.51.100.1/tcp/4001","/ip4/198.51.100.1/udp/4001/quic-v1"]} +``` + +- SHA-256 of the payload bytes (hex), which is the message passed to Ed25519: + + ``` + 198b43ea6aba9071845059c5e91eba3926df35926da35c61c1b7429fd7ffb3b8 + ``` + +- Signature (multibase base64): + + ``` + mlGWqELZSoVjY22w6NxT7TuNUj5BmQRlrv/x27jQHo5pM3CYZRlJ834bi0UazFuXlH2SuMBxdwfELdXXwVkjZBQ + ``` + +A conforming verifier accepts this record: extract the Ed25519 public key from the Peer ID, decode the signature from multibase, and verify it against the SHA-256 digest of the payload bytes. This vector was generated and round-trip verified with the signing code in [boxo v0.35.2](https://github.com/ipfs/boxo/blob/v0.35.2/routing/http/types/record_bitswap.go). + +### Unsigned record + +For servers that skip signature verification (see Signature Verification above), the same announcement without the `Signature` field: + +```json +{ + "Providers": [ + { + "Schema": "bitswap", + "Protocol": "transport-bitswap", + "Payload": {"Keys":["bafkreigur6gzxm3ykiol7ywou3iy3obruzs2q7boizj7oznznid34dzc3e"],"Timestamp":1725833163372,"AdvisoryTTL":86400000000000,"ID":"12D3KooWKcTtoUYcQVfURdwUZtPsanm97SdEf7DkesyxTz7kkD2z","Addrs":["/ip4/198.51.100.1/tcp/4001","/ip4/198.51.100.1/udp/4001/quic-v1"]} + } + ] +} +``` + +The reference server rejects this request with `403 Forbidden` (`signature verification failed`); non-verifying servers such as Bitsocial trackers accept it. -The lack of test fixtures and conformance tests is one of the contributing reasons why this API was never part of official `/routing/v1` specification. +This API never had an official conformance test suite; the historical absence of fixtures is one of the reasons it never became part of the official `/routing/v1` specification. -## Copyright +### Copyright Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). From 83febb2b72167b4743d223b70971e52609126f83 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Sat, 8 Aug 2026 03:01:43 +0200 Subject: [PATCH 4/4] IPIP-0526: document unspecified behavior Answer implementer questions from PR review: request limits, refresh cadence, address policy, deletion, and HTTP-level auth are undefined by the API; record the observed Kubo and index-provider behavior for each so servers stop inferring a contract from one deployment. --- src/ipips/ipip-0526.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/src/ipips/ipip-0526.md b/src/ipips/ipip-0526.md index e96809297..770b5c9c1 100644 --- a/src/ipips/ipip-0526.md +++ b/src/ipips/ipip-0526.md @@ -134,9 +134,11 @@ Where: `ID`, `Timestamp`, and `AdvisoryTTL` are required in practice: a record without `ID` fails signature verification (`403`), and the reference server assumes `Timestamp` and `AdvisoryTTL` are present and crashes on records without them (the client sees a dropped connection instead of an HTTP error). +The `Providers` array can carry multiple records, but the reference client always sends exactly one record per request. Multi-record requests are accepted by the reference server, yet remain unexercised by real clients (see the warning about non-atomic processing under Response Status Codes). + :::note -Implementation detail: At the time of writing this IPIP (2025Q4), [boxo v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) enforces `Schema` (must be `"bitswap"`, returns 400 otherwise) and signature verification (returns 403 on failure). However, `Protocol` is not validated: it is echoed in the response and never passed to the backing service. In [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0), `Timestamp` and `AdvisoryTTL` from the request are ignored: the server uses `time.Now()` for record timestamps and returns its own configured `cidTtl` in the response. Only `Keys`, `ID`, and `Addrs` are used for IPNI announcements, making parts of this specification effectively unused in practice. index-provider also accepts records from only one provider identity per instance (the configured `ProviderID`, or the first peer that publishes when none is configured); correctly signed records from any other peer are rejected with `500` and a `delegate error: provider ... isn't allowed` message. +Implementation detail: At the time of writing this IPIP (2025Q4), [boxo v0.35.2](https://github.com/ipfs/boxo/releases/tag/v0.35.2) enforces `Schema` (must be `"bitswap"`, returns 400 otherwise) and signature verification (returns 403 on failure). However, `Protocol` is not validated: it is echoed in the response and never passed to the backing service. In [index-provider v0.16.0](https://github.com/ipni/index-provider/releases/tag/v0.16.0), `Timestamp` and `AdvisoryTTL` from the request are ignored: the server uses `time.Now()` for record timestamps and returns its own configured `cidTtl` in the response. Only `Keys`, `ID`, and `Addrs` are used for IPNI announcements, making parts of this specification effectively unused in practice. index-provider also accepts records from only one provider identity per instance (the configured `ProviderID`, or the first peer that publishes when none is configured, tracked in memory until restart); correctly signed records from any other peer are rejected with `500` and a `delegate error: provider ... isn't allowed` message. ::: @@ -203,6 +205,16 @@ Where: - `Protocol`: Echoes `"transport-bitswap"`. - `AdvisoryTTL`: TTL in nanoseconds chosen by the server (not the value from the request), indicating how long the record will be cached. Clients SHOULD republish before this TTL expires. +#### Unspecified Behavior + +The API leaves the following undefined. Server implementations differ, and a client has no way to discover a given server's policy. + +- **Request limits**: There is no limit on records per request, keys per record, or body size, and no mechanism for a server to advertise one (no `413` or `429` semantics). The reference server buffers and decodes the whole body in memory; index-provider bounds requests only with read timeouts. In practice, request size is shaped by the client: Kubo sends at most `MaxProvideBatchSize` keys per request (default 100). +- **Address policy**: Nothing defines what a server may do with `Addrs`. The reference server passes them to the backing service verbatim, index-provider forwards them to IPNI unchanged (or ignores them wholesale when `ProviderID` and `Addrs` are configured), and Bitsocial trackers drop announced addresses that do not match the request IP. Filtering, rewriting, and reachability checks are all server-defined. +- **Refresh cadence**: Nothing consumes the response `AdvisoryTTL`: the reference client discards the returned value and re-announces on its own schedule (Kubo defaults to `Provide.DHT.Interval`, 22 hours, which stays under index-provider's default 24-hour expiry only by the choice of defaults). On the server side, index-provider treats a repeated announcement as a refresh: it resets the record's clock to the server's own time and publishes no duplicate IPNI advertisement. +- **Deletion**: There is no way to withdraw an announcement. The only removal path is to stop announcing and let the record expire server-side (index-provider: configured `CidTtl`, default 24 hours, evaluated only while processing later announcements). +- **HTTP authentication**: The record signature is the only authentication mechanism. The API defines no bearer tokens, API keys, or other HTTP-level credentials; operators who need a closed endpoint rely on network-level controls or on checks in the backing service. + ## Design rationale ### Bitswap-Specific Schema