Proposal: dynamic/lazy resources/list (and directory-read) backing for gateway/proxy and template-served catalogs

[sambhav]

Problem

resources/list can only be served from a static, in-memory set. Resources are materialized up front via AddResource (removed via RemoveResources), and (*Server).listResources enumerates that featureSet directly — there is no handler/callback that computes the list on demand. For two important classes of server this isn't just inconvenient, it's a structural mismatch.

1. Gateway / proxy servers (the important case)

A skill (or resource) gateway fronts one or more upstream catalogs it does not own — another registry, a remote index, or several backend MCP servers. The authoritative list lives elsewhere and changes independently of this process. To serve resources/list from a static set, the gateway would have to mirror the entire upstream catalog into memory at startup and keep it continuously in sync — which defeats the purpose of being a gateway, doesn't scale to large/unbounded upstreams, and is racy against upstream changes.

What a gateway actually needs is to answer resources/list by querying upstream on demand, passing the client's pagination cursor through. There is no first-class way to do that today.

2. Template-served assets are non-enumerable by construction

Skills are served as a directory of files. A server typically exposes a skill's files via a resource template (skill://<skill>/{+path}) plus one read handler, so it doesn't have to AddResource every individual file. But a URI template matches infinitely many URIs — you cannot derive the file list from the template definition. So those assets are fully readable yet absent from any static set: resources/list won't show them, and resources/directory/read (below) backed by the static set returns empty.

Relationship to resources/directory/read (#956 / SEP-2640)

The Skills-over-MCP WG is adding a resources/directory/read method (SEP-2640) so hosts can enumerate the files within a skill directory — e.g. to materialize a skill to disk. Implementing that verb in this SDK depends on custom-method support (#956). But even once the verb exists, it hits exactly this problem: for a template-served or gateway-proxied skill, the directory's children cannot come from the static set — resources/directory/read must be backed by a server-provided dynamic listing.

So the two are a pair:

• mcp: Allow registration of custom JSON-RPC methods #956 — adds the resources/directory/read verb (custom method).
• this issue — gives resources/list / directory-read a dynamic backing so template-served and gateway-proxied entries can actually be enumerated.

Neither alone lets a Go skills/gateway server both serve and enumerate a catalog whose contents aren't pre-materialized.

What already works (to scope precisely)

resources/read is already dynamic — content comes from a ResourceHandler, and resource templates (AddResourceTemplate + ResourceTemplate.Matches) let one handler serve an unbounded URI space. The asymmetry is the whole point: templates make reads unbounded, but nothing makes listing/enumeration unbounded or on-demand.

Workarounds today (and why they fall short)

1. Receiving middleware. Middleware wraps MethodHandler func(ctx, method string, req Request)(Result, error), so middleware can intercept resources/list and synthesize a *ListResourcesResult, short-circuiting the static handler. It works, but it's untyped (switch on a string method, type-assert req/Result), bypasses the SDK's built-in pagination, and sits awkwardly beside the otherwise-typed registration model.
2. Per-session server via the getServer callback (as suggested in Support for per session dynamic tools/prompts #216). Good for per-connection configuration, but it still requires materializing a concrete set per server — it doesn't help a catalog that is upstream, generated, or too large to enumerate eagerly.

Proposal

Add a first-class way to serve resources/list (and back resources/directory/read) from a handler, with cursor-based pagination handled by the SDK — analogous to how ResourceHandler/templates already make reads dynamic. Illustrative shape (not prescriptive):

// Serves resources/list on demand; receives the request (incl. pagination cursor)
// and returns a page of resources plus the next cursor.
type ListResourcesHandler func(context.Context, *ListResourcesRequest) (*ListResourcesResult, error)

opts := &mcp.ServerOptions{
    ListResourcesHandler: func(ctx context.Context, req *mcp.ListResourcesRequest) (*mcp.ListResourcesResult, error) {
        // e.g. proxy upstream using req.Params.Cursor, or enumerate a template's namespace ...
    },
}

When set, the handler would take precedence over (or compose with) the registered static set, and the same mechanism would back directory enumeration for a directory URI.

Questions for maintainers

• Was the static-set model for resources/list a deliberate choice (uniformity with tools/prompts feature sets), and would a dynamic list handler be acceptable — or is middleware / per-session-server the intended long-term answer for gateway and unbounded catalogs?
• If a handler is acceptable: should it replace or merge with the registered set, and should the same backing apply to resources/directory/read (mcp: Allow registration of custom JSON-RPC methods #956) and, for consistency, prompts/list / tools/list?

Happy to prototype if there's interest in the direction.

[piyushbag]

Planning to open a prototype PR for this proposal.

Approach:

• Add ListResourcesHandler on ServerOptions, mirroring the shape suggested in the issue
• When set with no static resources: pass client pagination cursors through to the handler (gateway/proxy case)
• When set with static AddResource entries: SDK paginates the static set first, then invokes the handler for subsequent pages
• Advertise the resources capability when the handler is set

resources/directory/read backing is out of scope until custom-method support (#956) lands; happy to extend once that verb exists.

[rehan243]

fwiw, we ran into almost the same headache with our skill gateway setup — trying to keep a local featureSet synced with upstream catalogs was just asking for trouble, especially once you’ve got multiple backend MCPs that update faster than you can poll. Honestly, mirroring the catalog in-memory gets ugly fast at scale; we had to hack around it by wrapping listResources with a custom handler that just called upstream and streamed results, but it was a mess because the SDK doesn’t let you plug in dynamic listing properly.

Pagination cursors are another pain point — turns out, passing them transparently to upstream works but only if the list handler’s stateless. The docs don’t really mention how to do this with templates (like skill://<skill>/{+path}) so we had to skip full enumeration and just serve file reads directly. That’s fine for single files, but as soon as clients want directory listings, you’re stuck.

We ended up moving to a lazy fetch pattern with gRPC calls on demand. Not gonna lie, took us a few tries to get resource removal working cleanly without stale entries hanging around. If you’re planning to handle unbounded upstreams, static sets just won’t cut it — you really need a pluggable resource lister.

[piyushbag]

Thanks for sharing the gateway war story, this matches what we hit reading #998.

I opened a prototype in #1002 that adds ListResourcesHandler on ServerOptions:

• handler-only servers get client pagination cursors passed through unchanged (gateway/proxy case)
• static AddResource entries are paginated first, then the handler runs for later pages

resources/directory/read for template directory enumeration is still deferred until #956 lands. Resource removal / stale catalog entries feel like a separate concern (listChanged / subscriptions) rather than the list handler itself.

Curious whether your gRPC lazy-fetch pattern would map cleanly onto the handler-only + cursor-passthrough path once this API exists.

[rehan243]

hm, yeah that version issue with pagination cursors sounds spot-on. we’ve actually had a similar headache with stale entries and listChanged hooks — kinda felt like a separate layer, but your approach seems cleaner. curious if you’ve tested it with larger-scale catalogs yet? wonder if the cursor passthrough holds up well under high churn.

[piyushbag]

@rehan243 Good question. So far I have only unit-level coverage: static plus dynamic pagination, cursor passthrough on handler-only servers, and the static-then-handler page split. I have not stress-tested a catalog that is changing quickly or wired up listChanged subscriptions on top of this yet. That still feels like a separate concern from the list handler itself.

If you have a concrete churn pattern (resources added or removed between pages, or upstream revving faster than the client walks the list), I would like to see it. It would help clarify whether transparent cursor passthrough is enough in practice or whether the handler contract needs an explicit invalidation hook.

The gateway lazy-fetch path you described maps pretty closely to what #1002 is aiming at. Happy to compare notes if you run into edge cases on your side.