router: Future-Proof Non-HTTP actor protocols

[keithmattix]

Motivation

atenet router's primary function is to receive an incoming request, parse the actor and ate space name from the Host header, and resume the actor before forwarding the request to the worker IP hosting that actor. While this is currently HTTP-only, the current hostname-style addressing scheme leaves open the possibility of supporting non-HTTP actor protocols by using TLS (the router uses SNI to parse the atespace:actor tuple). Even this has issues though as it's not possible to front a single wildcard TLS certificate for actors as the DNS name has two wildcarded components (..actors.resources.substrate.dev). In order to pave the way for HTTP-specific actor addressing schemes (e.g. paths or headers), this issue proposes a strategy to support actors with non-HTTP protocols.

Proposal

Substrate ingress standardizes on HTTP, HTTPS, and HTTP/2 (TLS and h2c) as the supported transports for calling an actor. Clients will explicitly dial the ingress load balancer (likely via a DNS name provisioned out-of-band) since the encoding will occur outside of the host header (exact shape tbd; we can discuss in #445). To support actors that don't speak HTTP, the ingress must support terminating CONNECT and forwarding the opaque TCP packets from tunnel. In this model, the ingress will be able to have a single parsing scheme for determining the atespace;actor tuple regardless of actor protocol.

Happy to take a stab at POCing this if we think it's a good idea

[bowei]

1. Need for double wildcard in DNS -- this feels like a separate issue from the non-HTTP actor protocol support? We might want to split that one off to its own discussion.

2. This is an interesting approach -- I'm assuming this would be in addition to the standard 80, 443 that we expose today. CONNECT would allow arbitrary ports, protocols without having to play games with the router setup. The router setup using K8s Service for instance becomes very awkward due to lack of support for ALL_PORTS etc.

This would enable something like?

$ export http_proxy=my-ate-router.ate.dev:8443
$ my-protocol-client my-actor.my-atespace.router.ate.dev:12345

We would end up with:

• 80, 443 on the router remains an easy to use front door. Note: we need to fix having 443 go to 443 on the Actor for consistency.
• You can use standard HTTP CONNECT (v1, v2, ...) to 8443 to use CONNECT tunneling for an opaque stream.

Right now the fields that would be interpreted would just be the :authority and related fields.

[bowei]

Lior Lieberman (@LiorLieberman)

[keithmattix]

1. That's fair; it is distinct from non-HTTP actor. That being said, I added it because I felt like non-HTTP actors was one of the reasons we were wanting to rely on DNS, so wanted to include it for context.
2. Yes, this would be an additional CONNECT port, and yes we could make ~whatever :authority conventions we want inside the tunnel because we're free from the limitation of being able to properly dial the actor.