Say which stage of reaching the LLM endpoint failed - #186
Merged
Conversation
An endpoint that could not be reached reported "LLM endpoint did not respond
within {TimeoutSeconds}s" — the same sentence a slow model produces. One budget
covered DNS, TCP, TLS and the model's answer alike, so with a profile at 360 s
an operator waited six minutes and was then pointed at the model while the real
cause was a firewall or an untrusted certificate. Four causes, one message,
none of them named.
Reaching an endpoint is not the same kind of work as waiting for a model to
think, so it no longer shares the budget. LlmConnectGuard gives name resolution
and the TCP connect 15 s together and fails with the stage in the message;
SocketsHttpHandler.ConnectTimeout bounds the rest at 30 s, which is the only
place that can bound the TLS handshake — the callback hands back the raw
transport stream and the handler negotiates on top of it.
The two are ordered on purpose and a test pins it: because DNS and TCP always
expire on their own shorter deadline, a ConnectTimeout that fires can only mean
the handshake stalled, and DescribeUnreachable says so instead of guessing.
Certificate rejections are named separately and point at the machine trust
store, since a CA that a browser accepts on a workstation is not thereby
trusted by the service account.
The answer-timeout message now states what it knows: the request was on the
wire and the model went quiet — the one case where raising TimeoutSeconds is
the right answer.
Also: the settings probe kept only HttpRequestException's own generic message
and dropped the inner exception, which is where the stage or the certificate
error actually is. Its client timeout moves 30 s -> 40 s so the handler's
named stage wins the race instead of a bare "HttpClient.Timeout elapsed".
Resolved addresses and per-stage timings go to Debug under a named category,
NodePilot.Ai.LlmConnect, so the "works from my machine, not from the service"
case can be settled by log rather than by guesswork.
Sev7eNup
added a commit
that referenced
this pull request
Aug 11, 2026
Carries the LLM connect diagnostics (#186): reaching an endpoint no longer shares its budget with the model's answer, and a failure names the stage it failed at — DNS, TCP, the TLS handshake, or the certificate — instead of reporting every one of them as "did not respond within {TimeoutSeconds}s". Bumped through `npm version --no-git-tag-version` per package rather than a text replace over the tree, so only the top-level version and its lockfile root entry move.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
An LLM endpoint that could not be reached reported "LLM endpoint did not respond within {TimeoutSeconds}s" — the same sentence a slow model produces. A single budget covered DNS, TCP, TLS and the model's answer, so on a profile with
TimeoutSeconds: 360an operator waited six minutes and was then pointed at the model, while the actual cause was a firewall or an untrusted certificate.This came out of a real case:
https://litellm-test.<internal>/v1, same network as NodePilot, no proxy involved. Both the settings probe and the chat failed, and the message said nothing that narrowed it down. Four distinct causes, one sentence, none of them named.This does not fix that installation's connectivity — it makes the cause visible in seconds instead of minutes, and names it.
What
Reaching an endpoint is not the same kind of work as waiting for a model to think, so it no longer shares the budget.
LLM endpoint DNS:LLM endpoint TCP:— distinguishes dropped (firewall) from refused (nothing listening), and lists the addresses actually triedLLM endpoint TLS:LLM endpoint TLS: … certificate— points at the machine trust storeTimeoutSecondsaccepted the request but sent no answerLlmConnectGuardowns DNS and TCP and fails with the stage named.SocketsHttpHandler.ConnectTimeoutbounds the rest — the only place that can bound the TLS handshake, because the callback hands back the raw transport stream and the handler negotiates on top of it.The ordering is load-bearing and a test pins it. Because DNS and TCP always expire on their own, shorter deadline, a
ConnectTimeoutthat fires can only mean the handshake stalled — that is howDescribeUnreachablenames the TLS stage rather than guessing. If those two constants ever cross, the inference silently becomes wrong, soHandshakeTimeout_ExceedsTheConnectPhaseBudgetfails the build instead.Certificate rejections are called out separately and say where the CA belongs: a certificate a browser accepts on a workstation is not thereby trusted by the service account.
Deliberately no new configuration key. 15 s is far beyond any healthy lookup or handshake — Windows abandons an unanswered SYN after ~21 s on its own — so a value that would need raising means the network is broken, which is now exactly what the message says.
Two smaller things in the same pass:
HttpRequestException's generic "An error occurred while sending the request" and dropped the inner exception, which is where the stage or the certificate error lives. Its client timeout moves 30 s → 40 s so the handler's named stage wins the race instead of a bare "HttpClient.Timeout elapsed".Debugunder the named categoryNodePilot.Ai.LlmConnect, switchable on its own viaSerilog:MinimumLevel:Override. That is the line that settles "works from my machine, not from the service" — a stale AAAA record and a different DNS suffix look identical from the outside.Verification
ConnectCallback, 5 on the classification).dotnet buildclean, no new warnings.