Perf: the count hook is dispatched as a full Lua function call — sandboxes pay ~3x on hot loops

[NeoXider]

Summary

LuaState.SetHook is the only mechanism a sandbox has for enforcing a wall-clock timeout, an
instruction budget and an allocation budget. Because the hook is invoked as a Lua function — a
call stack frame plus an awaited ValueTask per fire — the cost is dominated by the dispatch, not by
whatever the hook does. With an empty hook body at count = 4, a tight numeric loop takes 3.2x
longer.

Measurement

Standalone .NET 8 console (CoreCLR JIT), LuaCSharp v0.5.6 netstandard2.1, Windows 11.
Workload: local x = 0 for i = 1, 2000000 do x = x + i end. Hook installed as SetHook(fn, "", 4)
(the batch a sandbox realistically needs, see below). Median of three runs; 1,000,001 fires.

run	time	vs raw	per fire
raw, no hook	22 ms	1.00x	—
hook, empty body	72 ms	3.16x	~49 ns
hook, body = Stopwatch.GetTimestamp() + GC.GetTotalMemory(false)	127 ms	5.39x	~107 ns

So with an entirely empty body, ~45% of the per-fire cost is the call machinery itself. Under Unity's
Mono the share is far higher — measuring the same guard in the Editor on 0.5.5 we saw ~600 ns/fire
while the in-hook work costs ~60 ns there.

For reference, v0.5.5 measured 30 ms / 94 ms / 165 ms on the same machine, so #330's
LightAsyncValueTaskMethodBuilder work is worth ~25% — thank you — but the shape is unchanged.

Where the cost is

As far as we can read it, LuaVirtualMachine.ExecutePerInstructionHook creates a call stack frame,
pushes "count" + the argument onto the stack, and awaits hook.Func(...). That is a complete Lua
function call per fire, where a C Lua count hook is a plain function-pointer call.

Why a sandbox cannot just widen count

Our guard uses count = 4 deliberately. Time and step budgets are linear and would tolerate a wide
window, but the allocation budget does not: s = s .. s doubles the heap every ~4 instructions,
so a 64-instruction window permits up to ~16 doublings between two heap samples — out of memory long
before the next check. Safety granularity therefore trades directly against throughput, and the
exchange rate is set by the dispatch cost.

Ask

Would you consider a hook path that does not go through a Lua function call? Either would work for us:

1. A lightweight delegate overload, e.g. SetHook(Func<LuaState, bool> onCount, int count) (or
   Action), invoked directly from the dispatch loop with no call frame and no async state machine;
   returning false / throwing LuaRuntimeException aborts, as today.
2. First-class budget APIs — state.InstructionBudget / state.Deadline checked inline in the
   interpreter loop, raising a LuaRuntimeException on trip. This is what most embedders actually
   want from a count hook, and it would let the VM check a counter instead of calling out.

Two related observations (verified on v0.5.6; no action needed if they are intended)

1. SetHook from inside the hook works. A hook that re-arms itself with SetHook(self, "", 64)
   on its first fire drops from 1,000,001 fires to exactly 62,501 and throws nothing. We would like to
   rely on this for an adaptive window (widen it while the heap is far from the budget), so it would
   help to know whether it is supported behaviour worth documenting.
2. The count hook does not fire while a host (C#) function runs. A slow_host_call() that busies
   the CPU for 300 ms inside C# produced 0 fires between entry and exit. That means a wall-clock
   deadline enforced from the hook cannot cut a long host call — an important caveat for anyone using
   the hook as a watchdog. Documenting it, or offering an optional check at the host-call boundary,
   would close the hole.

Environment

• Numbers above: Windows 11, .NET 8 CoreCLR, LuaCSharp 0.5.6 (netstandard2.1).
• Production target: Unity 6000.3, Mono and IL2CPP, including single-threaded WebGL.
• Context: CoreAI uses Lua-CSharp as its modding VM for
  AI- and player-authored scripts, so every handler runs under this guard.

Thanks also for #329 and #331 landing in 0.5.6 — both were blockers for us and both are now verified
fixed on our side.