BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
PR SummaryMedium Risk Overview A new Supporting tweaks: Reviewed by Cursor Bugbot for commit 85c859a. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 48906c1. Configure here.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure #8895 +/- ##
====================================================================================================
Coverage ? 81.98%
====================================================================================================
Files ? 168
Lines ? 25146
Branches ? 717
====================================================================================================
Hits ? 20616
Misses ? 4323
Partials ? 207
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will degrade performance by 27.89%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
48906c1 to
c5a86df
Compare
8efe84c to
a0c8ad1
Compare
c5a86df to
feac27a
Compare
a0c8ad1 to
0ddf199
Compare
0ddf199 to
d051ddb
Compare
feac27a to
5f1207d
Compare
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1001 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1526 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 51 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 107 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 25 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
5f1207d to
d2c1bc0
Compare
d051ddb to
996e625
Compare
d2c1bc0 to
b96a707
Compare
996e625 to
0a0050d
Compare
0a0050d to
f03d9a1
Compare
b96a707 to
85c859a
Compare
| // MIR ops exist but are unconstructible (BinOp variants carry `!`) | ||
| &T![::core::math::mul] | ||
| | &T![::core::math::div] | ||
| | &T![::core::math::rem] | ||
| | &T![::core::math::r#mod] | ||
| | &T![::core::math::pow] | ||
| | &T![::core::bits::xor] | ||
| | &T![::core::bits::shl] | ||
| | &T![::core::bits::shr] => None, |
| // math functions without direct MIR ops | ||
| &T![::core::math::sqrt] | &T![::core::math::cbrt] | &T![::core::math::root] => None, |


🌟 What is the purpose of this PR?
Allows intrinsics to be used in value position (passed as arguments, bound to variables) by generating synthetic MIR wrapper bodies during reification. Previously, intrinsics like
<=could only appear at call sites where specialization rewrites them. Now they can flow through higher-order functions:🔍 What does this change?
Synthetic body generation:
Syntheticsstruct inCrossCompileStatemanages creation and caching of wrapper bodiesSyntheticBuildergenerates MIR bodies with fat closure ABI (unit env as first parameter)T!macro for matching qualified paths againstConstantSymbolarrays:&T![::core::cmp::gt]in slice patterns,sym::path::core::cmp::gt::CONSTfor the namebinary!/unary!macros deriving path and name from the same segmentsThin-call specialization:
rvalue_call_thin_specializerecognizesCall(Thin, Qualified(intrinsic), [])(the administrative thunk-force from the thunking phase) and produces the closure aggregate directly, skipping thunk body generation() -> ClosureTypewrapper to extract the actual signatureIntrinsic classification:
gt,lt,gte,lte,eq,ne), boolean (and,or,not), arithmetic (add,sub), bitwise (and,or,not)!payload):mul,div,rem,mod,pow,xor,shl,shrsqrt,cbrt,rootgraph::head::entities,graph::body::filter,graph::tail::collectproduce a specific user-facing diagnosticDiagnostics:
intrinsic_not_first_class: user-facing error for graph intrinsics in value positionsynthetic_binary_arity_mismatch/synthetic_unary_arity_mismatch: ICE for monomorphized type invariant violationsPre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
"<="as the entire program) is not supported. The thunking phase does not wrap the root body when it is a variable, so it never reaches the synthetic machinery. Tracked with arun: skiptest.🐾 Next steps
🛡 What tests cover this?
Compiletests (8 new):
reify/synthetic-intrinsic-value: basic synthetic body generation and thin-call specializationreify/synthetic-intrinsic-reused: same intrinsic used twice shares one body (caching)reify/synthetic-intrinsic-multiple: different intrinsics get separate bodiesreify/synthetic-graph-not-first-class: graph intrinsic in value position produces errorreify/synthetic-intrinsic-bare: bare intrinsic at root (run: skip)interpret/synthetic-intrinsic-value: end-to-end correctness (evaluates totrue)post_inline/synthetic-intrinsic-hof: full optimization cascade with constants (collapses toreturn true)post_inline/synthetic-intrinsic-hof-dynamic: HOF with dynamic inputs collapses to barex <= y❓ How to test this?
cargo run --package hashql-compiletest -- run --filter "test(synthetic)"📹 Demo
The
post_inline/synthetic-intrinsic-hof-dynamictest shows the full cascade. A higher-order function receiving<=as a value with dynamic inputs:All indirection eliminated. Indistinguishable from writing
["<=", x, y]directly.