perf: optimize execution tree by dkharms · Pull Request #578 · ozontech/seq-db

dkharms · 2026-10-15T19:26:23Z

Description

This PR fixes two problems in how AND subtrees of a search query are evaluated against a fraction:

suboptimal order of resolving token expressions into set of TID in AND subtrees;
suboptimal shape of AND subtrees in the eval tree;

`TID` resolve order

The order in which leaves are resolved into set of TID is effectively chosen by the user: it follows the query as written. This hurts needle searches over wide time ranges.

The cheapest strategy is to resolve the needle first: most likely it is absent from the fraction, so the whole AND subtree collapses and the fraction is skipped without touching the other leaves.

For example, in service:foo-* AND trace_id:needle it is much cheaper to resolve trace_id first and skip most fractions right away. In the same way, a *substring* leaf next to a needle is never scanned in fractions that do not contain the needle.

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`			`total`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`service:payment-backend-eu and trace_id:00000000000000000000000000000000`	hot	`11.31`	`7.52`	`-33.50%`	`1.19`	`0.85`	`-28.52%`	`11.47`	`7.74`	`-32.49%`	`13.59`	`8.88`	`-34.68%`	`107.25`	`44.77`	`-58.26%`	`30`	`30`	`0.00%`	`0`	`0`	`0.00%`
	cold	`1002.43`	`868.63`	`-13.35%`	`88.75`	`94.87`	`+6.90%`	`978.44`	`844.26`	`-13.71%`	`1153.41`	`1031.82`	`-10.54%`	`1171.55`	`1037.10`	`-11.48%`	`29`	`30`	`+3.45%`	`0`	`0`	`0.00%`
`trace_id:*deadbeefdeadbeefdead and transaction_id:tx-needle-zzzz`	hot	`1693.33`	`7.46`	`-99.56%`	`23.02`	`0.80`	`-96.54%`	`1704.37`	`7.27`	`-99.57%`	`2156.56`	`8.90`	`-99.59%`	`2156.56`	`54.13`	`-97.49%`	`17`	`30`	`+76.47%`	`0`	`0`	`0.00%`
	cold	`2945.21`	`680.82`	`-76.88%`	`41.23`	`77.69`	`+88.42%`	`2938.76`	`658.27`	`-77.60%`	`5482.67`	`1606.03`	`-70.71%`	`5482.67`	`2368.57`	`-56.80%`	`15`	`29`	`+93.33%`	`0`	`0`	`0.00%`

`AND` shape

If an AND subtree has conjuncts $c_1, c_2, \dots, c_n$, it is better to build the tree so that as few documents as possible flow up from the innermost node.

Conjuncts are sorted by size, which is the sum of the token frequencies stored in token blocks, and chained as $((c_1 \land c_2) \land c_3) \land \dots$. This way the innermost AND produces the fewest candidates for the outer ones to check.

Tokens below the sealing frequency threshold report zero and are treated as the rarest. Fractions without frequencies (active fractions, legacy formats) keep the query order.

Query	Type	`mean (ms)`			`stddev (ms)`			`p(50) (ms)`			`p(95) (ms)`			`p(99) (ms)`			`iterations`			`total`
Query	Type	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff	base	comp	diff
`level:6 and k8s_container:api-gateway and service:api-gateway-eu`	hot	`62.16`	`56.56`	`-9.01%`	`6.62`	`4.61`	`-30.35%`	`61.00`	`56.85`	`-6.79%`	`73.25`	`65.03`	`-11.22%`	`122.53`	`114.93`	`-6.20%`	`30`	`30`	`0.00%`	`4409665`	`4409665`	`0.00%`
	cold	`996.81`	`1081.46`	`+8.49%`	`139.47`	`139.55`	`+0.06%`	`1007.32`	`1086.48`	`+7.86%`	`1231.62`	`1251.50`	`+1.61%`	`3600.31`	`1257.20`	`-65.08%`	`27`	`28`	`+3.70%`	`4409665`	`4409665`	`0.00%`
`level:6 and k8s_container:api-gateway and service:api-gateway-eu and k8s_pod:api-gateway-us-4`	hot	`23.95`	`16.50`	`-31.12%`	`4.46`	`1.92`	`-56.94%`	`25.18`	`16.12`	`-35.98%`	`31.36`	`26.30`	`-16.13%`	`89.18`	`76.58`	`-14.13%`	`30`	`30`	`0.00%`	`0`	`0`	`0.00%`
	cold	`1049.88`	`844.49`	`-19.56%`	`109.51`	`105.79`	`-3.40%`	`1038.33`	`858.17`	`-17.35%`	`1219.10`	`979.49`	`-19.65%`	`1230.57`	`989.03`	`-19.63%`	`28`	`30`	`+7.14%`	`0`	`0`	`0.00%`

If you are struggling to build intuition check this out

level:6                    723 959
k8s_container:api-gateway  351 651
service:api-gateway-eu      51 317
k8s_pod:api-gateway-us-4     4 930

main: (((level ∧ container) ∧ service) ∧ pod)
levels: 3 2 1 1 1 2 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 1 2 1 1 1 1 1 1 2 … (+995)

new: (((pod ∧ service) ∧ container) ∧ level)
levels: 3 2 1

This is trace of post-order tree iteration. As you can see the the price of tree-traversal is reduced dramatically.

I have read and followed all requirements in CONTRIBUTING.md;
I used LLM/AI assistance to make this pull request;

If you have used LLM/AI assistance please provide model name and full prompt:

Model: {{model-name}}
Prompt: {{prompt}}

codecov-commenter · 2026-10-15T19:37:07Z

Codecov Report

❌ Patch coverage is 93.63057% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.18%. Comparing base (db7c170) to head (5cc3063).

Files with missing lines	Patch %	Lines
frac/processor/resolver.go	94.77%	7 Missing ⚠️
frac/processor/search.go	88.23%	2 Missing ⚠️
frac/processor/batch_eval_tree.go	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #578      +/-   ##
==========================================
+ Coverage   80.09%   80.18%   +0.09%     
==========================================
  Files         261      262       +1     
  Lines       19769    19910     +141     
==========================================
+ Hits        15834    15965     +131     
- Misses       3935     3945      +10

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2026-10-15T19:45:01Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`SearchLeaves/common_word_literal_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`127363.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`151.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`102929.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/common_word_literal_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`5466.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`84.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`6998.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/cooccur_empty/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`1979612.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`257.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`685454.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/cooccur_empty/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`27064.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`175.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`25065.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/literal_contains_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`55386.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`131.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`53687.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/literal_contains_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`5326.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`78.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`6048.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/lone_literal_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`55213.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`124.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`53549.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/lone_literal_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`5143.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`71.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`5727.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/or100_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`173141.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`886.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`231273.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/or100_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`44333.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`811.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`89357.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/or100_word_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`129903.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`564.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`118425.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/or100_word_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`21068.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`496.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`23143.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_dense_rare_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2215912.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`319.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`844325.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_dense_rare_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`28214.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`211.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`29912.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_dense_rare_last/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2196697.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`318.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`841384.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_dense_rare_last/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`28273.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`211.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`29787.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_wms_literal_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2570130.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`765.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`994866.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_wms_literal_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`59593.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`664.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`78379.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_wms_literal_last/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2564195.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`765.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`1000676.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/order_wms_literal_last/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`59818.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`664.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`77911.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_dense_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2206348.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`345.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`2345525.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_dense_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`42227.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`213.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`1439701.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_level_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`1966555.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`777.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`5616910.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_level_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`78168.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`675.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`4761231.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_sparse_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2216089.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`345.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`2339356.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_sparse_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`42122.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`213.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`1448018.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_wms_first/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`1949237.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`777.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`5620360.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/scanall_wms_first/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`78258.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`675.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`4780880.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/service_literal_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`55528.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`137.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`54762.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/service_literal_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`5465.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`84.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`6876.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_contains_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`3797304.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`1420.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`12899683.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_contains_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`18414.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`104.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`2127265.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_literal_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`55387.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`131.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`53665.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_literal_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`5326.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`78.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`6045.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_literal_present/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2562681.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`765.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`998346.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_literal_present/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`59604.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`664.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`78674.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_suffix_absent/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`4295443.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`1637.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`11271995.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/wms_suffix_absent/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`27862.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`106.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`1127664.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/words_absent_last/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`83065.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`170.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`61618.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/words_absent_last/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`6361.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`113.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`10410.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/words_present/cold-4`	`------`	`c8a66f`
	`NaN B/op`	`2412394.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`389.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`1303549.00 ns/op`	`NaN`	`🔴`
`SearchLeaves/words_present/warm-4`	`------`	`c8a66f`
	`NaN B/op`	`30419.00 B/op`	`NaN`	`🔴`
	`NaN allocs/op`	`256.00 allocs/op`	`NaN`	`🔴`
	`NaN ns/op`	`252250.00 ns/op`	`NaN`	`🔴`

github-actions · 2026-10-27T15:51:42Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`Indexer-4`	`64e2ef`	`bdac2d`
	`564879420.00 B/op`	`663696870.00 B/op`	`1.17`	`🔴`

github-actions · 2026-11-09T08:32:47Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`Indexer-4`	`db7c17`	`faedd6`
	`676128822.00 B/op`	`751609853.00 B/op`	`1.11`	`🔴`

dkharms added the performance Features or improvements that positively affect seq-db performance label Oct 15, 2026

dkharms force-pushed the 0-lazy-tid-resolve branch from b40c2a6 to 49cdcf5 Compare October 15, 2026 19:31

eguguchkin added this to the v0.82.0 milestone Oct 19, 2026

eguguchkin requested review from cheb0 and eguguchkin October 19, 2026 10:59

cheb0 approved these changes Oct 21, 2026

View reviewed changes

dkharms force-pushed the 0-lazy-tid-resolve branch from 49cdcf5 to 1b2fbbf Compare October 27, 2026 12:06

eguguchkin modified the milestones: v0.82.0, v0.83.0 Nov 2, 2026

perf: optimize execution tree

5cc3063

dkharms force-pushed the 0-lazy-tid-resolve branch from 1b2fbbf to 5cc3063 Compare November 9, 2026 08:23

Navigation Menu

Repository navigation

perf: optimize execution tree - #578

perf: optimize execution tree#578
dkharms wants to merge 1 commit into
mainfrom
0-lazy-tid-resolve

dkharms commented Oct 15, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

github-actions Bot commented Nov 9, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

dkharms commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

TID resolve order

AND shape

Uh oh!

codecov-commenter commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Oct 15, 2026

🔴 Performance Degradation

Uh oh!

github-actions Bot commented Oct 27, 2026

🔴 Performance Degradation

Uh oh!

github-actions Bot commented Nov 9, 2026

🔴 Performance Degradation

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dkharms commented Oct 15, 2026 •

edited

Loading

`TID` resolve order

`AND` shape

codecov-commenter commented Oct 15, 2026 •

edited

Loading