add blazingly fast instance_cache implementation for K2 - #1683
Open
denisichh wants to merge 28 commits into
Open
add blazingly fast instance_cache implementation for K2#1683denisichh wants to merge 28 commits into
denisichh wants to merge 28 commits into
Conversation
denisichh
force-pushed
the
dzubarev/k2/add-instance-cache
branch
from
August 28, 2026 09:43
454d684 to
80eabdd
Compare
denisichh
force-pushed
the
dzubarev/k2/add-instance-cache
branch
from
September 8, 2026 09:16
4ebc228 to
81cee3f
Compare
PetrShumilov
requested changes
Sep 9, 2026
| } | ||
|
|
||
| void ClassDeclaration::compile_class_name_hash(CodeGenerator& W, ClassPtr klass) { | ||
| // hash of the class name, computed once at compile time -- same for every instance, |
Contributor
There was a problem hiding this comment.
I see that it would be better not to introduce a new field. We already have get_hash in class_instance, which can be helpful, but it has a specific corner case: class_instance<T>{}.get_hash() == nullptr, while class_instance<T>{}.alloc().get_hash() != nullptr. I think you have two possible options:
- Adjust the existing get_hash to provide the required semantics.
- Add a new method (e.g., get_type_hash or similar).
|
|
||
| compile_class_method(FunctionSignatureGenerator(W).set_const_this(), klass, "size_t virtual_builtin_sizeof()", "sizeof(*this)"); | ||
|
|
||
| compile_class_method(FunctionSignatureGenerator(W).set_const_this(), klass, "size_t virtual_builtin_alignof()", "alignof(" + klass->src_name + ")"); |
| // memory must be aligned to alignof(T) and >= estimate_memory_usage() bytes | ||
| // caller must pin it with a special ExtraRefCnt (e.g. for_instance_cache), since the instance never frees it. | ||
| // Returns a null instance if memory is unfit. | ||
| inline class_instance clone_in(vk::span<std::byte> memory) const noexcept; |
Contributor
There was a problem hiding this comment.
Do we have an ability to return something like variant?
| // constructs an instance in externally provided memory (no allocation/ownership) | ||
| // leaves it null if memory is smaller than sizeof(T) or misaligned | ||
| template<class... Args> | ||
| inline class_instance<T> alloc(vk::span<std::byte> memory, Args&&... args) noexcept __attribute__((always_inline)); |
Contributor
There was a problem hiding this comment.
I think that would be better to rename into alloc_in based on 2 reasons:
- More regular and coherent with clone_in
- Do not produce a new overload
| std::enable_if_t<std::is_polymorphic<S>{}, class_instance> virtual_builtin_clone_in(vk::span<std::byte> memory) const noexcept { | ||
| class_instance res; | ||
| if (o) { | ||
| if (unlikely(memory.size() < o->virtual_builtin_sizeof() || reinterpret_cast<std::uintptr_t>(memory.data()) % o->virtual_builtin_alignof() != 0)) { |
Contributor
There was a problem hiding this comment.
I recommend to add nullptr check for memory.data()
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites instance_cache_store/fetch/update_ttl/delete in the K2 runtime: instead of msgpack-serializing and shipping instances to a separate cache process over RPC, instances are now deep-copied directly into shared memory via the platform's k2::alloc_shared_memory/publish_shared_memory/get_shared_memory.
Storage layout: class_name_hash(u64) | class_instance shell | inner data. On fetch, the block is reinterpreted in place rather than deserialized.
Key pieces