assignment1: implement load/isContiguous/view/permute/slice - #57
Open
2595001965 wants to merge 3 commits into
Open
assignment1: implement load/isContiguous/view/permute/slice#572595001965 wants to merge 3 commits into
2595001965 wants to merge 3 commits into
Conversation
实现 argmax、embedding、linear、rms_norm、rope、self_attention、swiglu 的 CPU 版本，均支持 Float32、Float16、BFloat16 三种数据类型。 实现要点： - 统一在算子入口做设备/dtype/形状/连续性校验，非 CPU 设备保留 CUDA 分支占位。 - f16/bf16 先一次性解码为 float 再计算：类型转换函数在独立编译单元中无法内联， 放进最内层循环会带来大量跨模块调用开销。 - argmax 只在严格大于时更新下标，与 torch.max 返回首个最大值的行为一致； 最大值按字节整元素拷贝，保证与输入按位相等。 - embedding 按整行 memcpy，天然支持任意 dtype 并保持按位一致。 另外将 model/ 与 *.safetensors 加入 .gitignore，避免提交下载的模型权重。 test/ops 下 7 个算子测试全部通过。
Add a C++ Qwen2 model behind the llaisysQwen2Model C API, wired up through ctypes to python/llaisys/models/qwen2.py. Weights are allocated in the backend and loaded from safetensors by handing the raw bf16 bytes to tensorLoad, since the numpy backend cannot represent bf16 and torch is not allowed in the wrapper. KV cache is stored per layer as [maxseq, nkvh, dh]. K/V for new tokens are projected directly into their cache slots, so attention reads the whole history with no copying and each decode step only computes the new token. Also make linear usable at model scale: it previously decoded the entire weight matrix to float per call, which is 933MB for lm_head. It now decodes one weight row at a time and parallelizes over output columns. The fp16/bf16 conversion helpers moved inline into types.hpp, which is what made per-element decoding cheap enough to do this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.