Skip to content

[Feature]: Greater CUTLASS Integration #16

Closed

Description

@Ra4ster

What problem does this solve?

  • cudaMalloc and cudaFree round-trip passes in energy/sum reduce
  • Sequential softmax
  • Unoptimized activations

Proposed solution

  • Persistent scalar scratch buffer — kills the cudaMalloc/cudaFree round-trip in the energy/sum reductions.
  • Softmax kernel parallelized — one block per row instead of one thread, with a caught-and-fixed race in the reduction.
  • GELU/Tanh/Sigmoid CUTLASS epilogues.

Alternatives considered

No response

Area

CUDA backend

Checklist

  • I checked Roadmap.md and this isn't already tracked there.
  • I'm willing to help implement this (optional, just helps with prioritization).

Activity

  1. self-assigned this
    on Oct 4, 2026
  2. linked a pull request that will close this issueGraph capture fix #17on Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions