Source review in progress

Sparse Attention, State Space Models & Linear Transformers

Mamba 2, RWKV, FlashAttention-3, sub-quadratic attention, and hybrid state-space transformer backbones

This brief is being checked against individual publications, benchmarks and methods. Its earlier generated source list and confidence labels did not meet the publication standard. The topic remains available here while the evidence is reviewed.

Review the method Explore the hubs

Context Engineering & Long-Context Architecture AI Inference Optimization Runtimes & Serving Engines NVIDIA Blackwell & Rubin GPU Architecture Mixture-of-Experts (MoE) & Multi-Head Latent Attention

Sparse Attention, State Space Models & Linear Transformers

Related topics