Topics about AI infra and MLSys: framework, computation and communication.
Articles
- Megatron CP, HCP and DCP
- [paper] mHC: Manifold-Constrained Hyper-Connections
- Gated Delta Network
- The last time to learn Softmax
- [paper] DWDP
- Some concepts in ML/DL/LLM
- PyTorch memory allocation during tensor shape manipulation
- Tracing mechanisms in PyTorch
- Build AI frameworks from source with NVIDIA GPU support