Sparse-from-start training
Train a sparse network while updating its connections and weights.
TECHNOLOGY
Learning network connections
CHT uses network structure to guide connection updates during training.
Neural connections
Existing structureCoactivation
01
Connections strengthen
Neurons that activate together can form stronger connections.
Adapting existing models
Train and adapt existing models for sparsity, as in our public Qwen release.
BRAIN-INSPIRED LEARNING
Neural connections
and sparse learning
CHT applies principles of brain connectivity to neural network training.
Cannistraci–Hebb Training (CHT) was developed by Professor Carlo Vittorio Cannistraci and his team at Tsinghua University. It uses network topology to guide connection regrowth during training.
CHT updates connections during training and is evaluated on specific models and tasks.
Research papers
Research on network structure, dynamic sparse training and hardware efficiency.
NeurIPS 2026NeurIPS 2026ICLR 2024NeurIPS 2025NeurIPS 2025PMLR 2026
Channel-wise dynamic sparse training for convolutional neural networks
Cannistraci-Hebb Channel-wise Dynamic Sparse Training of Convolutional Neural Networks with Contextual Modulation, poster.
Semi-structured dynamic sparse training for hardware-efficient deep learning
SCHTs: A Semi-Structured Dynamic Sparse Training Framework for Hardware-Efficient Deep Learning, poster.
Network shape intelligence in ultra-sparse learning
Yingtao Zhang et al. · How network organization supports learning.
Brain network science for Transformers and LLMs
Yingtao Zhang et al. · Soft connection sampling and gradual density decay for dynamic sparse training.
Adaptive Cannistraci–Hebb network automata
Jialin Zhao et al. · Adaptive topology rules for link prediction in complex networks.
CHT meets N:M semi-structured sparsity
Jiaqing Lyu et al. · CHTs24 and sparse retraining for hardware-friendly 2:4 formats.
Research and applications
Joint evaluation of sparse models and chips
A 1B model at 1:4 sparsity delivers 2.67× compiler-estimated input throughput on the same platform.
Compiler-level validationRead
Joint evaluation of a sparse Qwen3-8B model
A specified Qwen3-8B training experiment reaches 11.00 PPL through adaptation, sparsification and training.
Model-level validationRead
Physics-informed sparse modeling for GaN devices
Combining physics-informed learning and sparse structures to explore efficient simulation for GaN devices.
Research & validation targetsRead
Contact us
Contact us about research or collaboration.
Contact SparseMind