- Add RPE, Rotary positional embeddings
- Fix experiment code, update models to work without separate config
- Test on TPUv3-8
- Run first training runs comparing DeiT with absolute learned vs. rotary pos embeddings
- Add class-attention layers, layerscale (CaiT)
- Add CvT
- Add TNT, Twins
forked from NZ99/self-attention-experiments-vision
ShivanshuPurohit/self-attention-experiments-vision
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
A project about replicating, evaluating and scaling up self-attention based models in vision.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- Python 100.0%