Self-Attention experiments in Vision

To do

Add RPE, Rotary positional embeddings
Fix experiment code, update models to work without separate config
Test on TPUv3-8
Run first training runs comparing DeiT with absolute learned vs. rotary pos embeddings
Add class-attention layers, layerscale (CaiT)
Add CvT
Add TNT, Twins

Name		Name	Last commit message	Last commit date
Latest commit History 40 Commits
experiments		experiments
models		models
.deepsource.toml		.deepsource.toml
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
autoaugment.py		autoaugment.py
input_pipeline.py		input_pipeline.py
utils.py		utils.py