This repo contains the official PyTorch code and pre-trained models for ACmix.

• On the Integration of Self-Attention and Convolution

• 2022.4.13 Update ResNet training code.

  Notice: Self-attention in ResNet is adopted following Stand-Alone Self-Attention in Vision Models, NeurIPS 2019. The sliding window pattern is extremely inefficient unless with carefully designed CUDA implementations. Therefore, it is highly recommended to use ACmix on SAN (with more efficient self-attention pattern) or Transformer-based models instead of vanilla ResNet.

We explore a closer relationship between convolution and self-attention in the sense of sharing the same computation overhead (1×1 convolutions), and combining with the remaining lightweight aggregation operations.

• Top-1 accuracy on ImageNet v.s. Multiply-Adds

Please go to the folder ResNet, Swin-Transformer for specific docs.

If you have any question, please feel free to contact the authors. Xuran Pan: pxr18@mails.tsinghua.edu.cn.

Our code is based on SAN, PVT, and Swin Transformer.

If you find our work is useful in your research, please consider citing:

@misc{pan2021integration,
      title={On the Integration of Self-Attention and Convolution}, 
      author={Xuran Pan and Chunjiang Ge and Rui Lu and Shiji Song and Guanfu Chen and Zeyi Huang and Gao Huang},
      year={2021},
      eprint={2111.14556},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}