Official implementation for paper Vexel-Net: Vectors-Composed Features Oriented Neural Network.

Extracting features robust to transformation and perturbation upon data is important for deep learning related research tasks and applied areas. Among existing methods, Capsule based methods utilize vectorized features, and voting mechanism in extracting features from images, and show robustness after data go through affine transformation and adversarial attack. However, capsule based methods need long time for training due to sophisticated iterative mechanism, and their dense matrix weights or extra pose scalars lead to heavy computation burden. In this research, we group neurons into vectors without denoting activation with extra scalars, and propose a non-iterative low-complexity A&B algorithm to conduct the vote process between layers. In addition we design different weight-sharing schemes to balance the representation ability and computation complexity of our algorithm. Experiment results on image classification, affine transformation and adversarial attack show that our method demonstrates better prediction ability and more robust performance with lighter computation burden.

Our codes are implemented benefiting from SR-CapsNet, we recommend that interested readers will gain a lot after reading their paper Self-Routing Capsule Networks and looking into their repository.

pytorch >= 2.0.1+cu118
pytorch lightning >= 2.1.0
pytorch vision >= 0.15.2+cu118
einops >= 0.6.1
numpy >= 1.23.5

Other python libraries need for this repository will be installed when installing pytorch lightning. If met problem with libraries for this repository, you are welcome to submit issues.

Args are defined in utils/configUtils.py , if users want to specify different values for args, we suggest modifying args in utils/configUtils.py, just cleanly run main.py. And other arguments for models and optimization are defined in main.py as python dictoinaries.

The configurations and parameters of networks for image classifications are already recorded in our paper, please refer to it if interested.

Just modify args for viewpoints by assign values for --exp and famaliar, look at data/data.py, data/utils.py for how these args work.

Specify values for args --typeAttack, --epsAttack and --targeted in utils/configUtils/py in order to generate adversarial samples and run adversarial_attack.py for adversarial attack test. Interested readers can refer to adversarial_attack.py for loading checkpoints for LightningModule.

All ablation experiments are conducted by changing args in main.py and changing network architecture in models/networks/VexNet.py. Please pay attention to channels, kernel_size, and padding of Vexel conv layers when modifying network architecture. We listed used configurations as follows.

As for ablation on initial Vexel channels and dimension, just refer to main.py and modify corresponding values in argsNet.

We adopt Pytorch and Pytorch Lightning for implementation our methods as well as other networks, please refer to doc of load_from_checkpoint for loading trained LightningMotule checkpoints.

Logs for approaches in our experiments will be uploaded to google drive folders afterwards.