European Conference on Computer Vision (ECCV 2024), Arxive, HuggingFace

Saeed Ebrahimi★, Sahar Rahimi★, Ali Dabouei, Nasser Nasrabadi

★ Equal contribution

Aiming to enhance Face Recognition (FR) on Low-Quality (LQ) inputs, recent studies suggest incorporating synthetic LQ samples into training. Although promising, the quality factors that are considered in these works are general rather than FR-specific, e.g., atmospheric turbulence, resolution, etc. Motivated by the observation of the vulnerability of current FR models to even small Face Alignment Errors (FAE) in LQ images, we present a simple yet effective method that considers FAE as another quality factor that is tailored to FR. We seek to improve LQ FR by enhancing FR models' robustness to FAE. To this aim, we formalize the problem as a combination of differentiable spatial transformations and adversarial data augmentation in FR. We perturb the alignment of the training samples using a controllable spatial transformation and enrich the training with samples expressing FAE. We demonstrate the benefits of the proposed method by conducting evaluations on IJB-B, IJB-C, IJB-S (+4.3% Rank1), and TinyFace (+2.63%)

Visual comparison of aligned (a) and alignment-perturbed (b) samples from the IJB-B dataset. (c, d, e) The performance difference between aligned inputs and those with slight FAE. Models exhibit robustness to FAE in HQ samples but suffer significant performance drops in LQ faces, with over 50% reduction in TAR@FAR=1e-5. Results from two distinct ResNet-100 trained on MS1MV3 using ArcFace/AdaFace objective.

• We introduce Face Alignment Error (FAE) as an image degradation factor tailored for FR which has previously been ignored in LQ FR studies.
• We propose an optimization method that is specifically tailored to increase the FR model robustness against FAE.
• We show that the proposed optimization can greatly increase the FR performance in real-world LQ evaluations such as IJB-S and TinyFace. Moreover, our framework achieves these improvements without sacrificing the performance on datasets with both HQ and LQ samples such as IJB-B and IJB-C.
• We empirically show that the proposed method is a plug-and-play module, providing an orthogonal improvement to SOTA FR methods.

★ Re-runs with official code due to missing trained checkpoints on the specified dataset in the official repository

Download and prepare datasets from InsightFace repository

The total batch size we used for training was 2048 on four Nvidia RTX 6000 ADA. To have stable training, choose the learning rate based on the total batch size on your machine:

config.lr = (0.1*config.batch_size*config.ngpus)/(1024)

Please modify the

config.ngpus = 4

according to your resources in configs.

Then, for training on one machine using four GPUs:

torchrun --nproc_per_node=4 train_v2.py configs/ms1mv2_r100

@misc{saadabadi2024arofacealignmentrobustnessimprove,
      title={ARoFace: Alignment Robustness to Improve Low-Quality Face Recognition}, 
      author={Mohammad Saeed Ebrahimi Saadabadi and Sahar Rahimi Malakshan and Ali Dabouei and Nasser M. Nasrabadi},
      year={2024},
      eprint={2407.14972},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.14972}, 
}

Here are some great resources we benefit from:

• ArcFace and AdaFace for the face recognition module.
• advertorch, RobustAdversarialNetwork, and CFSM for the adversarial regularization.

If there is a question regarding any part of the code, or it needs further clarification, please create an issue or send me an email: me00018@mix.wvu.edu.