SelfReg: Self-Supervised Contrastive Regularization for Domain Generalization

SelfReg: Self-supervised Contrastive Regularization for Domain Generalization

domain adaptation VS domain generalization

proposed method

• use positive pairs (of the same class) in domain generalization
• a contrastive learning-style approach
• a loss based on the intermediate features + a loss based on the logits
  • regardless of domain, order the input in the batch based on class
  • after running through feature extractor & classifier, calculate losses
  • $L_{feature}$ and $L_{logit}$ calculated through a regularization block
    • CDPL into shuffling
    • loss with shuffled version & with a version that applies mixup to the shuffled ones
• gradient stabilization
  • loss clipping
    • self-regularization loss was too dominant after initial training phase
  • inter-domain curriculum learning
    • incrementally add in classes
    • use SWA afterward (at the epoch level)