Existing self-supervised adversarial training (self-AT) methods rely on hand-crafted adversarial attack strategies for PGD attacks, which fail to adapt to the evolving learning dynamics of the model and do not account for instance-specific characteristics of images. This results in sub-optimal adversarial robustness and limits the alignment between clean and adversarial data distributions. To address this, we propose ASTrA (Adversarial Self-supervised Training with Adaptive-Attacks), a novel framework introducing a learnable, self-supervised attack strategy network that autonomously discovers optimal attack parameters through exploration-exploitation in a single training episode. ASTrA leverages a reward mechanism based on contrastive loss, optimized with REINFORCE, enabling adaptive attack strategies without labeled data or additional hyperparameters. We further introduce a mixed contrastive objective to align the distribution of clean and adversarial examples in representation space. ASTrA achieves state-of-the-art results on CIFAR10, CIFAR100, and STL10 while integrating seamlessly as a plug-and-play module for other self-AT methods. ASTrA shows scalability to larger datasets, demonstrates strong semi-supervised performance, and is resilient to robust overfitting, backed by explainability analysis on optimal attack strategies. Project page for source code and other details at https://prakashchhipa.github.io/projects/ASTrA.

Perspective distortion (PD) leads to substantial alterations in the shape, size, orientation, angles, and spatial relationships of visual elements in images. Accurately determining camera intrinsic and extrinsic parameters is challenging, making it hard to synthesize perspective distortion effectively. The current distortion correction methods involve removing distortion and learning vision tasks, thus making it a multi-step process, often compromising performance. Recent work leverages the Möbius transform for mitigating perspective distortions (MPD) to synthesize perspective distortions without estimating camera parameters. Möbius transform requires tuning multiple interdependent and interrelated parameters and involving complex arithmetic operations, leading to substantial computational complexity. To address these challenges, we propose Log Conformal Maps (LCM), a method leveraging the logarithmic function to approximate perspective distortions with fewer parameters and reduced computational complexity. We provide a detailed foundation complemented with experiments to demonstrate that LCM with fewer parameters approximates the MPD. We show that LCM integrates well with supervised and self-supervised representation learning, outperform standard models, and matches the state-of-the-art performance in mitigating perspective distortion over multiple benchmarks, namely Imagenet-PD, Imagenet-E, and Imagenet-X. Further LCM demonstrate seamless integration with person re-identification and improved the performance. Source code is made publicly available at https://github.com/meenakshi23/Log-Conformal-Maps. 

Perspective distortion (PD) causes unprecedented changes in shape, size, orientation, angles, and other spatial relationships of visual concepts in images. Precisely estimating camera intrinsic and extrinsic parameters is a challenging task that prevents synthesizing perspective distortion. Non-availability of dedicated training data poses a critical barrier to developing robust computer vision methods. Additionally, distortion correction methods make other computer vision tasks a multi-step approach and lack performance. In this work, we propose mitigating perspective distortion (MPD) by employing a fine-grained parameter control on a specific family of Möbius transform to model real-world distortion without estimating camera intrinsic and extrinsic parameters and without the need for actual distorted data. Also, we present a dedicated perspectively distorted benchmark dataset, ImageNet-PD, to benchmark the robustness of deep learning models against this new dataset. The proposed method outperforms existing benchmarks, ImageNet-E and ImageNet-X. Additionally, it significantly improves performance on ImageNet-PD while consistently performing on standard data distribution. Notably, our method shows improved performance on three PD-affected real-world applications—crowd counting, fisheye image recognition, and person re-identification—and one PD-affected challenging CV task: object detection. The source code, dataset, and models are available on the project webpage at https://prakashchhipa.github.io/projects/mpd.

The challenge of Out-Of-Distribution (OOD) robustness remains a critical hurdle towards deploying deep vision models. Vision-Language Models (VLMs) have recently achieved groundbreaking results. VLM-based open-vocabulary object detection extends the capabilities of traditional object detection frameworks, enabling the recognition and classification of objects beyond predefined categories. Investigating OOD robustness in recent open-vocabulary object detection is essential to increase the trustworthiness of these models. This study presents a comprehensive robustness evaluation of the zero-shot capabilities of three recent open-vocabulary (OV) foundation object detection models: OWL-ViT, YOLO World, and Grounding DINO. Experiments carried out on the robustness benchmarks COCO-O, COCO-DC, and COCO-C encompassing distribution shifts due to information loss, corruption, adversarial attacks, and geometrical deformation, highlighting the challenges of the model’s robustness to foster the research in this field.

Self-supervised representation learning (SSL) has emerged as a fundamental paradigm in representation learning, enabling models to learn meaningful representations without requiring labeled data. Despite its success, SSL remains constrained by two core challenges: (i) lack of robustness against real-world distribution shifts and adversarial perturbations, and (ii) lack of domain-awareness, limiting its usability beyond natural scenes. These limitations arise from the generic invariance assumptions in SSL, which rely on predefined augmentations to learn representations but suffer to generalize when exposed to unseen environmental distortions, adversarial attacks, and domain-specific nuances. Existing SSL approaches—whether contrastive learning, knowledge distillation, or information maximization—do not explicitly account for these factors, making them vulnerable in real-world applications and suboptimal in specialized domains.

This thesis aims to enhance both robustness and domain-awareness in a modular, plug-and-play manner, ensuring that the advancements are applicable across different joint embedding architecture and method (JEAM)-based SSL approaches and adaptable to future developments in SSL. To achieve this, this thesis follows a guiding principle-leveraging invariant representations to improve robustness and domain-awareness in a modular and plug-and-play manner without altering fundamental SSL objectives. This principle guides that improvements can be seamlessly integrated into existing and future SSL approaches.

To systematically address the above-stated core challenges, this thesis begins with a foundational study of SSL approaches, identifying the common schema that underlies different SSL approaches. This unification provides a conceptual view of SSL methods, allowing us to isolate the domain-sensitive and domain-agnostic components across approaches. This conceptual outcome set the stage to establish precisely where improvements are needed to enhance robustness and domain-awareness across methods as current SSL methods fail under real-world challenges.

Next, the thesis conducts a large-scale empirical evaluation of existing SSL methods against relevant robustness benchmarks, uncovering their failures under distribution shifts caused by real-world environmental challenges. This evaluation reveals a significant decline in the robustness performance of existing SSL methods across different SSL approaches. It establishes the fundamental research gap and motivates the advancements introduced in this thesis.

The first advancement focuses on robustness against distribution shifts, particularly geometric distortions such as perspective distortion (PD), which are prevalent in real-world environment but not addressed by existing SSL methods. Since PD introduces nonlinear spatial transformations, standard affine augmentations fail to model these effects, leading to degraded representation stability. To address this, this thesis introduces Möbius-based mitigating perspective distortion (MPD) and log conformal maps (LCM), mathematically grounded transformations that enable robustness without requiring perspective-distorted training data and estimation of camera parameters. These methods are additionally adapted to multiple real-world computer vision applications—including crowd counting, object detection, person re-identification, and fisheye view recognition—showcasing their effectiveness. Further, addressing the non-availability of dedicated perspectively distorted benchmark, ImageNet-PD robustness benchmark is developed to fill the gap.

Beyond environmental challenges, another critical real-world challenge is adversarial attacks. SSL methods are highly susceptible to adversarial attacks, as the learned representations lack perturbation-invariant constraints. Existing adversarial training approaches in SSL rely on brute-force attack strategies, which fail to adapt dynamically. To address this, this thesis introduces adversarial self-supervised training with adaptive-attacks (ASTrA), where attack strategies evolve dynamically based on the model’s learning dynamics and establish a correspondence between attack parameters and training examples, optimizing adversarial perturbations in a learnable manner. Unlike conventional adversarial training, ASTrA ensures robustness while maintaining SSL’s efficiency and scalability.

While robustness, in this thesis, focuses on real-world challenges in natural scenes, domain-awareness focuses on specialized visual domains beyond natural scenes. Standard SSL augmentations are designed for variations in natural scenes, making them ill-suited for specialized fields such as medical imaging and industrial mining material inspection. This thesis introduces domain-awareness in SSL that incorporates domain-specific information into SSL’s view generation process. Particularly, (i) magnification prior contrastive similarity (MPCS) makes learned representations invariant to magnifications for histopathology images by inducing varying magnifications in the view generation process, improving breast cancer recognition. (ii) depth contrast explicitly enforces modality alignment between material images and attained height of materials on conveyor belt, ensuring that the learned representations become aware of physical properties, thereby improving material classification.

Beyond robustness and domain-awareness, SSL’s ability to generalize with limited data is advantageous for its practicality. While the loss objective in SSL is generally domain-agnostic, its effectiveness relies on large-scale data. In this direction, this thesis explores functional knowledge transfer (FKT), where self-supervised and supervised learning objectives are jointly optimized, enabling SSL representations to adapt dynamically to supervised tasks. This approach enhances generalization in low-data regimes.

In conclusion, this thesis provides a foundation for robust and domain-aware self-supervised representation learning in a modular manner, highlighting its applicability to existing and future JEAM-based SSL approaches, which can inherit these advancements and adapt to emerging challenges.

Electroencephalography (EEG) is a fundamental tool in the non-invasive evaluation of brain activity, providing insights into the intricate dynamics at play within neurode-generative disorders. Conventional methodologies often lack in effectively capturing the temporal and intricate intra- and inter-channel dynamics, leading to diminished predictive accuracy. To address this problem, we present an innovative framework that effectively captures temporal along with intra- and inter-channel dynamics for EEG analysis aimed at predicting neu-rodegenerative disorders, explicitly targeting Alzheimer's and dementia. The proposed method involves constructing aggregated recurrence matrices from EEG channels followed by kernel formation and convolution operation, effectively encapsulating intra- and inter-channel spatiotemporal patterns, thereby achieving a more comprehensive representation of neural dynamics. The proposed approach was validated using public datasets, revealing competitive performance. Implementation details with codes will be accessible on GitHub.

Handwritten signatures in biometric authentication leverage unique individual characteristics for identification, offering high specificity through dynamic and static properties. However, this modality faces significant challenges from sophisticated forgery attempts, underscoring the need for enhanced security measures in common applications. To address forgery in signature-based biometric systems, integrating a forgery-resistant modality, namely, noninvasive electroencephalography (EEG), which captures unique brain activity patterns, can significantly enhance system robustness by leveraging multimodality’s strengths. By combining EEG, a physiological modality, with handwritten signatures, a behavioral modality, our approach capitalizes on the strengths of both, significantly fortifying the robustness of biometric systems through this multimodal integration. In addition, EEG’s resistance to replication offers a high-security level, making it a robust addition to user identification and verification. This study presents a new multimodal SignEEG v1.0 dataset based on EEG and hand-drawn signatures from 70 subjects. EEG signals and hand-drawn signatures have been collected with Emotiv Insight and Wacom One sensors, respectively. The multimodal data consists of three paradigms based on mental, & motor imagery, and physical execution: i) thinking of the signature’s image, (ii) drawing the signature mentally, and (iii) drawing a signature physically. Extensive experiments have been conducted to establish a baseline with machine learning classifiers. The results demonstrate that multimodality in biometric systems significantly enhances robustness, achieving high reliability even with limited sample sizes. We release the raw, pre-processed data and easy-to-follow implementation details.

This work presents a novel domain adaption paradigm for studying contrastive self-supervised representation learning and knowledge transfer using remote sensing satellite data. Major state-of-the-art remote sensing visual domain ef-forts primarily focus on fully supervised learning approaches that rely entirely on human annotations. On the other hand, human annotations in remote sensing satellite imagery are always subject to limited quantity due to high costs and domain expertise, making transfer learning a viable alternative. The proposed approach investigates the knowledge transfer of self-supervised representations across the distinct source and target data distributions in depth in the remote sensing data domain. In this arrangement, self-supervised contrastive learning- based pretraining is performed on the source dataset, and downstream tasks are performed on the target datasets in a round-robin fashion. Experiments are conducted on three publicly avail-able datasets, UC Merced Landuse (UCMD), SIRI-WHU, and MLRSNet, for different downstream classification tasks versus label efficiency. In self-supervised knowledge transfer, the pro-posed approach achieves state-of-the-art performance with label efficiency labels and outperforms a fully supervised setting. A more in-depth qualitative examination reveals consistent evidence for explainable representation learning. The source code and trained models are published on GitHub1.