In the Multi-task Learning (MTL) framework, every task demands distinct feature representations, ranging from low-level to high-level attributes. It is vital to address the specific (feature/parameter) needs of each task, especially in computationally constrained environments. This work, therefore, introduces Layer-Optimized Multi-Task (LOMT) models that utilize structured sparsity to refine feature selection for individual tasks and enhance the performance of all tasks in a multi-task scenario. Structured or group sparsity systematically eliminates parameters from trivial channels and, sometimes, eventually, entire layers within a convolution neural network during training. Consequently, the remaining layers provide the most optimal features for a given task. In this two-step approach, we subsequently leverage this sparsity-induced optimal layer information to build the LOMT models by connecting task-specific decoders to these strategically identified layers, deviating from conventional approaches that uniformly connect decoders at the end of the network. This tailored architecture optimizes the network, focusing on essential features while reducing redundancy. We validate the efficacy of the proposed approach on two datasets, i.e., NYU-v2 and CelebAMask-HD datasets, for multiple heterogeneous tasks. A detailed performance analysis of the LOMT models, in contrast to the conventional MTL models, reveals that the LOMT models outperform for most task combinations. The excellent qualitative and quantitative outcomes highlight the effectiveness of employing structured sparsity for optimal layer (or feature) selection.

Multi-Task Learning (MTL) enables simultaneous learning of multiple tasks in a shared framework following the principle of ‘sharing to learn,’ to improve the performance of all the tasks. Despite its advantages, MTL also comes with several challenges. One critical issue is negative transfer, where training on one task may degrade the performance of other tasks due to conflicts in feature representations or the sharing of incompatible knowledge between tasks. MTL requires careful optimization of knowledge sharing between tasks, as over-sharing may lead to task interference, and under-sharing may prevent important knowledge transfer. Additionally, MTL systems also struggle with scalability when adapting to new tasks without retraining.

The main contributions of this thesis focus on how meta-learning can enhance MTL by addressing the above-mentioned key challenges. Meta-learning leverages the knowledge from learning multiple tasks to dynamically adapt the learning process for newtasks. Thereby focusing on ‘learning (what) to share.’ This work introduces solutions to create a unified framework by combining the adaptability of meta-learning with the task-sharing capabilities of MTL to promote effective MTL. The contributions of this thesis are organized in three dimensions. The first dimension focuses on combining MTLand meta-learning, resulting in the Multi-Task Meta-Learning (MTML) framework. The second dimension introduces structured sparsity to MTL, leading to the development of Layer-Optimized Multi-Task (LOMT) models, Structured Parameter Sparsity for Efficient Multi-Task Learning (SPARSE-MTL), and, finally, meta-sparsity. The third dimension investigates soft parameter sharing for multi-modal, multi-task feature alignment, enabling effective collaboration between different modalities.

The first major contribution (C1) is MTML, a framework that employs meta-learning to enable adaptive and efficient knowledge sharing across tasks in MTL. It uses a bi-level meta-optimization strategy to dynamically balance task-specific and shared knowledge; thereby allowing the network to learn faster and generalize better to unseen tasks while maintaining good performance across multiple tasks. This thesis further introduces the structured sparsity technique, particularly channel-wise group sparsity in multi-task settings, resulting in LOMT models (C2) and SPARSE-MTL (C3). Structured sparsity reduces redundant parameters in shared architectures of MTL, with the aim of preventing over-sharing by optimizing feature sharing across all the tasks. However, managing the sparsity level across tasks is a challenge, as the optimal degree of sparsification varies for different tasks and task combinations. To address this, meta-sparsity (C4), an extension of SPARSE-MTL and MTML, incorporates meta-learning to dynamically learn the optimal sparsity patterns across tasks. This ensures the efficient sharing of features while minimizing task interference.

In addition to the hard parameter sharing approaches, this thesis also explores soft parameter sharing for multi-modal data through a multi-task multi-modal feature alignment approach (C5). This method focuses on object detection tasks across RGB and infrared (IR) modalities. This approach aims to achieve effective knowledge sharing by employing channel-wise structured regularization to higher network layers to align semantic features between modalities and retain modality-specific features in lower layers. Therefore, it leverages the complementary strengths of RGB and IR data to enhance object detection performance across diverse conditions.

In summary, this thesis demonstrates how the integration of meta-learning and structured sparsity addresses fundamental challenges in MTL, resulting in more adaptable, efficient, and scalable systems. On a broader scale, this thesis also paves the way for parsimonious multi-task models, contributing to sustainable machine learning.

Electroencephalography (EEG) is a fundamental tool in the non-invasive evaluation of brain activity, providing insights into the intricate dynamics at play within neurode-generative disorders. Conventional methodologies often lack in effectively capturing the temporal and intricate intra- and inter-channel dynamics, leading to diminished predictive accuracy. To address this problem, we present an innovative framework that effectively captures temporal along with intra- and inter-channel dynamics for EEG analysis aimed at predicting neu-rodegenerative disorders, explicitly targeting Alzheimer's and dementia. The proposed method involves constructing aggregated recurrence matrices from EEG channels followed by kernel formation and convolution operation, effectively encapsulating intra- and inter-channel spatiotemporal patterns, thereby achieving a more comprehensive representation of neural dynamics. The proposed approach was validated using public datasets, revealing competitive performance. Implementation details with codes will be accessible on GitHub.

Model sparsification in deep learning promotes simpler, more interpretable models with fewer parameters. This not only reduces the model’s memory footprint and computational needs but also shortens inference time. This work focuses on creating sparse models optimized for multiple tasks with fewer parameters. These parsimonious models also possess the potential to match or outperform dense models in terms of performance. In this work, we introduce channel-wise l1/l2 group sparsity in the shared convolutional layers parameters (or weights) of the multi-task learning model. This approach facilitates the removal of extraneous groups i.e., channels (due to l1 regularization) and also imposes a penalty on the weights, further enhancing the learning efficiency for all tasks (due to l2 regularization). We analyzed the results of group sparsity in both single-task and multi-task settings on two widely-used multi-task learning datasets: NYU-v2 and CelebAMask-HQ. On both datasets, which consist of three different computer vision tasks each, multi-task models with approximately 70% sparsity outperform their dense equivalents. We also investigate how changing the degree of sparsification influences the model’s performance, the overall sparsity percentage, the patterns of sparsity, and the inference time.

Integrating knowledge across different domains is an essential feature of human learning. Learning paradigms such as transfer learning, meta-learning, and multi-task learning reflect the human learning process by exploiting the prior knowledge for new tasks, encouraging faster learning and good generalization for new tasks. This article gives a detailed view of these learning paradigms and their comparative analysis. The weakness of one learning algorithm turns out to be a strength of another, and thus, merging them is a prevalent trait in the literature. Numerous research papers focus on each of these learning paradigms separately and provide a comprehensive overview of them. However, this article reviews research studies that combine (two of) these learning algorithms. This survey describes how these techniques are combined to solve problems in many different fields of research, including computer vision, natural language processing, hyper-spectral imaging, and many more, in a supervised setting only. Based on the knowledge accumulated from the literature, we hypothesize a generic task-agnostic and model-agnostic learning network – an ensemble of meta-learning, transfer learning, and multi-task learning, termed Multi-modal Multi-task Meta Transfer Learning. We also present some open research questions, limitations, and future research directions for this proposed network. The aim of this article is to spark interest among scholars in effectively merging existing learning algorithms with the intention of advancing research in this field. Instead of presenting experimental results, we invite readers to explore and contemplate techniques for merging algorithms while navigating through their limitations.

Handwritten signatures in biometric authentication leverage unique individual characteristics for identification, offering high specificity through dynamic and static properties. However, this modality faces significant challenges from sophisticated forgery attempts, underscoring the need for enhanced security measures in common applications. To address forgery in signature-based biometric systems, integrating a forgery-resistant modality, namely, noninvasive electroencephalography (EEG), which captures unique brain activity patterns, can significantly enhance system robustness by leveraging multimodality’s strengths. By combining EEG, a physiological modality, with handwritten signatures, a behavioral modality, our approach capitalizes on the strengths of both, significantly fortifying the robustness of biometric systems through this multimodal integration. In addition, EEG’s resistance to replication offers a high-security level, making it a robust addition to user identification and verification. This study presents a new multimodal SignEEG v1.0 dataset based on EEG and hand-drawn signatures from 70 subjects. EEG signals and hand-drawn signatures have been collected with Emotiv Insight and Wacom One sensors, respectively. The multimodal data consists of three paradigms based on mental, & motor imagery, and physical execution: i) thinking of the signature’s image, (ii) drawing the signature mentally, and (iii) drawing a signature physically. Extensive experiments have been conducted to establish a baseline with machine learning classifiers. The results demonstrate that multimodality in biometric systems significantly enhances robustness, achieving high reliability even with limited sample sizes. We release the raw, pre-processed data and easy-to-follow implementation details.

This work presents a novel self-supervised pre-training method to learn efficient representations without labels on histopathology medical images utilizing magnification factors. Other state-of-the-art works mainly focus on fully supervised learning approaches that rely heavily on human annotations. However, the scarcity of labeled and unlabeled data is a long-standing challenge in histopathology. Currently, representation learning without labels remains unexplored in the histopathology domain. The proposed method, Magnification Prior Contrastive Similarity (MPCS), enables self-supervised learning of representations without labels on small-scale breast cancer dataset BreakHis by exploiting magnification factor, inductive transfer, and reducing human prior. The proposed method matches fully supervised learning state-of-the-art performance in malignancy classification when only 20% of labels are used in fine-tuning and outperform previous works in fully supervised learning settings for three public breast cancer datasets, including BreakHis. Further, It provides initial support for a hypothesis that reducing human-prior leads to efficient representation learning in self-supervision, which will need further investigation. The implementation of this work is available online on GitHub

This work proposes Multi-task Meta Learning (MTML), integrating two learning paradigms Multi-Task Learning (MTL) and meta learning, to bring together the best of both worlds. In particular, it focuses simultaneous learning of multiple tasks, an element of MTL and promptly adapting to new tasks, a quality of meta learning. It is important to highlight that we focus on heterogeneous tasks, which are of distinct kind, in contrast to typically considered homogeneous tasks (e.g., if all tasks are classification or if all tasks are regression tasks). The fundamental idea is to train a multi-task model, such that when an unseen task is introduced, it can learn in fewer steps whilst offering a performance at least as good as conventional single task learning on the new task or inclusion within the MTL. By conducting various experiments, we demonstrate this paradigm on two datasets and four tasks: NYU-v2 and the taskonomy dataset for which we perform semantic segmentation, depth estimation, surface normal estimation, and edge detection. MTML achieves state-of-the-art results for three out of four tasks for the NYU-v2 dataset and two out of four for the taskonomy dataset. In the taskonomy dataset, it was discovered that many pseudo-labeled segmentation masks lacked classes that were expected to be present in the ground truth; however, our MTML approach was found to be effective in detecting these missing classes, delivering good qualitative results. While, quantitatively its performance was affected due to the presence of incorrect ground truth labels. The the source code for reproducibility can be found at https://github.com/ricupa/MTML-learn-how-to-adapt-to-unseen-tasks.