Meta-Learning for Medium-shot Sparse Learning via Deep Kernels

Document Type : Original Article

Authors

• Zohreh Adabi Firuzjaee
• Sayed Kamaledin Ghiasi-Shirazi

Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

10.22067/cke.2022.77529.1060

Abstract

Few-shot learning assumes that we have a very small dataset for each task and trains a model on the set of tasks. For real-world problems, however, the amount of available data is substantially much more; we call this a medium-shot setting, where the dataset often has several hundreds of data. Despite their high accuracy, deep neural networks have a drawback as they are black-box. Learning interpretable models has become more important over time. This study aims to obtain sample-based interpretability using the attention mechanism. The main idea is reducing the task training data into a small number of support vectors using sparse kernel methods, and the model then predicts the test data of the task based on these support vectors. We propose a sparse medium-shot learning algorithm based on a metric-based Bayesian meta-learning algorithm whose output is probabilistic. Sparsity, along with uncertainty, effectively plays a key role in interpreting the model's behavior. In our experiments, we show that the proposed method provides significant interpretability by selecting a small number of support vectors and, at the same time, has a competitive accuracy compared to other less interpretable methods.

Keywords

• Bayesian Meta-Learning
• Medium-shot Learning
• Sample-based Interpretability
• Sparse Kernel
• Attention

Main Subjects

• Machine learning

---