Zero-Shot Learning for Visual Recognition and Detection Tasks

Loading...
Thumbnail Image

Date

Journal Title

Journal ISSN

Volume Title

Publisher

DOI

Abstract

This dissertation explores the potential of zero-shot learning (ZSL) in addressing the shortcomings of supervised learning methods while handling unseen visual concepts. The motivation is promoting that in addition to mitigating the usual problem of annotated data scarcity of supervised learning, ZSL provides the ability to seamlessly adapt to increasing task complexity and comprehend novel scene environments. To this end, this thesis contributes towards five different computer vision applications involving two fundamental vision elements - recognition and detection. The primary objective is to elucidate the necessity of ZSL towards enhancing the usability of such applications in real-life scenarios by suggesting improvements to the existing state-of-the-art methods and addressing both task-specific and zero-shot challenges. The first contributory chapter proposes a novel algorithm called DiRaC-I that attends to visual diversity and semantic rarity in an object domain to intelligently determine the classes from a dataset that should be treated as “seen classes” for training zero-shot image classification (ZSC) models. Experiments reveal that a wide array of ZSC models benefit from using our algorithm. The second contribution aims to improve zero-shot object detection (ZSD) by mitigating the semantic confusion problem using a generative framework with novel triplet loss and cyclic consistency constraints. State-of-the-art results are achieved for two benchmark datasets, implying a reduction in false positive rates and higher quality of generated unseen class visual features. The third contribution deals with the increased difficulty of working with video data for zero-shot action recognition (ZSAR), where our proposed framework LoCATe-GAT enhances multi-scale local context with graph attention. We demonstrate that objects and action environments play the dual role of distinguishability and function similarity. The fourth contributory chapter discusses the highly challenging zero-shot human-object interaction detection (ZSHOID) task, where our proposed framework Funnel-HOI infuses a novel asymmetric co-attention mechanism with an HOI-specific loss function, outperforming previous methods. The final contributory chapter introduces the new task of zero-shot underwater gesture recognition (ZSUGR), where a hybrid transformer-GAN network is proposed, and several insights are provided for the future.

Description

Sur, Arijit

Citation

Endorsement

Review

Supplemented By

Referenced By

Creative Commons license

Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc-sa/4.0/