Incremental-Decremental Methods for Multi-view Discriminant Analysis
No Thumbnail Available
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In recent years, the use of multi-view data has attracted much attention from the machine learning community. Multiple views of an object are obtained using different sensors and contain different features that describe the object. Such complementary nature of the information contributed by multiple views leads to improved performance of the trained model. Many multi-view learning algorithms have been proposed to make use of multi-view data. However, these algorithms predominantly belong to the batch learning paradigm. Batch learning methods need all the training data at the start of the training process. These methods cannot incorporate new data once the training has been completed. If such a situation occurs, these methods must discard the trained model and retrain on the updated dataset, thereby proving expensive in terms of training time and memory.
Incremental methods solve this problem by updating the trained model after each increment without needing the historical data. Many incremental counterparts of single-view learning algorithms have been proposed in recent years. Some methods also support decremental unlearning of data when a subset of existing data is deleted. However, there are only a handful of incremental algorithms for multi-view data. The increment can be of two types in a multi-view context- data sample increment and view increment.
We present four multi-view methods in this thesis. Three of which are incremental methods equipped to update a trained model without needing the historical data, and one is a batch method. Two of the incremental methods are data sample incremental methods. One supports incremental learning for 1D multi-view data, and the other supports incremental learning and decremental unlearning for 2D multi-view data. The third method is a view incremental multi-view method that supports the addition and deletion of views. While formulating the 2D incremental method, we also formulated a 2D multi-view batch method due to the absence of a multi-view batch method for 2D data. All of these methods are based on Multi-view Discriminant Analysis (MvDA).
The first method is Multi-view Incremental Discriminant Analysis (MvIDA), which updates a trained model to incorporate new data samples. MvIDA requires only the old model and newly added data to update the model. Depending on the nature of increments, MvIDA is presented as two cases, sequential MvIDA and chunk MvIDA. Both of the cases are equipped to handle data samples from previously unseen classes. The experiments conducted on three widely used 1D multi-view datasets show that through order independence and faster construction of the optimal discriminant subspace, MvIDA addresses the issues faced by MvDA in an incremental setting.% We have compared the proposed method against its batch counterpart MvDA and single-view counterpart Incremental Linear Discriminant Analysis (ILDA) on various parameters.
The second method is 2D Multi-view Discriminant Analysis (2DMvDA), a 2D multi-view classification method based on discriminant analysis. It uses the 2D image matrices directly instead of extracting 1D features from them ensuring the preservation of spatial information and reduction in the sizes of scatter matrices. This leads to better classification accuracy and a considerable reduction in the computational cost. The experiments carried out on four image-based multi-view datasets show that, using less time and memory, 2DMvDA achieves a classification accuracy at par or better than its 1D and single-view counterparts.
We present an incremental version of 2DMvDA as 2D Multi-view Incremental Decremental Discriminant Analysis (2DMvIDDA) that provides a way to incorporate new data samples or discard the old ones without needing the historical data. The updates are done using just the old model and the data samples to be added or deleted. We also present 2DMvIDDA as an umbrella method that can be transformed into other methods that are based on discriminant analysis, such as MvDA, MvIDA, 2DMvDA, 2DLDA (2D Linear Discriminant Analysis), and ILDA (Incremental Linear Discriminant Analysis). Through the experiments on four image-based multi-view datasets, we show that the proposed method is order-independent and converges to the same discriminant subspace as 2DMvDA and builds a better model than other relevant methods with less time and memory.
Lastly, we present View Incremental Decremental Multi-view Discriminant Analysis (VIDMvDA) that updates a learned model without retraining when new views are added or existing views are deleted. VIDMvDA is presented in two forms: incremental learning and decremental unlearning. It provides closed-form solutions to update the within-class and the between-class scatter, and it can also be used for 2D data by changing only one parameter. We prove that using significantly less training time and memory, VIDMvDA constructs a similar discriminant subspace and classification accuracy as its batch counterpart.
Description
Supervisor: V Vijaya Saradhi
Keywords
Multi-view Learning, Incremental Learning, Discriminant Analysis, Classification