Working papers
(To be submitted in CVPR 2024)
Link to Paper (coming soon)
Abstract : Tackling distribution shifts during inference has been a challenging task for Machine Learning (ML) models. The reason to difficulty in generalization over different domains can be attributed to the tendency of ML models to exploit and use false and spurious correlations in train data for optimizing their respective objectives, which are not present during inference. The spurious correlations can be as simple as cow in a green background while training, and still possibly failing a model to identify cow on a beach in inference time. Recent works show that using Mixture-Of-Experts with foundation models helps in improving performance of ML models against distribution shifts. However, these models use learnt gating network to select experts in MoE. In this work, we show that learnt gating networks inhibit the performance of these models due to inaccurate routing during inference. To this end, we present a simple change to existing MoE architectures to choose the experts without gating networks while leaving the structure of foundation models intact. Our devised method uses confidence of experts to select expert from MoE, and introduces a regularizer to Empirical Risk Minimization (ERM) objective for tuning the confidence of experts. We empirically compare the accuracy of our devised method with other state-of-art MoE methods in Domain Generalization (DG) and with vanilla foundation models in computer vision on 4 DG datasets: PACS, VLCS, Office-Home, and TerraIncognita.
(Submitted to WACV 2024)
Link to Paper (coming soon)
Abstract : This work presents an adaptive group testing framework for the range-based high dimensional near neighbor search problem. The proposed method detects high-similarity vectors from an extensive collection of high dimensional vectors, where each vector represents an image descriptor. Our method efficiently marks each item in the collection as neighbor or non-neighbor on the basis of a cosine distance threshold without exhaustive search. Like other methods in the domain of large scale retrieval, our approach exploits the assumption that most of the items in the collection are unrelated to the query. Unlike other methods, it does not assume a large difference between the cosine similarity of the query vector with the least related neighbor and that with the least unrelated non-neighbor. Following the procedure of binary splitting, a multi-stage adaptive group testing algorithm, we split the set of items to be searched into half at each step, and perform dot product tests on smaller and smaller subsets, many of which we are able to prune away. We experimentally show that our method achieves a speedup over exhaustive search by a factor of more than ten with an accuracy same as that of exhaustive search, on a variety of large datasets. For theoretical analysis, we model the distribution of cosine distances between a query vector and the vectors in the collection using a truncated exponential distribution, and demonstrate that this is a good model for cosine distances in practice. Equipped with this, we present a theoretical analysis of the expected number of distance computations per query and the probability that a pool with a certain number of members will be pruned. In this way, our method exploits very useful and practical distributional properties unlike other methods. In our method, all required data structures are created purely offline. Moreover, our method does not impose any strong assumptions on the number of true near neighbors, is adaptible to streaming settings where new vectors are dynamically added to the database, and does not require any parameter tuning.
Posters/Presentations
(Presented in CS Research Symposium, IITB)
Abstract : Rendering virtual objects in augmented reality presents a formidable challenge, primarily in ensuring an immersive viewer experience. The ambient lighting conditions within the environment where these virtual objects are placed can vary significantly, causing a noticeable discrepancy between the rendered object and its surroundings. To address this issue, various techniques have been proposed, with one common approach being the precomputation of lighting information within the virtual object. In this research, we introduce a novel methodology for the efficient and accurate prediction of the spherical harmonics representing environmental illumination. Our approach leverages a reflective spherical probe positioned within the environment to capture its lighting conditions, and predict the spherical harmonics of the ambient lighting. The spherical harmonics are then used over bake lighting in the textures of the virtual object. Our method circumvents data-driven techniques, thereby achieving superior generalization, reduced inference time and enhancement in the overall quality of augmented reality object rendering. We implement our pipeline for rendering virtual objects using Unity with the help of ARcore package.