Kushal Arora

Hello!! I am Kushal Arora, a senior research scientist at Toyota Research Institute. I am working on pretraining, scaling, and improving reasoning and instruction-following abilities of vision-language-action (VLA) models. Check out some of our recent research below.

Before this, I was a Ph.D. student at McGill University and Mila where I was jointly supervised by Prof. Jackie Cheung and Prof. Doina Precup. I got an amazing opportunity to intern with amazing mentors at Borealis AI, Microsoft Research, and Meta AI during my Ph.D.

Selected Publications

A critical evaluation of ai feedback for aligning large language models

Sharma, Archit, Keh, Sedrick, Mitchell, Eric, Finn, Chelsea, Arora, Kushal, and Kollar, Thomas

arXiv preprint arXiv:2402.12366 Feb 2024
DataComp-LM: In search of the next generation of training sets for language models

Li, Jeffrey, Fang, Alex, Smyrnis, Georgios, Ivgi, Maor, Jordan, Matt, Gadre, Samir, Bansal, Hritik, Guha, Etash, Keh, Sedrick, Arora, Kushal, and others,

arXiv preprint arXiv:2406.11794 Jun 2024
The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation

Arora, Kushal, O’Donnell, Timothy J., Precup, Doina, Weston, Jason, and Cheung, Jackie C.K.

In Under Submission, ICML 2023 Jan 2023

Abs

State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling. This degeneration usually shows up in the form of incoherence, lack of vocabulary diversity, and self-repetition or copying from the context. In this paper, we postulate that “human-like” generations usually lie in a narrow and nearly flat entropy band, and violation of these entropy bounds correlates with degenerate behavior. Our experiments show that this stable narrow entropy zone exists across models, tasks, and domains and confirm the hypothesis that violations of this zone correlate with degeneration. We then use this insight to propose an entropy-aware decoding algorithm that respects these entropy bounds resulting in less degenerate, more contextual, and "human-like" language generation in open-ended text generation settings.
DIRECTOR: Generator-Classifiers For Supervised Language Modeling

Arora, Kushal, Shuster, Kurt, Sukhbaatar, Sainbayar, and Weston, Jason

In Asian Association for Computational Linguistics: AACL 2022 Sep 2022

Abs

Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions. The standard language modeling setup fails to address these issues. In this paper, we introduce a new architecture, {}sc Director}, that consists of a unified generator-classifier with both a language modeling and a classification head for each output token. Training is conducted jointly using both standard language modeling data, and data labeled with desirable and undesirable sequences. Experiments in several settings show that the model has competitive training and decoding speed compared to standard language models while yielding superior results, alleviating known issues while maintaining generation quality. It also outperforms existing model guiding approaches in terms of both accuracy and efficiency.
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation

Arora, Kushal, El Asri, Layla, Bahuleyan, Hareesh, and Cheung, Jackie

In Findings of the Association for Computational Linguistics: ACL 2022 May 2022

Abs

Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis for this brittleness of generation models is that it is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this paper, we verify this hypothesis by analyzing exposure bias from an imitation learning perspective. We show that exposure bias leads to an accumulation of errors during generation, analyze why perplexity fails to capture this accumulation of errors, and empirically show that this accumulation results in poor generation quality.