Ritvik Agarwal

Excited to share that our extended abstract titled "Incorporating Foundation Model Priors in Modeling Novel Objects for Robot Instruction Following in Unstructured Environments" has been accepted for presentation at the Workshop on 3D Visual Representations for Robot Manipulation at ICRA 2024! Visit our website for more details: https://lnkd.in/gcmfMJgu Workshop: https://lnkd.in/gA38HrpH 📝 Authors: Moksh Malhotra, Aman Tambi, Sandeep Zachariah, P.V.M. Rao, Rohan Paul 🙏 Special thanks to our project advisors Prof. Rohan Paul and Prof. P.V.M. Rao for their invaluable guidance and support throughout this research journey! In our paper, we tackle the challenge of acquiring object models when a robotic manipulator is sent in unstructured and unknown environments. We proposes a novel approach to utilizes foundation models with RGB-Depth images to acquire high-fidelity semantic 3D model of objects, required for robotic manipulation. We demonstrate our pipeline in simulation as well as real-world scenarios. Looking forward to discussing our research and exchanging ideas with fellow researchers and professionals at the conference. Stay tuned for more updates! #ICRA2024 #Robotics #Research #AI #3DVisualization #RobotManipulation

11

2 Comments

Jianbo Shi and Jitendra Malik's paper on "Normalized Cuts and Image Segmentation" (https://lnkd.in/dcyU5Skr) published in 2000, is a seminal work that introduces an interesting method for image segmentation by using concepts from graph theory and linear algebra. It is one of the most highly cited papers in Computer Vision with over 20k+ citations. By modeling the image as a graph, where pixels represent nodes and the edges between them encode the similarity, the segmentation problem transforms into one of graph partitioning. The segments then can be thought of as subgraphs. The key idea is to partition the graph to minimize the normalized cut (a criterion that evaluates the cut cost: the total weight of the edges removed to split the graph) relative to the total edge connections within each group. This problem is then elegantly converted into a linear algebra problem: finding the partition corresponds to solving for the second smallest eigenvector of the Laplacian matrix of the graph. Here's a short report on the entire paper, prepared as part of our project component for the Linear Algebra and its Applications course last semester. Please feel free to comment or address any issues if there are any! #ImageSegmentation #MachineLearning #ComputerVision #DataScience #ScientificResearch #LinearAlgebra #GraphTheory #Report #project

125

5 Comments

🚀 Exciting breakthrough in Indian AI! Introducing Sarvam-1, India's first open-source LLM built by Sarvam specifically for Indian languages! 🇮🇳 🔍 Key highlights: 🔹2B parameter model trained on 4T tokens high-quality data with 2T tokens of Indic language data  🔹Leveraged Llama 3.1 405B model to generate high-quality synthetic training data of ~2 trillion tokens across academic, technical & general domains, with content evenly distributed across the 10 supported Indian languages, except for Hindi, which constitutes about 20% of the dataset 🔹Superior token parity score (1.4-2.1 tokens per word vs 4-8 in other models) 🔹Languages supported: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Oriya, Punjabi, Tamil, and Telugu + English 🔹Outperforms larger models like Gemma-2B and Llama-3.2-3B on key benchmarks 🔹4-6x faster inference compared to larger models 📈 Benchmark Performance: 🔹Outperforms Gemma-2B and Llama-3.2-3B 🔹Exceptional TriviaQA scores: 86.11% across Indic languages 🔹Strong MMLU (38.22%) and ARC-Challenge (46.71%) and IndicGenBench results 🔹Outstanding translation metrics: 46.81 chrF++ score on Flores 🔹Strong performance in both knowledge and reasoning tasks 🌟 Training Data : Document quality 🔹2x longer documents vs standard web data 🔹3x more high-quality samples Content distribution 🔹8x more scientific/technical content 🔹6x more programming documentation 🔹Balanced domain representation 💡 Technical Innovations: 🔹Ground-up optimization for Indian languages 🔹NeMo Curator domain and quality classifier models to improve data quality 🔹Custom tokenizer with revolutionary efficiency (1.4-2.1 tokens/word vs industry standard 4-8) 🔹Optimized architecture: 2048 hidden size, 16 attention heads, 8,192 maximum position embeddings 🔹Trained using NVIDIA's NeMo framework 🔹Trained on Yotta Data Services Private Limited's Shakti cluster using 1,024 GPUs for 5 days 💭 As someone passionate to build SLMs for India, I'm particularly excited about this development. What aspects of Sarvam-1 intrigue you the most? 🤝 Open Source Community: The model is open-sourced, inviting developers, researchers, and innovators to build upon this foundation. What applications do you envision building with Sarvam-1? Share your ideas below! 👇 Vivek Raghavan Pratyush Kumar Sunil Gupta #SLM #LLM #GenAI

41

1 Comment

EMNLP 2024 results are out, we got in three papers, congrats to students :) First paper of many, Vatsal, Pranshu, Suyash, Janit, Kunal, Srija and Adnan, Congrats 🎉 🎊 🍾 🎈 Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets (main) - Vatsal Gupta, PRANSHU PANDYA, Tushar Kataria, Vivek Gupta, Dan Roth tldr: make models robust on multiple adversarial challenge set simultaneously  arxiv: https://lnkd.in/ei-65Y6g Knowledge-Aware Reasoning over Multimodal Semi-structured Tables (finding) Suyash Vardhan Mathur, Jainit Bafna, Kunal Kartik, Harshita Khandelwal, Manish Shrivastava, Vivek Gupta, Mohit Bansal, Dan Roth tldr: introduce the novel task of reasoning on multimodal tabular data, text + images in a structure (first of a kind) arxiv: https://lnkd.in/e5ycusfn   Unraveling the Truth: Do LLMs really Understand Charts? A Deep Dive into Consistency and Robustness (finding) Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta, Dan Roth           tldr: VLM although seem very accurate on chart, are not all robust and consistent in performance arxiv: https://lnkd.in/eFFA-sVX We also got an workshop paper, at BlockBoxNLP 2024 @EMNLP, here the details Enhancing Question Answering on Charts Through Effective Pre-training Tasks Ashim Gupta, Vivek Gupta, Shuo Zhang, Yujie He, Ning Zhang , Shalin Shah, Ph.D. tldr: models is not robust & consistent on charts, can we use pre-training on simple auxiliary tasks to enhance model robustness arxiv: https://lnkd.in/egbRHm-4

382

12 Comments

Shashank Sonkar and colleagues are back with another great paper and matching GitHub repo, "Pedagogical Alignment of Large Language Models". You can read the paper on arXiv here - https://lnkd.in/enZfCJgi, and browse the code here - https://lnkd.in/gyW5hSzp. They demonstrate an interesting approach to generating synthetic data to train open weights models to behave more pedagogically, as well as developing some new evaluation metrics. I think there's lots here for those of us interested in this kind of work to learn from and build on. (If synthetic data generation is new do you, check out this post from answer.ai earlier this week - https://lnkd.in/ecsG7R4b.)

38

4 Comments

Metacognition in AI Agents AI agents [1] often make mistakes because they commit to inefficient plans or become stuck in loops, repeatedly trying the same approach [2]. Why does this happen, and what can be done? One perspective is that these agents lack metacognition- the ability to plan, monitor, and evaluate their own thinking and problem-solving processes [4] . This limitation partly stems from current autoregressive models generating new tokens based on previous context, making it challenging to abandon poor plans or approaches once they are in the 'context'. In my opinion, implementing metacognition modules for AI agents will accelerate improved behaviors, making them more useful and reliable. We will likely achieve this through two main approaches: - Model improvements (e.g., new architectures, fine-tuning models to exhibit implicit metacognition, or implementing fine-grained control over context usage etc) - Building dedicated metacognition modules that interleave model reasoning with predefined logic programs [3,4,5] Chapter 6 in the upcoming @manning book on Multi-Agent systems with #autogen will touch on this topic. Sign up here: https://mng.bz/eVP9 . Clip below is from this roundtable conversation on AI and software engineering - https://lnkd.in/gMbe7ghb. References: [1] Multi-Agent Systems with Autogen Book - https://mng.bz/eVP9  [2] Multi-Agent LLM Applications | A Review of Current Research, Tools, and Challenges - https://lnkd.in/eQajE3_Q  [3] Sam Schillace - Think with the model, Plan with the code - https://lnkd.in/gBnFtg7n [4] A Prefrontal Cortex-inspired Architecture for Planning in Large Language Models by my colleague Ida Momennejad and team. https://lnkd.in/gwdbyr_v [5] Shinn, Noah, et al "Reflexion: Language agents with verbal reinforcement learning." Advances in Neural Information Processing Systems 36 (2024). https://lnkd.in/gV6fMW7B #aiagents #multiagentbook #softwareengineering #responsibleai

111

9 Comments

Till now, we have tried everything but nothing matches the performance of transformers-based LLMs. 🤖✨ But because of the quadratic complexity of the attention computation, it takes 30 minutes for an 8B LLM to process a prompt of 1M tokens (i.e., the pre-filling stage) on a single A100 GPU. 🕒💻 The solutions to this often trade off accuracy for efficiency! ⚖️⚡ To solve for this, Microsoft has introduced MInference (Milliontokens Inference) 🚀 A sparse calculation method designed to accelerate pre-filling of long-sequence processing. 🧩🔄 Specifically, they identify three unique patterns in long-context attention: 1️⃣ Matrices-the A-shape 🔺 2️⃣ Vertical-Slash ➖ 3️⃣ Block-Sparse 🔲 which can be leveraged for efficient sparse computation on GPUs. 💡💪 The idea is to determine the optimal pattern for each attention head offline and dynamically build sparse indices based on the assigned pattern during inference. 🧠⚙️ The result? MInference effectively reduces inference latency by up to 10x for pre-filling on an A100, while maintaining accuracy! 🚀📉✅ 📄 Paper: https://lnkd.in/gjFgFGpc 🛠️ Demo: https://lnkd.in/gJ63Ki-S 💻 Code: https://lnkd.in/gFpKpc4T

19

2 Comments

Text chunks in retrieval may risk losing context. The below article introduces contextualizing each chunk using an LLM. The added cost and latency required for contextualizing is partially addressed through a lightweight LLM (haiku) and prompt caching. Along with other known techniques (BM25, reranking, etc.), a significant reduction in retrieval failure is achieved. Overall, a good article to read. https://lnkd.in/g_5fgAN8

10

❝ How do you taste your Wine? With the right Glass, served from the right Bottle. How do you utilize your LLM? With the right Task, served from the right Compute environment. ❞ If the quote above caught your interest, check out my blog post that was featured on the Toyota Connected India Medium channel last week: Leveraging LLMs in Cloud - 🔗 https://lnkd.in/gfHsdevH --- I hope this blog adds a bit of amusement for the seasoned AI/ML engineers 🧠👨‍💻 with the carefully crafted images and perhaps even the silly wine analogies 😉. Whether you're a cloud engineer ☁️ 👨‍💻, a product owner of cloud-based apps, or someone new to Generative AI in the cloud, this blog offers a detailed, beginner-friendly look into the selection and deployment of LLMs in cloud environments. --- A heartfelt thanks 🙏 to my colleagues at Toyota Connected India for taking the time to review, shape this piece and even share a creative writeup like below in the original post 🫡 👇 🤔 Curious to dive in? Click the link below to explore the blog on Medium! 🌐 🔗 https://lnkd.in/gfHsdevH

18

Brain Teaser #10: Suppose we are fine-tuning a large language model using Low-Rank Adaptation (LoRA). We possess two low-rank update matrices, δWA​ and δWB​​, both associated with the rank "r". Considering our objective to preserve the model's current knowledge while incorporating new data from diverse features, what "mathematical rationale" should guide the selection of one of the update matrices (δWA​ or δWB) for low-rank adaptation? Hint: Consider the specific characteristics or properties of the matrices that can help in developing mathematical reasoning. Here is my response: The norm ratio, calculated as the ratio of the nuclear norm to the Frobenius norm of the update matrix, is an essential metric for assessing how updates impact a model. It provides insights into the model's stability and adaptability. Here’s a brief overview: https://lnkd.in/ekB_q9XX #MachineLearning #DeepLearning #NaturalLanguageProcessing #LoRA #FineTuning #AI #DataScience #ModelOptimization #NeuralNetworks #ArtificialIntelligence #TechInnovation #DataDriven #KnowledgeIntegration #Research #LLMs

30

4 Comments

RLHF was thought to be the secret behind aligning ChatGPT to give 'smart' responses. Implementing RLHF using PPO optimization is a complex engg task but initially there seemed to be no way around it. 🛠️ The road from the original complex RLHF implementation to the more recent simpler model fine-tuning/alignment strategies is long and unwinding. 👉 The initial RLHF setup used 3 (or 4?) copies of the pretrained model. First is the next-token pretrained model (also called SFT model), one for external reward model and one for the new model being fine-tuned. PPO algorithm was used for learning. Led to many asking do we really need RLHF/PPO ? 👉 DPO got rid of the external reward model by reformulating the loss fn and reduced the copies to two - the original SFT model and the new model being trained. 👉 SPO finally reduced the copies to a single one. They construct a new intrinsic loss function which trains the model while learning from (good, bad) samples. The average log probability of a sequence is used as the implicit reward, same as the usual generation likelihood. Very clean formulation and writing. 🤔 In hindsight, SPO is obvious but why so long? (ORPO was close enough). Two key ideas seem to make it work: length normalization of loss and an extra margin parameter. 🤨 While both DPO and SPO claim full victory on compute and memory efficiency, the jury is still out on whether they yield “better” models than PPO (used in RLHF) setting, on say downstream reasoning tasks like GSM8k. 🎇 I really wish that implementing alignment training becomes as easy as next-token training. Hope that SPO holds its ground and whatever remains of PPO can be brought back to SPO++. --- Stay informed about latest LLM updates with our newsletter: offnote.substack.com

39

3 Comments