Aditya Malte

Microsoft Research introduced OmniParser – a breakthrough in multimodal large model for GUI agents, now available on Hugging Face! OmniParser is a compact screen parsing module that leverages the power of large vision-language models like GPT-4V to transform UI screenshots into structured elements. This enables AI agents to interact with user interfaces (UI) effectively using purely visual inputs. With OmniParser, researchers are able to tackle some of the toughest challenges in GUI-based agent technology: Reliable identification of interactive icons within the UI. Deep understanding of UI semantics to accurately map actions to specific regions on the screen. Powered by models like GPT-4V, OmniParser takes intelligent agents to the next level, enabling them to perform precise actions across diverse operating systems and applications. In fact, it recently achieved top performance in the WindowsAgentArena benchmark! 🎉 OmniParser is now available open-source on GitHub!  https://lnkd.in/gZge9yta It's an interesting read for vision language model researchers and enthusiasts. #MicrosoftResearch #AI #MachineLearning #VisionLanguageModels #OmniParser #GPT4V #HuggingFace #ArtificialIntelligence #UserInterface #ScreenParsing

11

1 Comment

Is GPT-4o good at table recognition? Not quite! OCR looks fine, but tripped by spanning cell table structure! 🤩 However, our UniTable, with only 124M param trained by self-supervision, shows great understanding! 👉 UniTable paper: arxiv.org/abs/2403.04822 👉 UniTable code: https://lnkd.in/e4a7YMxJ #MachineLearning #GPT-4o #Table #Document

83

7 Comments

🎉 Excited to Share a New Achievement! 🎉 I’m thrilled to have earned the NVIDIA Generative AI LLMs (NCA-GENL) Certification! 🚀 This certification validates my foundational understanding of developing, integrating, and maintaining AI-driven applications using Generative AI and Large Language Models (LLMs) with NVIDIA solutions. Over the past few months, I’ve delved deep into a variety of resources, including courses and webinars on topics such as Hugging Face's NLP, Building RAG Agents for LLMs, Diffusion Models, DeepLearning.AI's Prompt Engineering, and LangChain for LLM Application Development, to name a few. The journey has been challenging, but incredibly rewarding. I’m particularly excited about the knowledge I’ve gained regarding NVIDIA’s AI tech stack, including DGX, AI Enterprise, NeMo, NIM, and TensorRT. This certification not only strengthens my understanding of the current AI landscape but also equips me with the skills to contribute to developing cutting-edge AI applications. A big thank you to all the amazing resources and communities that helped me along the way. I look forward to applying this knowledge to future projects and continuing to grow in this exciting field! #NVIDIA #GenerativeAI #LLMs #Certification #AI #DeepLearning #MachineLearning #LangChain #HuggingFace

56

3 Comments

Continuing from my last blog, Machine Learning in Production: Key Takeaways from My Latest Course, where I explored deployment strategies such as canary and blue-green deployments, I’m now shifting focus toward an equally critical aspect of AI development: model-centric vs. data-centric AI development. These two approaches can greatly influence how successful your models will be when deployed in production, and understanding them can optimize your workflow. Let's break them down in more detail.

6

🔎 Demystifying LLMs: A New Video Series! 🔎 Struggling to wrap your head around Large Language Models (LLMs)? I feel you! That's why I'm launching a new video series, LLM Explained, in collaboration with Uthiralakshmi S. from Worcester Polytechnic Institute. Each episode will be a bite-sized 2-minute explainer, tackling a specific LLM concept. This week, we're diving into the foundations: tokenization and embedding! Whether you're a seasoned data scientist or just starting your AI journey, this series is designed to make LLMs accessible and engaging.✨ Stay tuned for the next episode! In the meantime, hit that follow button and join the conversation! If you learned something valuable, share it with your network! 👨‍💻 👥 #GenAI #LLM #LargeLanguaeModels #DeepLearning #NeuralNetwork #AI #MachineLearning #Tokens #Tokenization #Embedding #Education #AIEducation #Data #DataScience #NEU #WPI

18

2 Comments

⏫ Leveled up my understanding of retrieval systems (R+AG) 🔍 and put those learnings to work by improving GitHub Issues management! Here’s a breakdown of what I explored: ✨ Embedding-based Retrievals – How embeddings drive more meaningful search by mapping concepts into vector spaces. ⚠️ Pitfalls of Simple Vector Search – Identified common issues like irrelevant results and learned how to tackle them effectively. 🔍 Query Expansion with LLMs – Used Large Language Models to broaden search coverage and uncover deeper insights. ⚖️ Cross-Encoder Re-Ranking – Tuned result rankings with cross-encoders to bring the best matches to the top. 🛠️ Embedding Adaptors – Learned how to tweak embeddings for different tasks to keep retrieval systems flexible and efficient. Super excited to apply these concepts further! If you're working on retrieval systems, let’s connect and exchange ideas! 🚀 #AI #ChromaDB #Retrieval #LLMs #GitHub

11

5 Comments

Hi everyone! I’m graduating this December with an MS in Applied Statistics & Data Science from The University of Texas at Arlington, and I’m looking for full time roles in software engineering, and data-science adjacent domains. Until June last year, I gave my all to building awesome healthcare experiences at Clinikk and learned from a great team about building with empathy, conviction and ownership. In that time, I learned about EHR systems, insurance, inventory management, and building a healthy engineering team culture, among a million other things you only learn when you have a front-row seat to how a startup works. I also grew as a T shaped engineer: I can work across the stack, but have deep knowledge in/passion for great frontend experiences, and I build with JS/TS. I know my way around React, Remix, Next and Svelte, and would love to learn Angular or Vue, if that’s what you use! In the end, the feature parity among all these tools has taught me that it’s basically a matter of picking what *you* feel most productive in, and having the drive to build tools that help others reliably do the best they can at work, every day. Over my graduate studies at UTA, I’ve gotten more data-literate. I can help you crunch numbers, design user studies, do experimentation on your products, and tell rich, visual stories. Whether these stories are meant to rally your team behind a shared vision, or to build a trusted brand customers want to use, I’m super interested in that too! I can work in R, Python, VBA, and SAS. If you could use my skills but work with other tools, you know I’m hungry to learn - give me a shot! I’ve also learned how to build and deploy ML models, and love building small tools to automate most things I can. Ask me about some and I’d love to share more! (Or you can take a peek at https://lnkd.in/gQBA3mnG) If you like what you hear and think you have a role that I might fit into, please reach out! TL;DR: software engineer, starry-eyed 🤩 data science grad student looking for a job starting December 2024. Can run circles around web engineering problems and jog around data science ones, but I’m in it for the long run. Attracted to interesting problems and not bound by tech stacks. #healthcare #fhir #ehr #datascience #fintech #banking #nodejs #react #svelte #python #dataanalytics

73

6 Comments

I’m thrilled to announce that I’ll be serving as a Chair of the NAACL 2025 Industry Track, and our Call for Papers is now open! https://lnkd.in/gBM2F4Fv If you're working on real-world NLP applications, I highly encourage you to submit your work. With the rapid development observed recently in Generative Ai and large language models (LLM), this year we are particularly encouraging submissions of papers leveraging this technology to develop real-world applications. Potential topics include, but are not limited to: - Best practices and lessons learned - Continuous maintenance and improvement of deployed systems - Case studies from design to deployment - Design of application-relevant datasets - Ethics and bias in deployed NLP systems - Handling unexpected user behavior - Implementation at speed, scale, and low-cost - Negative results related to real-world applications - Novel previously unsolved NLP problems - Offline and online system evaluation methodologies - Online learning for deployed systems - Robustness against various perturbations in deployed systems - System combination and hybridization In addition, opinion/vision papers related to real-world applications are also welcome! The conference will be held in Albuquerque #naacl2025 #industrytrack #nlproc

64

3 Comments