Archive · 12 Nov 2025
IIT Madras AI4Bharat Launches Indic LLM-Arena for Indian Languages
Imagine computers learning to understand and talk like people in India, which means understanding many different languages and how people mix them, like speaking Hindi and English together.
AI4Bharat at IIT Madras has created a new tool called Indic LLM-Arena to test how good AI models are at this.
It works by letting lots of people anonymously compare responses from different AI models, like voting for the better answer in a game.
This helps rank the AIs based on what real people think is best for Indian languages and culture.
They want to make this tool open for everyone to use and improve.
This is important because India has many languages, and AI needs to understand them to be truly useful for everyone here.
This will help companies choose the best AI for their needs and make sure AI serves Indian customers well.
AI4Bharat at IIT Madras has launched the Indic LLM-Arena, an open-source benchmark for testing AI models on Indian languages and context.
The benchmark uses a crowd-sourced, human-in-the-loop approach where anonymous users vote on LLM responses.
Indic LLM-Arena supports text-based inputs and code-mix scenarios, with plans to expand to vision and audio capabilities.
The platform aims to evaluate LLMs on natural Indian communication styles, including language mixing.
The initiative provides a leaderboard for researchers and businesses to compare and select AI models for Indian use-cases.
- Who
- AI4Bharat, a research lab at IIT Madras
- What
- Introduced Indic LLM-Arena, an open-source benchmark test to evaluate Large Language Models (LLMs) on Indian languages, context, and safety.
- When
- Announced on Monday, November 10, 2025
- Where
- IIT Madras, India
- Why
- To address the lack of local benchmarks for evaluating Indic LLMs, enabling researchers and businesses to compare model performance on Indian-specific use-cases and languages.
Focus on Linguistic Nuance and Context
Open-sourcing for Broader Accessibility
Evaluating Indian Communication Styles
Focus on Linguistic Nuance and Context
The benchmark aims to evaluate how well LLMs understand the natural, fluid way Indians communicate, including code-switching (mixing languages like Hinglish or Tanglish).
Open-sourcing for Broader Accessibility
All anonymized data, code, and pipelines will be released under an open-source license to allow community inspection and further development.
- Developer
- AI4Bharat, IIT Madras
- Support
- Google Cloud
- Benchmark Name
- Indic LLM-Arena
- Evaluation Method
- Crowd-sourced, human-in-the-loop, blind, side-by-side comparison
- Current Capabilities
- Text-based inputs across multiple Indian languages and code-mix scenarios
- Future Plans
- Expansion to omni models (vision, audio) and AI agents
- Release Type
- Open-source
