Spot New AI Models on Arena Leaderboard with a Scraper

This guide shows you how to build a small Python scraper that watches the LMArena (formerly Chatbot Arena) leaderboard and alerts you when a new AI model appears. You’ll learn to fetch the page, parse the model names, compare them against a stored list, and get notified—all with about 50 lines of code. No prior scraping experience is needed, but basic Python knowledge helps.

What You’ll Need

• Python 3.8 or newer installed on your computer
• Internet connection to access the leaderboard
• A text editor or IDE (like VS Code, PyCharm, or even Notepad)
• Optional: a Telegram bot token or email SMTP settings if you want push notifications

Step-by-Step

1. Set up a project folder. Create a new directory on your computer, e.g., arena-scraper. Inside it, create a file named scraper.py.
2. Install required libraries. Open a terminal in that folder and run: pip install requests beautifulsoup4. These let you fetch the webpage and parse HTML.
3. Fetch the leaderboard page. In scraper.py, import the libraries and write a function that uses requests.get() to retrieve the page. The URL is https://lmarena.ai/leaderboard (or https://chat.lmsys.org/?leaderboard). Add a User-Agent header to avoid being blocked.
4. Parse the HTML to extract model names. Use BeautifulSoup to find the table rows. Inspect the page in your browser to see the structure—model names are usually in <td> elements with a specific class or within links. A common approach is to find all <td> with class model-name or similar. If that fails, look for <div> elements with the text pattern like “Model (Arena Score)�.
5. Create a list of current models. Iterate through the parsed elements, extract the text, strip whitespace, and store them in a Python list. Print them to verify you got the right data.
6. Store the previous snapshot. Create a file named known_models.txt in the same folder. On first run, write the current list to it. On subsequent runs, read this file to get the old list.
7. Compare and detect new models. After fetching the current list, compute the difference: new_models = set(current) - set(old). If new_models is not empty, you have new entries.
8. Notify yourself. For each new model, print a message like “New model detected: GPT-5�. For a more useful alert, you can send a Telegram message using the requests library to the Bot API, or write to a log file. Simple print is enough to start.
9. Update the snapshot. After processing, overwrite known_models.txt with the current list so the next run only sees truly new models.
10. Schedule the script. To run automatically, use a cron job (Linux/macOS) or Task Scheduler (Windows) to execute python scraper.py every hour or daily. For testing, run it manually a few times.

Common Mistakes

• Hardcoding the HTML structure. The leaderboard markup can change. Instead of relying on exact class names, use more flexible selectors like table tr td:first-child or search for text patterns.
• Not handling page changes. If the site adds pagination or lazy-loading, your scraper might miss models. Check if the full list is present in the initial HTML; if not, you need to use a headless browser like Selenium (but that’s more complex).
• Ignoring rate limits. Hitting the site too frequently may get your IP blocked. Add a delay between requests (e.g., time.sleep(60) if you run every minute).
• Forgetting to update the file. If you don’t overwrite the snapshot, you’ll get notifications for the same models every time.
• Using print only for output. If the script runs in the background, you won’t see prints. Use a log file or a notification service.

Tips

• Use a virtual environment. Keep dependencies isolated by running python -m venv venv and activating it before installing packages.
• Add error handling. Wrap the network request in a try/except to catch timeouts and connection errors. This prevents the script from crashing.
• Make it a function. Structure your code with functions like fetch_models() and compare_models() for easier testing.
• Log to a file. Use Python’s logging module to write to arena.log with timestamps—useful for debugging.
• Check the page structure first. Open the leaderboard in your browser, right-click on a model name, and select “Inspect� to see the exact HTML. This saves trial and error.
• Consider using the official API. LMArena has a public API endpoint (/api/leaderboard) that returns JSON. If that works, it’s much more reliable than scraping HTML. Try https://lmarena.ai/api/leaderboard in your browser first.

Troubleshooting

• Script returns empty list. The page structure might have changed. Re-inspect the HTML and update your selector. Alternatively, try the API endpoint mentioned above.
• HTTP 403 Forbidden. The site is blocking your request. Add a browser-like User-Agent header, e.g., headers = {'User-Agent': 'Mozilla/5.0'}. Also try adding a Referer header.
• Script runs but no new models ever. You might be comparing the same list because you’re not updating the file correctly. Ensure you write the current list after each run, and that the file path is correct.
• Notifications don’t arrive. If using Telegram, double-check your bot token and chat ID. For email, verify SMTP settings and that you’ve allowed less secure apps (if using Gmail).
• Script crashes with “No module named bs4�. You haven’t installed BeautifulSoup. Run pip install beautifulsoup4 again in the correct environment.
• Page loads but models are in a JavaScript-rendered table. If the HTML you fetch doesn’t contain model names, the site uses JavaScript to load data. You’ll need to use Selenium or Playwright to render the page, or find the underlying API call in the browser’s Network tab.

Frequently Asked Questions

Is scraping the leaderboard allowed?

LMArena’s terms of service don’t explicitly prohibit scraping, but they may have rate limits. Be respectful: run the script every few hours, not every second. If you’re concerned, use the official API if available.

Can I get notified via email?

Yes. Use Python’s smtplib to send an email when a new model is found. You’ll need an SMTP server (e.g., Gmail’s) and your credentials. Be careful with security—use app-specific passwords.

What if the leaderboard changes its URL?

Monitor the LMArena website or their social media for announcements. Update the URL in your script accordingly. The scraper will fail with a 404 or empty list if the URL is wrong.

How often should I run the script?

New models are added irregularly—sometimes daily, sometimes weekly. Running every 6-12 hours is a good balance between timeliness and being polite to the server. You can adjust based on how quickly you need to know.

Can I also track model removals?

Yes. Instead of just new_models, also compute removed_models = set(old) - set(current) and print those. This helps you see if a model was taken down.

Do I need to know HTML to do this?

Basic familiarity helps, but you can inspect the element in your browser and copy the selector. The guide gives you a starting point; you may need to tweak it based on the current page structure.

Related

• Stop Google Translate Saving Transcripts
• Rochester Electronics Alternatives: EOL Parts Guide
• Galaxy S9 Won’t Activate? Fix It in 8 Steps