AI Agent Screenshots of Indian Job Listings

This guide walks you through using an AI agent (like a browser automation tool) to automatically capture screenshots of job listings from Indian job portals such as Naukri, Indeed India, or LinkedIn. You’ll set up a simple script that visits a search URL, waits for the page to load, and saves a clean screenshot of each listing—perfect for building a visual job board or tracking openings over time.

What You’ll Need

• A computer with Python installed (or Node.js if you prefer JavaScript).
• An AI agent framework or library—for this guide we’ll use Playwright (Python) because it’s free, reliable, and has built-in screenshot features.
• Basic familiarity with running commands in a terminal.
• An internet connection and access to the job portal you want to capture (some sites may require login—see Troubleshooting).

Step-by-Step

1. Install Playwright: Open your terminal and run pip install playwright. Then install the browser binaries with playwright install chromium. This downloads a headless browser that the agent will control.
2. Write a basic script: Create a new file named job_screenshot.py. Start with the following skeleton:

   from playwright.sync_api import sync_playwright
   
   with sync_playwright() as p:
       browser = p.chromium.launch(headless=True)
       page = browser.new_page(viewport={'width': 1280, 'height': 800})
       # ... your code here ...
       browser.close()
       

3. Navigate to a search results page: Use page.goto('https://www.naukri.com/software-jobs?k=python') (replace with your target URL). For Indeed India, use something like https://in.indeed.com/jobs?q=python.
4. Wait for the page to load: Job listings often load dynamically. Use page.wait_for_selector('div.job_seen_beacon') for Indeed, or page.wait_for_selector('.jobTuple') for Naukri. This ensures the content is present before screenshotting.
5. Take a full-page screenshot: Use page.screenshot(path='job_listing.png', full_page=True). This captures the entire scrollable page, not just the visible viewport.
6. Capture individual listings (optional but better): Instead of one long screenshot, loop through each job card element and screenshot it individually. Example for Indeed:

   listings = page.query_selector_all('div.job_seen_beacon')
   for i, listing in enumerate(listings):
       listing.screenshot(path=f'job_{i}.png')
       

7. Add a delay between captures: Use page.wait_for_timeout(2000) to avoid being blocked by the site and to let any lazy-loaded content appear.
8. Handle pagination: To capture multiple pages, find the ‘Next’ button and click it. For example, on Naukri: page.click('a:has-text("Next")'). Then repeat the screenshot loop.
9. Run the script: Execute python job_screenshot.py in your terminal. The screenshots will be saved in the same folder as the script.
10. Review and organize: Check the saved images. You may want to rename them with the job title or date, e.g., job_20250325_1.png. You can also add a timestamp to the filename in the script.

Common Mistakes

• Not waiting for elements: Screenshotting before the page fully loads results in blank or partial images. Always use wait_for_selector with a specific job card selector.
• Using the wrong selector: Job portal class names change frequently. Verify the current selector by inspecting the page in your browser (right-click → Inspect).
• Ignoring login walls: Some portals (like LinkedIn) require login to view full listings. Your script will hit a login page—you’ll need to handle authentication (see Troubleshooting).
• Taking only a viewport screenshot: By default, page.screenshot() captures only the visible area. Use full_page=True or element-specific screenshots to get the whole listing.
• Not handling pop-ups: Many Indian job sites show location or notification pop-ups. Add a step to close them, e.g., page.click('button:has-text("Not Now")') if present.

Tips

• Use a headful browser for debugging: Set headless=False while testing so you can see what the agent sees. Switch back to headless for production.
• Add a user agent: Some sites block default headless browsers. Set a realistic user agent in the browser launch options, e.g., user_agent='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'.
• Use a proxy or rotate IPs if you’re scraping many pages: For heavy use, consider a proxy service to avoid IP bans. For a few dozen screenshots, this is usually unnecessary.
• Organize screenshots into folders: Create a new directory for each run using the date: os.makedirs('screenshots_20250325', exist_ok=True) and save files inside.
• Combine with OCR (optional): If you need to extract text from the screenshots, you can later run them through an OCR tool like Tesseract.

Troubleshooting

• Blank screenshots: The page didn’t load properly. Increase the wait time, or use page.wait_for_load_state('networkidle') before taking the screenshot.
• Site blocks the script: You see a CAPTCHA or a 403 error. Try adding a longer delay between actions, use a real user agent, or switch to a different browser (e.g., Firefox via Playwright). If the site is aggressive, you may need to use a paid scraping service.
• Login required: For sites like LinkedIn, you can either manually log in once in a non-headless browser and save the session cookies, or use a tool like playwright-stealth to avoid detection. Simpler: use a job portal that doesn’t require login for basic search (Naukri and Indeed usually work without login for viewing listings).
• Element not found: The selector you used is outdated. Re-inspect the page and update the selector in your script. Use a more generic selector like div[class*='job'] if needed.
• JavaScript-heavy pages: Some portals load listings via infinite scroll. Instead of pagination, you may need to scroll down repeatedly: page.mouse.wheel(0, 5000) and wait, then screenshot.

Frequently Asked Questions

Is this legal?

Capturing screenshots for personal use is generally fine, but automated scraping may violate a site’s terms of service. Check the portal’s robots.txt and terms. For personal, non-commercial use, it’s usually acceptable, but avoid aggressive scraping.

Can I use this for any Indian job site?

Yes, the same Playwright script works for most job portals. You just need to adjust the search URL and the CSS selectors for job cards. Sites like Naukri, Indeed, Shine, and Monster are all accessible.

How do I capture only the job description, not the whole page?

Instead of a full-page screenshot, locate the specific element containing the job description (e.g., div.job-description) and use element.screenshot(). This gives a clean, focused image.

Can I schedule this to run daily?

Yes. Use a task scheduler (like cron on Linux/Mac or Task Scheduler on Windows) to run your Python script at a set time. Make sure your computer is on and connected to the internet.

What if the job listing is behind a login?

You have two options: (1) Use a portal that doesn’t require login for search results (most allow viewing without login). (2) Automate login by storing your credentials in environment variables and using page.fill() to enter them. Be careful with credentials—never hardcode them in the script.

Can I extract text from the screenshots?

Yes, but that’s a separate step. After capturing the screenshots, you can use an OCR library like pytesseract to extract text. Alternatively, you can use Playwright’s inner_text() method to grab the text directly from the page without screenshots.

Related

• Unrecognized Database Format: Causes & Fixes
• Unified Write Filter: What It Is & How It Works
• DDNS Dynamic DNS Service Explained