This guide shows you how to extract structured data—like invoice numbers, dates, line items, and totals—from PDF invoices using Python, then validate that extracted data for accuracy and completeness. You’ll learn a practical pipeline using popular libraries like pdfplumber for text extraction and pandas for validation, with clear steps you can adapt to your own invoice formats.
What You’ll Need
- Python 3.8 or newer installed on your system.
- A PDF invoice file (or several) to test with.
- Install required libraries:
pip install pdfplumber pandas - Basic familiarity with Python and command line.
Step-by-Step
- Set up your project folder. Create a directory, e.g.,
invoice_extractor/, and place your PDF invoices inside a subfolder likeinvoices/. - Write a function to extract text from a PDF. Use
pdfplumberto open the PDF and concatenate text from all pages. For example:import pdfplumber def extract_text(pdf_path): with pdfplumber.open(pdf_path) as pdf: return "\n".join(page.extract_text() or "" for page in pdf.pages) - Define regex patterns for key fields. Common invoice fields include invoice number, date, total amount, and vendor name. For instance:
import re patterns = { "invoice_number": r"Invoice\s*#?\s*([A-Z0-9-]+)", "date": r"Date\s*[:]?\s*(\d{1,2}/\d{1,2}/\d{2,4})", "total": r"Total\s*[:$]?\s*([\d,]+(?:\.\d{2})?)" }Adjust these to match your invoice layout.
- Extract data using the patterns. Write a function that applies the regex to the extracted text and returns a dictionary of field-value pairs. Handle cases where a field is missing by setting it to
None. - Handle line items. If your invoices contain a table of items, use
pdfplumber‘sextract_table()method. For example:with pdfplumber.open(pdf_path) as pdf: page = pdf.pages[0] table = page.extract_table()Then convert the table into a list of dictionaries with column names.
- Store extracted data in a structured format. Use
pandasto create a DataFrame for all invoices. Each row represents one invoice with columns for each field. - Validate the extracted data. Write validation checks:
- Required fields: Ensure invoice number, date, and total are not missing.
- Date format: Check that dates parse correctly using
datetime.strptime(). - Numeric totals: Convert total strings to floats and check they are positive.
- Consistency: If line items are extracted, verify that the sum of line item amounts matches the total (within a small tolerance).
- Output the results. Save the validated data to a CSV or Excel file using
df.to_csv()ordf.to_excel(). Include a column for validation status (e.g., “Pass” or “Fail”) and error messages. - Test on multiple invoices. Run your script on a sample of invoices to see how well it handles variations. Adjust regex and table extraction as needed.
Common Mistakes
- Hardcoding regex patterns that are too specific. Invoice formats vary; use flexible patterns and test on multiple samples.
- Ignoring multi-page invoices. Some invoices span multiple pages; make sure to extract text from all pages, not just the first.
- Assuming tables are always perfectly structured. PDF tables often have merged cells or missing borders; use
extract_table()with caution and clean the output. - Not handling missing data gracefully. Your code should not crash when a field is absent; return
Noneand flag it. - Forgetting to strip whitespace and special characters. Extracted text may contain extra spaces or newlines; clean strings before parsing.
Tips
- Use
pdfplumber‘sextract_words()to get word-level data if you need to locate fields by position. - For complex layouts, consider using
camelotortabula-pyfor table extraction, but they require Java or external dependencies. - Always keep a backup of the original PDFs; extraction is not lossless.
- Use logging to track which invoices failed validation and why.
- For recurring invoices from the same vendor, you can create a template with fixed coordinates for faster extraction.
Troubleshooting
- No text extracted: The PDF may be scanned (image-based). Use OCR with
pytesseractorpdf2imageto convert pages to images first. - Regex not matching: Print the extracted text to see the exact format. Use regex101.com to test patterns.
- Table extraction returns None: Try
page.find_tables()to see if tables are detected. Adjust table settings likevertical_strategy. - Date parsing fails: Some invoices use formats like “2023-01-15” or “Jan 15, 2023”. Write a function that tries multiple formats.
- Total mismatch with line items: This could be due to discounts, taxes, or rounding. Include a tolerance (e.g., 0.01) and flag only significant discrepancies.
- Memory issues with many PDFs: Process invoices one by one and store results incrementally rather than loading all at once.
Frequently Asked Questions
Can I extract data from scanned invoices?
Yes, but you need OCR. Use pytesseract with Tesseract installed, or cloud OCR services. The accuracy depends on image quality.
How do I handle invoices with different layouts?
Create a set of regex patterns and table extraction strategies for each layout, then detect the layout based on keywords or structure. Alternatively, use a machine learning approach with training data.
What if the invoice has no line items, just a total?
That’s fine; your validation should only check required fields. Adjust your extraction to skip table parsing if no table is found.
Can I use this for currencies other than USD?
Yes, but adjust the regex to match currency symbols and decimal separators. For example, European formats use commas for decimals.
Is pdfplumber the best library for this?
It’s a good open-source choice for text-based PDFs. For complex tables, consider camelot. For scanned docs, use OCR tools.
How do I validate that the total is correct?
Sum the line item amounts and compare to the total field. Allow a small tolerance for rounding. If they don’t match, flag it for manual review.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.
More cards in this drawer
- Tongue Escape Codes: What They Are0.20 movements/s
- Claude Code Workflow Habits Worth Trying
- HDMI 2.1 Explained: Why It Matters for Gaming
- Veeam Plugin for VMware Cloud Director: How It Works
- Best Path of Titans Server Hosting: Top Picks
- Oppo First Foldable: 120Hz Display & Snapdragon 888