Python requirements.txt: Key Things to Know

Python’s requirements.txt file lists the packages your project depends on, so anyone can recreate the same environment with one command. You write the package names and versions, and pip install -r requirements.txt installs them. This guide covers the essentials: syntax, version pinning, common pitfalls, and best practices.

What Is requirements.txt and Why Use It?

A requirements.txt file is a plain text file that tells pip which Python packages to install. It is the simplest way to share dependencies for a project, whether you’re working alone or with a team. Instead of manually installing each package, you run one command and get the exact environment you need.

The file lives at the root of your project (or wherever you decide) and is typically committed to version control. This means every collaborator and every deployment server can install the same dependencies, avoiding the classic “it works on my machine” problem.

Basic Syntax and Format

Each line in requirements.txt is a package specification. The simplest form is just the package name:

requests
numpy

You can also specify a version with == (exact), >= (minimum), <= (maximum), or ~= (compatible release, meaning at least the given version but less than the next major version). Examples:

requests==2.31.0
numpy>=1.24
pandas~=2.0

You can also use > and < for strict inequalities, but be careful: they can lead to unexpected upgrades. The most common and safest practice is to pin exact versions with ==.

How to Generate requirements.txt

You can write the file by hand, but it’s error-prone. The standard way is to use pip freeze:

pip freeze > requirements.txt

This outputs all installed packages and their exact versions. However, it includes every package in your current environment, not just the ones your project uses. That’s why many developers use pipreqs or pip-tools to generate a cleaner list based on actual imports.

If you use a virtual environment (which you should), activate it first, then run pip freeze. That way you only capture the packages relevant to that project.

Version Pinning and Why It Matters

Pinning means specifying exact versions for every dependency. For example:

django==4.2.5

Why pin? Because a package update can introduce breaking changes. If you leave the version unpinned, you might get a newer version later that breaks your code. Pinning ensures reproducibility: the same file always installs the same versions.

But pinning has a downside: you won’t get security patches or bug fixes automatically. You need to actively update the file. A common compromise is to use ~= for compatible releases, which allows patch updates but not minor or major changes. For example, django~=4.2.5 allows 4.2.6 but not 4.3.

Common Problems and Solutions

• Unpinned dependencies: If you don’t pin, a future install might pull a different version, causing subtle bugs. Solution: always pin, or use a lock file like pip-tools generates.
• Platform-specific packages: Some packages have different versions for Windows vs Linux. You can use environment markers like pywin32; sys_platform == 'win32' to conditionally include them.
• Transitive dependencies: pip freeze lists all packages, including ones you didn’t directly install. If you only want direct dependencies, use pipreqs or manually curate the file.
• Comments and blank lines: You can add comments with # and blank lines for readability. They are ignored by pip.
• Local packages: If you need to install a package from a local path or a VCS, you can specify it with -e . (editable) or git+https://....

Best Practices for Managing Dependencies

1. Use a virtual environment for each project. This isolates dependencies and prevents conflicts.
2. Pin exact versions for reproducible builds. For applications, exact pins are ideal. For libraries, you might use compatible ranges.
3. Separate development dependencies into a different file, like requirements-dev.txt, which can include testing tools like pytest or black.
4. Use pip-tools or poetry for more advanced dependency management. These tools generate a lock file that pins all transitive dependencies too.
5. Update regularly but deliberately. Check for outdated packages with pip list --outdated and update one at a time, testing as you go.

Alternatives to requirements.txt

While requirements.txt is the most common, there are other tools:

• Pipenv: Uses a Pipfile and Pipfile.lock for more robust dependency management, similar to npm’s package.json.
• Poetry: Uses pyproject.toml and poetry.lock. It handles virtual environments and packaging, and is popular for libraries.
• conda: For data science, conda environments use environment.yml and can manage non-Python dependencies.

These tools offer better lock files and dependency resolution, but requirements.txt remains the simplest and most universal approach.

Frequently Asked Questions

What is the difference between pip freeze and pipreqs?

pip freeze lists every package installed in the current environment, including transitive dependencies. pipreqs scans your project’s imports and generates a minimal list of direct dependencies. Use pip freeze for a complete snapshot, and pipreqs for a clean starting point.

Should I commit requirements.txt to version control?

Yes, absolutely. Committing it ensures everyone uses the same dependencies. It also helps with deployment and debugging. Just make sure it’s kept up to date.

How do I update a single package in requirements.txt?

Change the version number in the file, then run pip install -r requirements.txt again. Or use pip install --upgrade package_name and then update the file manually or with pip freeze.

Can I use wildcards or ranges in requirements.txt?

Yes, you can use * for any version, but that’s risky. Better to use >= or ~= for ranges. For example, requests>=2.0 allows any version above 2.0, but requests~=2.0 allows 2.x but not 3.0.

What does -e . mean in requirements.txt?

It means “editable install” of the current directory. It’s used for local development, typically when you’re working on a package itself. It installs the package in development mode, so changes to the code take effect immediately.

Why do I get “Could not find a version that satisfies the requirement”?

This error usually means the package name is misspelled, the version doesn’t exist, or the package is not available for your Python version or platform. Double-check the name and version, and ensure you’re using a compatible Python version.

Related

• Zimbra Backup Mailbox: Full Guide
• Brexit Divide in UK Tech: How It Split the Industry
• Embedded Analytics Match Customer Needs to the Right Experience?