Python pip、requirements.txt と仮想環境

Managing Python dependencies is one of the first hurdles every developer faces. Whether you are installing packages with pip, pinning versions in requirements.txt, or isolating projects with virtual environments, getting the workflow right saves hours of debugging. This guide covers everything you need to know about pip, requirements files, virtual environments, and modern tools like pip-tools, pyproject.toml, and uv.

pip Basics: Essential Commands

pip is the standard package installer for Python. It connects to the Python Package Index (PyPI) and installs packages into your Python environment. Here are the commands you will use every day.

Check your pip version:

# Check pip version
pip --version
# pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)

# Check which Python pip is linked to
pip -V
python -m pip --version

Install, uninstall, and inspect packages:

# Install a package
pip install requests

# Install a specific version
pip install requests==2.31.0

# Uninstall a package
pip uninstall requests

# List all installed packages
pip list

# Show details about a specific package
pip show requests
# Name: requests
# Version: 2.31.0
# Location: /home/user/.venv/lib/python3.12/site-packages
# Requires: certifi, charset-normalizer, idna, urllib3

# List outdated packages
pip list --outdated

Upgrade pip itself and packages:

# Upgrade pip itself
python -m pip install --upgrade pip

# Upgrade a package to the latest version
pip install --upgrade requests

# Upgrade all outdated packages (bash one-liner)
pip list --outdated --format=columns | tail -n +3 | awk '{print $1}' | xargs pip install --upgrade

Search and get package info:

# pip search is disabled on PyPI — use the website or:
pip index versions requests
# Available versions: 2.31.0, 2.30.0, 2.29.0, ...

# Show package info without installing
pip show requests

# Download without installing (useful for offline installs)
pip download requests -d ./packages/

requirements.txt: Format and Version Pinning

A requirements.txt file lists all the packages your project depends on. It is the standard way to share and reproduce Python environments.

Basic format:

# requirements.txt

# Pinned versions (recommended for production)
requests==2.31.0
flask==3.0.0
sqlalchemy==2.0.23

# Minimum version
numpy>=1.24.0

# Compatible release (allows patches: >=1.4.0, <2.0.0)
django~=4.2.0

# Version range
celery>=5.3.0,<6.0.0

# Install from git repository
git+https://github.com/user/repo.git@main#egg=mypackage

# Install from local file
./libs/my-local-package/

# Include another requirements file
-r requirements-base.txt

Generate requirements.txt from your current environment:

# Generate from current environment (includes ALL packages)
pip freeze > requirements.txt

# The output looks like:
# certifi==2023.11.17
# charset-normalizer==3.3.2
# idna==3.6
# requests==2.31.0
# urllib3==2.1.0

# WARNING: pip freeze includes transitive dependencies too
# For cleaner results, use pip-tools (see below)

Install all dependencies from requirements.txt:

# Install all dependencies from requirements.txt
pip install -r requirements.txt

# Install from multiple files
pip install -r requirements.txt -r requirements-dev.txt

# Dry run — show what would be installed without installing
pip install -r requirements.txt --dry-run

Virtual Environments: Why You Need Them

A virtual environment is an isolated Python installation that keeps project dependencies separate. Without it, installing a package for one project can break another. Virtual environments solve the "it works on my machine" problem.

Why Virtual Environments Matter

• Different projects can use different versions of the same package.
• You do not need admin/root privileges to install packages.
• You can reproduce exact environments on any machine.
• Uninstalling a project means just deleting the venv folder.

Creating and Using venv

The venv module is built into Python 3.3+:

# Create a virtual environment named ".venv"
python -m venv .venv

# Or with a custom name
python3 -m venv myproject-env

# Create with access to system packages
python -m venv --system-site-packages .venv

# Create with a specific Python version (if multiple installed)
python3.12 -m venv .venv

Activate the virtual environment:

# Linux / macOS (bash/zsh)
source .venv/bin/activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Windows (cmd.exe)
.venv\Scripts\activate.bat

# Fish shell
source .venv/bin/activate.fish

# Your prompt changes to show the active venv:
# (.venv) user@host:~/project$

# Verify you are using the venv Python
which python    # Linux/macOS
where python    # Windows
# Should point to .venv/bin/python or .venv\Scripts\python.exe

Deactivate when you are done:

# Deactivate the virtual environment
deactivate

# Your prompt returns to normal:
# user@host:~/project$

Always add the venv folder to .gitignore:

# .gitignore
.venv/
venv/
env/
.env/

# Also ignore compiled Python files
__pycache__/
*.py[cod]
*.egg-info/

venv vs virtualenv vs conda: Which Should You Use?

Python has several tools for creating virtual environments. Here is when to use each one.

# virtualenv — install and use
pip install virtualenv
virtualenv .venv
source .venv/bin/activate   # same activation as venv

# conda — create and manage environments
conda create -n myproject python=3.12
conda activate myproject
conda install numpy pandas
conda deactivate

# conda — export and reproduce environment
conda env export > environment.yml
conda env create -f environment.yml

pip install Options: Beyond the Basics

pip supports many flags that control how packages are installed. Here are the most useful ones.

Install from requirements file:

pip install -r requirements.txt
pip install -r requirements.txt -r requirements-dev.txt

Install in editable/development mode (creates a symlink so changes are reflected immediately):

# Install current directory in editable mode
pip install -e .

# Install with optional extras in editable mode
pip install -e ".[dev,test]"

# Install a local package from another directory
pip install -e ../my-other-package/

Install to user directory (no sudo needed):

# Install to ~/.local (no admin rights needed)
pip install --user requests

# Packages go to:
# Linux: ~/.local/lib/python3.x/site-packages/
# macOS: ~/Library/Python/3.x/lib/python/site-packages/
# Windows: %APPDATA%\Python\Python3x\site-packages\

Force fresh download (skip cache):

# Skip the cache — useful in CI or to force re-download
pip install --no-cache-dir requests

# See where pip cache is stored
pip cache dir

# Clear all cached packages
pip cache purge

Install to a custom directory:

# Install to a specific directory
pip install --target ./vendor requests

Use a custom PyPI index:

# Use a custom PyPI mirror
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple requests

# Set default index permanently
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

Use a private index in addition to PyPI:

# Use a private index alongside PyPI
pip install --extra-index-url https://private.pypi.company.com/simple mypackage

Version Specifiers Explained

Understanding version specifiers is critical for managing dependencies correctly. Each specifier controls which versions pip is allowed to install.

For most projects, use ~= (compatible release) or pin exact versions with ==.

# Examples in requirements.txt:
requests==2.31.0        # Exact: always install 2.31.0
django~=4.2.0           # Compatible: >=4.2.0, <5.0.0
flask>=3.0,<4.0         # Range: any 3.x version
numpy>=1.24             # Minimum: 1.24 or higher
celery!=5.3.1           # Exclude: any version except 5.3.1

requirements.txt Best Practices

A well-maintained requirements file prevents dependency hell. Follow these practices to keep your projects stable.

Pin Exact Versions in Production

Always pin exact versions for production deployments. Loose version ranges lead to unpredictable behavior.

# BAD — loose versions lead to "works on my machine" problems
requests
flask>=2.0
sqlalchemy

# GOOD — exact pins for reproducible builds
requests==2.31.0
flask==3.0.0
sqlalchemy==2.0.23
werkzeug==3.0.1
jinja2==3.1.2

Separate Dev and Production Dependencies

Keep development tools out of production:

# requirements.txt (production)
flask==3.0.0
gunicorn==21.2.0
sqlalchemy==2.0.23
redis==5.0.1

# requirements-dev.txt (development only)
-r requirements.txt
pytest==7.4.3
pytest-cov==4.1.0
black==23.12.0
mypy==1.7.1
ruff==0.1.8

# Install for development:
pip install -r requirements-dev.txt

# Install for production (Docker, CI):
pip install -r requirements.txt

Use Constraints Files

Constraints files set version limits without adding packages as dependencies:

# constraints.txt — limits versions without adding as deps
urllib3<2.0
cryptography>=41.0.0
setuptools>=69.0

Use constraints when installing:

pip install -r requirements.txt -c constraints.txt

Add Comments for Context

Document why specific versions are pinned:

# requirements.txt with comments
requests==2.31.0          # HTTP client
flask==3.0.0              # Web framework
sqlalchemy==2.0.23        # ORM
celery==5.3.6             # Task queue
redis==5.0.1              # Celery broker backend
Pillow==10.1.0            # Pinned: 10.2.0 has regression in JPEG handling
cryptography==41.0.7      # Pinned: 42.x requires Rust 1.63+

pyproject.toml: Modern Python Packaging

pyproject.toml is the modern standard for Python project configuration (PEP 621). It replaces setup.py, setup.cfg, and can even serve as a replacement for requirements.txt in many workflows.

Basic project dependencies in pyproject.toml:

# pyproject.toml
[build-system]
requires = ["setuptools>=69.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-project"
version = "1.0.0"
description = "A sample Python project"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.31.0",
    "flask>=3.0.0,<4.0.0",
    "sqlalchemy~=2.0",
    "pydantic>=2.5.0",
]

Optional dependencies (dev, test, docs):

# pyproject.toml — optional dependency groups
[project.optional-dependencies]
dev = [
    "pytest>=7.4",
    "pytest-cov>=4.1",
    "black>=23.12",
    "ruff>=0.1.8",
    "mypy>=1.7",
]
docs = [
    "sphinx>=7.2",
    "sphinx-rtd-theme>=2.0",
]
test = [
    "pytest>=7.4",
    "pytest-asyncio>=0.23",
    "httpx>=0.25",  # for testing async HTTP
]

Install with optional dependencies:

# Install with optional dependencies
pip install ".[dev]"
pip install ".[dev,test,docs]"

# Editable install with extras
pip install -e ".[dev]"

Popular build backends:

setuptools — the classic, most widely used. hatchling — modern, fast, opinionated. flit-core — minimal, for simple packages. poetry-core — used by Poetry ecosystem.

# setuptools (classic)
[build-system]
requires = ["setuptools>=69.0", "wheel"]
build-backend = "setuptools.build_meta"

# hatchling (modern)
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# flit (minimal)
[build-system]
requires = ["flit_core>=3.4"]
build-backend = "flit_core.buildapi"

# poetry-core
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

pip-tools: Lock Your Dependencies

pip-tools bridges the gap between loose requirements and fully locked dependency files. It consists of two commands: pip-compile and pip-sync.

pip-compile: Generate Locked Requirements

Write your top-level dependencies in requirements.in and let pip-compile resolve the full dependency tree:

# Install pip-tools
pip install pip-tools

# Create requirements.in (your top-level deps)
# requirements.in
flask
requests
sqlalchemy
celery[redis]

# Compile to locked requirements.txt
pip-compile requirements.in

# Output: requirements.txt with ALL transitive deps pinned
# certifi==2023.11.17        # via requests
# charset-normalizer==3.3.2  # via requests
# click==8.1.7               # via flask
# flask==3.0.0               # via -r requirements.in
# ...

# Compile for a different Python version
pip-compile --python-version 3.11 requirements.in

# Upgrade all packages to latest compatible versions
pip-compile --upgrade requirements.in

# Upgrade a specific package
pip-compile --upgrade-package flask requirements.in

pip-sync: Match Your Environment Exactly

pip-sync ensures your virtual environment matches the lock file exactly — it installs missing packages and removes extras:

# Sync your environment to match requirements.txt exactly
pip-sync requirements.txt

# Sync with multiple files (e.g., prod + dev)
pip-sync requirements.txt requirements-dev.txt

# Dry run — see what would change
pip-sync --dry-run requirements.txt

Recommended workflow:

# Step 1: Write your direct dependencies in .in files
# requirements.in        — production deps
# requirements-dev.in    — dev deps (includes -r requirements.in)

# Step 2: Compile to locked .txt files
pip-compile requirements.in
pip-compile requirements-dev.in

# Step 3: Install with pip-sync
pip-sync requirements-dev.txt  # for development
pip-sync requirements.txt      # for production

# Step 4: When adding a new dependency
# Edit requirements.in, then re-run pip-compile

# Step 5: When upgrading
pip-compile --upgrade requirements.in
pip-sync requirements.txt

Common pip Errors and Fixes

Here are the most frequent pip errors and how to resolve them.

"No matching distribution found"

This error means the package does not exist for your Python version or platform:

# ERROR: No matching distribution found for some-package==1.0.0

# Fix 1: Check if the package name is correct
pip index versions some-package

# Fix 2: Check your Python version — some packages drop old Python support
python --version
pip install some-package  # without version pin

# Fix 3: Check if the package has wheels for your platform
pip install some-package --verbose

# Fix 4: The package may be on a different index
pip install some-package --extra-index-url https://other-index.com/simple

SSL Certificate Errors

SSL errors usually happen behind corporate proxies or on systems with outdated certificates:

# ERROR: Could not fetch URL: SSL: CERTIFICATE_VERIFY_FAILED

# Fix 1: Update certificates
pip install --upgrade certifi

# Fix 2: Trust the hosts (corporate proxy workaround)
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org requests

# Fix 3: Set permanently in pip config
pip config set global.trusted-host "pypi.org files.pythonhosted.org"

# Fix 4: Use a custom certificate bundle
pip install --cert /path/to/company-ca-bundle.crt requests

Permission Denied Errors

Never use sudo pip install. Use virtual environments or --user instead:

# ERROR: Permission denied (NEVER use sudo pip install!)

# Fix 1: Use a virtual environment (BEST solution)
python -m venv .venv
source .venv/bin/activate
pip install requests  # works without sudo

# Fix 2: Install to user directory
pip install --user requests

# Fix 3: On Debian/Ubuntu with "externally-managed-environment"
# ALWAYS use a venv — this error protects your system Python
python3 -m venv .venv && source .venv/bin/activate && pip install requests

Version Conflicts

When two packages need incompatible versions of the same dependency:

# ERROR: package-a requires dep>=2.0 but package-b requires dep<2.0

# Fix 1: Check what's conflicting
pip check

# Fix 2: See the dependency tree
pip install pipdeptree
pipdeptree

# Fix 3: Try to find compatible versions
pip install "package-a>=1.0" "package-b>=2.0" --dry-run

# Fix 4: Use pip-compile to resolve automatically
# Write both in requirements.in and let pip-compile find compatible versions
pip-compile requirements.in

uv: The Fast pip Replacement

uv is a Rust-based Python package installer and resolver. Built by the Astral team (makers of Ruff), it is a drop-in replacement for pip that is 10-100x faster.

Install uv:

# Install uv (standalone — no Python required)
# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Or with pip
pip install uv

# Or with Homebrew
brew install uv

Use uv as a pip replacement:

# Drop-in pip replacement — same commands, much faster
uv pip install requests
uv pip install -r requirements.txt
uv pip install -e ".[dev]"
uv pip uninstall requests
uv pip list
uv pip freeze
uv pip compile requirements.in -o requirements.txt

Create and manage virtual environments:

# Create a virtual environment with uv (much faster than python -m venv)
uv venv

# Create with a specific Python version
uv venv --python 3.12

# Activate (same as regular venv)
source .venv/bin/activate    # Linux/macOS
.venv\Scripts\activate       # Windows

# Full project workflow with uv
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt

Speed comparison (real-world project with 100+ dependencies):

uv is fully compatible with requirements.txt, pyproject.toml, and the existing Python packaging ecosystem. It is production-ready and a recommended upgrade from pip for new projects.

Frequently Asked Questions

Related Tools

• JSON Formatter — validate and format JSON output from Python scripts and APIs.
• Hash Generator — generate MD5, SHA-256, and other hashes for file integrity verification.