Python pip、requirements.txt 与虚拟环境

管理 Python 依赖是每个开发者面临的第一个障碍。无论你是用 pip 安装包、在 requirements.txt 中锁定版本，还是用虚拟环境隔离项目，掌握正确的工作流程可以节省数小时的调试时间。本指南涵盖了关于 pip、requirements 文件、虚拟环境以及 pip-tools、pyproject.toml、uv 等现代工具的所有知识。

pip 基础：必备命令

pip 是 Python 的标准包安装工具。它连接到 Python 包索引（PyPI）并将包安装到你的 Python 环境中。以下是你每天都会用到的命令。

检查 pip 版本：

# Check pip version
pip --version
# pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)

# Check which Python pip is linked to
pip -V
python -m pip --version

安装、卸载和查看包：

# Install a package
pip install requests

# Install a specific version
pip install requests==2.31.0

# Uninstall a package
pip uninstall requests

# List all installed packages
pip list

# Show details about a specific package
pip show requests
# Name: requests
# Version: 2.31.0
# Location: /home/user/.venv/lib/python3.12/site-packages
# Requires: certifi, charset-normalizer, idna, urllib3

# List outdated packages
pip list --outdated

升级 pip 本身和包：

# Upgrade pip itself
python -m pip install --upgrade pip

# Upgrade a package to the latest version
pip install --upgrade requests

# Upgrade all outdated packages (bash one-liner)
pip list --outdated --format=columns | tail -n +3 | awk '{print $1}' | xargs pip install --upgrade

搜索和获取包信息：

# pip search is disabled on PyPI — use the website or:
pip index versions requests
# Available versions: 2.31.0, 2.30.0, 2.29.0, ...

# Show package info without installing
pip show requests

# Download without installing (useful for offline installs)
pip download requests -d ./packages/

requirements.txt：格式和版本锁定

requirements.txt 文件列出了项目所依赖的所有包。它是共享和复现 Python 环境的标准方式。

基本格式：

# requirements.txt

# Pinned versions (recommended for production)
requests==2.31.0
flask==3.0.0
sqlalchemy==2.0.23

# Minimum version
numpy>=1.24.0

# Compatible release (allows patches: >=1.4.0, <2.0.0)
django~=4.2.0

# Version range
celery>=5.3.0,<6.0.0

# Install from git repository
git+https://github.com/user/repo.git@main#egg=mypackage

# Install from local file
./libs/my-local-package/

# Include another requirements file
-r requirements-base.txt

从当前环境生成 requirements.txt：

# Generate from current environment (includes ALL packages)
pip freeze > requirements.txt

# The output looks like:
# certifi==2023.11.17
# charset-normalizer==3.3.2
# idna==3.6
# requests==2.31.0
# urllib3==2.1.0

# WARNING: pip freeze includes transitive dependencies too
# For cleaner results, use pip-tools (see below)

从 requirements.txt 安装所有依赖：

# Install all dependencies from requirements.txt
pip install -r requirements.txt

# Install from multiple files
pip install -r requirements.txt -r requirements-dev.txt

# Dry run — show what would be installed without installing
pip install -r requirements.txt --dry-run

虚拟环境：为什么需要它

虚拟环境是一个隔离的 Python 安装，用于将项目依赖分开管理。如果没有虚拟环境，为一个项目安装包可能会破坏另一个项目。虚拟环境解决了"在我机器上能跑"的问题。

为什么虚拟环境很重要

• 不同项目可以使用同一个包的不同版本。
• 无需管理员/root 权限即可安装包。
• 可以在任何机器上精确复现环境。
• 卸载项目只需删除 venv 文件夹。

创建和使用 venv

venv 模块内置于 Python 3.3+：

# Create a virtual environment named ".venv"
python -m venv .venv

# Or with a custom name
python3 -m venv myproject-env

# Create with access to system packages
python -m venv --system-site-packages .venv

# Create with a specific Python version (if multiple installed)
python3.12 -m venv .venv

激活虚拟环境：

# Linux / macOS (bash/zsh)
source .venv/bin/activate

# Windows (PowerShell)
.venv\Scripts\Activate.ps1

# Windows (cmd.exe)
.venv\Scripts\activate.bat

# Fish shell
source .venv/bin/activate.fish

# Your prompt changes to show the active venv:
# (.venv) user@host:~/project$

# Verify you are using the venv Python
which python    # Linux/macOS
where python    # Windows
# Should point to .venv/bin/python or .venv\Scripts\python.exe

完成后停用：

# Deactivate the virtual environment
deactivate

# Your prompt returns to normal:
# user@host:~/project$

始终将 venv 文件夹添加到 .gitignore：

# .gitignore
.venv/
venv/
env/
.env/

# Also ignore compiled Python files
__pycache__/
*.py[cod]
*.egg-info/

venv vs virtualenv vs conda：该用哪个？

Python 有多种创建虚拟环境的工具。以下是各自的适用场景。

# virtualenv — install and use
pip install virtualenv
virtualenv .venv
source .venv/bin/activate   # same activation as venv

# conda — create and manage environments
conda create -n myproject python=3.12
conda activate myproject
conda install numpy pandas
conda deactivate

# conda — export and reproduce environment
conda env export > environment.yml
conda env create -f environment.yml

pip install 选项：进阶用法

pip 支持许多控制包安装方式的标志。以下是最常用的选项。

从 requirements 文件安装：

pip install -r requirements.txt
pip install -r requirements.txt -r requirements-dev.txt

以可编辑/开发模式安装（创建符号链接，更改立即生效）：

# Install current directory in editable mode
pip install -e .

# Install with optional extras in editable mode
pip install -e ".[dev,test]"

# Install a local package from another directory
pip install -e ../my-other-package/

安装到用户目录（无需 sudo）：

# Install to ~/.local (no admin rights needed)
pip install --user requests

# Packages go to:
# Linux: ~/.local/lib/python3.x/site-packages/
# macOS: ~/Library/Python/3.x/lib/python/site-packages/
# Windows: %APPDATA%\Python\Python3x\site-packages\

强制重新下载（跳过缓存）：

# Skip the cache — useful in CI or to force re-download
pip install --no-cache-dir requests

# See where pip cache is stored
pip cache dir

# Clear all cached packages
pip cache purge

安装到自定义目录：

# Install to a specific directory
pip install --target ./vendor requests

使用自定义 PyPI 索引：

# Use a custom PyPI mirror
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple requests

# Set default index permanently
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

在 PyPI 之外使用私有索引：

# Use a private index alongside PyPI
pip install --extra-index-url https://private.pypi.company.com/simple mypackage

版本说明符详解

理解版本说明符对于正确管理依赖至关重要。每个说明符控制 pip 允许安装哪些版本。

对于大多数项目，使用 ~=（兼容版本）或用 == 锁定精确版本。

# Examples in requirements.txt:
requests==2.31.0        # Exact: always install 2.31.0
django~=4.2.0           # Compatible: >=4.2.0, <5.0.0
flask>=3.0,<4.0         # Range: any 3.x version
numpy>=1.24             # Minimum: 1.24 or higher
celery!=5.3.1           # Exclude: any version except 5.3.1

requirements.txt 最佳实践

良好维护的 requirements 文件可以防止依赖地狱。遵循这些实践来保持项目稳定。

在生产环境中锁定精确版本

生产部署时始终锁定精确版本。宽松的版本范围会导致不可预测的行为。

# BAD — loose versions lead to "works on my machine" problems
requests
flask>=2.0
sqlalchemy

# GOOD — exact pins for reproducible builds
requests==2.31.0
flask==3.0.0
sqlalchemy==2.0.23
werkzeug==3.0.1
jinja2==3.1.2

分离开发和生产依赖

将开发工具与生产环境分开：

# requirements.txt (production)
flask==3.0.0
gunicorn==21.2.0
sqlalchemy==2.0.23
redis==5.0.1

# requirements-dev.txt (development only)
-r requirements.txt
pytest==7.4.3
pytest-cov==4.1.0
black==23.12.0
mypy==1.7.1
ruff==0.1.8

# Install for development:
pip install -r requirements-dev.txt

# Install for production (Docker, CI):
pip install -r requirements.txt

使用约束文件

约束文件设定版本限制，但不将包添加为依赖：

# constraints.txt — limits versions without adding as deps
urllib3<2.0
cryptography>=41.0.0
setuptools>=69.0

安装时使用约束：

pip install -r requirements.txt -c constraints.txt

添加注释说明上下文

记录为什么锁定特定版本：

# requirements.txt with comments
requests==2.31.0          # HTTP client
flask==3.0.0              # Web framework
sqlalchemy==2.0.23        # ORM
celery==5.3.6             # Task queue
redis==5.0.1              # Celery broker backend
Pillow==10.1.0            # Pinned: 10.2.0 has regression in JPEG handling
cryptography==41.0.7      # Pinned: 42.x requires Rust 1.63+

pyproject.toml：现代 Python 打包

pyproject.toml 是 Python 项目配置的现代标准（PEP 621）。它取代了 setup.py、setup.cfg，在许多工作流中甚至可以替代 requirements.txt。

pyproject.toml 中的基本项目依赖：

# pyproject.toml
[build-system]
requires = ["setuptools>=69.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-project"
version = "1.0.0"
description = "A sample Python project"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.31.0",
    "flask>=3.0.0,<4.0.0",
    "sqlalchemy~=2.0",
    "pydantic>=2.5.0",
]

可选依赖（开发、测试、文档）：

# pyproject.toml — optional dependency groups
[project.optional-dependencies]
dev = [
    "pytest>=7.4",
    "pytest-cov>=4.1",
    "black>=23.12",
    "ruff>=0.1.8",
    "mypy>=1.7",
]
docs = [
    "sphinx>=7.2",
    "sphinx-rtd-theme>=2.0",
]
test = [
    "pytest>=7.4",
    "pytest-asyncio>=0.23",
    "httpx>=0.25",  # for testing async HTTP
]

安装可选依赖：

# Install with optional dependencies
pip install ".[dev]"
pip install ".[dev,test,docs]"

# Editable install with extras
pip install -e ".[dev]"

流行的构建后端：

setuptools — 经典且使用最广泛。hatchling — 现代、快速、有主见。flit-core — 极简，适合简单包。poetry-core — Poetry 生态系统使用。

# setuptools (classic)
[build-system]
requires = ["setuptools>=69.0", "wheel"]
build-backend = "setuptools.build_meta"

# hatchling (modern)
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

# flit (minimal)
[build-system]
requires = ["flit_core>=3.4"]
build-backend = "flit_core.buildapi"

# poetry-core
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

pip-tools：锁定你的依赖

pip-tools 弥合了宽松 requirements 和完全锁定依赖文件之间的差距。它由两个命令组成：pip-compile 和 pip-sync。

pip-compile：生成锁定的 Requirements

在 requirements.in 中写入顶层依赖，让 pip-compile 解析完整的依赖树：

# Install pip-tools
pip install pip-tools

# Create requirements.in (your top-level deps)
# requirements.in
flask
requests
sqlalchemy
celery[redis]

# Compile to locked requirements.txt
pip-compile requirements.in

# Output: requirements.txt with ALL transitive deps pinned
# certifi==2023.11.17        # via requests
# charset-normalizer==3.3.2  # via requests
# click==8.1.7               # via flask
# flask==3.0.0               # via -r requirements.in
# ...

# Compile for a different Python version
pip-compile --python-version 3.11 requirements.in

# Upgrade all packages to latest compatible versions
pip-compile --upgrade requirements.in

# Upgrade a specific package
pip-compile --upgrade-package flask requirements.in

pip-sync：精确匹配你的环境

pip-sync 确保你的虚拟环境与锁定文件完全匹配 — 它安装缺失的包并移除多余的包：

# Sync your environment to match requirements.txt exactly
pip-sync requirements.txt

# Sync with multiple files (e.g., prod + dev)
pip-sync requirements.txt requirements-dev.txt

# Dry run — see what would change
pip-sync --dry-run requirements.txt

推荐的工作流程：

# Step 1: Write your direct dependencies in .in files
# requirements.in        — production deps
# requirements-dev.in    — dev deps (includes -r requirements.in)

# Step 2: Compile to locked .txt files
pip-compile requirements.in
pip-compile requirements-dev.in

# Step 3: Install with pip-sync
pip-sync requirements-dev.txt  # for development
pip-sync requirements.txt      # for production

# Step 4: When adding a new dependency
# Edit requirements.in, then re-run pip-compile

# Step 5: When upgrading
pip-compile --upgrade requirements.in
pip-sync requirements.txt

常见 pip 错误及修复

以下是最常见的 pip 错误以及如何解决它们。

"No matching distribution found"

此错误表示该包不存在于你的 Python 版本或平台上：

# ERROR: No matching distribution found for some-package==1.0.0

# Fix 1: Check if the package name is correct
pip index versions some-package

# Fix 2: Check your Python version — some packages drop old Python support
python --version
pip install some-package  # without version pin

# Fix 3: Check if the package has wheels for your platform
pip install some-package --verbose

# Fix 4: The package may be on a different index
pip install some-package --extra-index-url https://other-index.com/simple

SSL 证书错误

SSL 错误通常发生在企业代理后面或证书过期的系统上：

# ERROR: Could not fetch URL: SSL: CERTIFICATE_VERIFY_FAILED

# Fix 1: Update certificates
pip install --upgrade certifi

# Fix 2: Trust the hosts (corporate proxy workaround)
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org requests

# Fix 3: Set permanently in pip config
pip config set global.trusted-host "pypi.org files.pythonhosted.org"

# Fix 4: Use a custom certificate bundle
pip install --cert /path/to/company-ca-bundle.crt requests

权限被拒绝错误

永远不要使用 sudo pip install。请使用虚拟环境或 --user：

# ERROR: Permission denied (NEVER use sudo pip install!)

# Fix 1: Use a virtual environment (BEST solution)
python -m venv .venv
source .venv/bin/activate
pip install requests  # works without sudo

# Fix 2: Install to user directory
pip install --user requests

# Fix 3: On Debian/Ubuntu with "externally-managed-environment"
# ALWAYS use a venv — this error protects your system Python
python3 -m venv .venv && source .venv/bin/activate && pip install requests

版本冲突

当两个包需要同一依赖的不兼容版本时：

# ERROR: package-a requires dep>=2.0 but package-b requires dep<2.0

# Fix 1: Check what's conflicting
pip check

# Fix 2: See the dependency tree
pip install pipdeptree
pipdeptree

# Fix 3: Try to find compatible versions
pip install "package-a>=1.0" "package-b>=2.0" --dry-run

# Fix 4: Use pip-compile to resolve automatically
# Write both in requirements.in and let pip-compile find compatible versions
pip-compile requirements.in

uv：极速 pip 替代品

uv 是基于 Rust 的 Python 包安装器和解析器。由 Astral 团队（Ruff 的创造者）打造，它是 pip 的直接替代品，速度快 10-100 倍。

安装 uv：

# Install uv (standalone — no Python required)
# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Or with pip
pip install uv

# Or with Homebrew
brew install uv

用 uv 替代 pip：

# Drop-in pip replacement — same commands, much faster
uv pip install requests
uv pip install -r requirements.txt
uv pip install -e ".[dev]"
uv pip uninstall requests
uv pip list
uv pip freeze
uv pip compile requirements.in -o requirements.txt

创建和管理虚拟环境：

# Create a virtual environment with uv (much faster than python -m venv)
uv venv

# Create with a specific Python version
uv venv --python 3.12

# Activate (same as regular venv)
source .venv/bin/activate    # Linux/macOS
.venv\Scripts\activate       # Windows

# Full project workflow with uv
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt

速度对比（包含 100+ 依赖的真实项目）：

uv 完全兼容 requirements.txt、pyproject.toml 和现有的 Python 打包生态系统。它已经可以用于生产环境，推荐新项目使用 uv 替代 pip。

常见问题

相关工具

• JSON 格式化工具 — 验证和格式化 Python 脚本和 API 的 JSON 输出。
• 哈希生成器 — 生成 MD5、SHA-256 和其他哈希值，用于文件完整性验证。