Amazon S3 是数百万应用文件存储的骨干。本指南涵盖 S3 SDK、预签名 URL、存储桶策略和 CloudFront 集成。
S3 基础：存储桶与对象
S3 将数据作为对象存储在存储桶中。每个对象由文件数据加元数据组成，通过键（本质上是文件路径）唯一标识。
S3 Core Concepts:
Bucket — Top-level container (globally unique name, tied to a region)
Object — A file stored in a bucket (up to 5TB per object)
Key — The object's path/name: "uploads/2026/user-123/photo.jpg"
Prefix — Virtual folder: "uploads/2026/" (S3 has no real folders)
Region — Where the bucket lives: us-east-1, eu-west-1, ap-southeast-1
Storage Classes:
Standard — Frequently accessed data, 99.99% availability
Standard-IA — Infrequent access, lower cost, retrieval fee
One Zone-IA — Lower cost, single AZ (no cross-AZ replication)
Intelligent-Tiering — Auto-move between tiers based on access patterns
Glacier Instant — Archive with millisecond retrieval
Glacier Flexible — Archive with minutes-to-hours retrieval
Glacier Deep Archive — Cheapest storage, 12-48h retrieval使用 AWS SDK v3 上传文件
AWS SDK v3 采用模块化设计，只导入需要的命令，显著减少包大小。
// AWS SDK v3 — File Upload to S3
import {
S3Client,
PutObjectCommand,
GetObjectCommand,
DeleteObjectCommand,
ListObjectsV2Command,
} from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage'; // For multipart upload
import { fromEnv } from '@aws-sdk/credential-providers';
const s3 = new S3Client({
region: process.env.AWS_REGION || 'us-east-1',
credentials: fromEnv(), // Reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
});
const BUCKET = process.env.S3_BUCKET_NAME!;
// 1. Simple upload (files < 5MB)
async function uploadFile(key: string, file: Buffer, contentType: string) {
const command = new PutObjectCommand({
Bucket: BUCKET,
Key: key, // e.g., 'uploads/2026/user-123/avatar.jpg'
Body: file,
ContentType: contentType,
// Optional: set cache headers
CacheControl: 'max-age=31536000',
// Optional: make publicly readable
// ACL: 'public-read',
// Optional: custom metadata
Metadata: {
'uploaded-by': 'server',
'original-name': 'avatar.jpg',
},
});
const result = await s3.send(command);
return {
url: `https://${BUCKET}.s3.amazonaws.com/${key}`,
etag: result.ETag,
versionId: result.VersionId,
};
}
// 2. Multipart upload for large files (recommended for > 100MB)
async function uploadLargeFile(key: string, stream: NodeJS.ReadableStream, contentType: string) {
const upload = new Upload({
client: s3,
params: {
Bucket: BUCKET,
Key: key,
Body: stream,
ContentType: contentType,
},
partSize: 10 * 1024 * 1024, // 10 MB parts
queueSize: 4, // 4 concurrent uploads
});
upload.on('httpUploadProgress', (progress) => {
console.log(`Uploaded: ${progress.loaded}/${progress.total} bytes`);
});
return upload.done();
}预签名 URL 实现安全直传
预签名 URL 允许客户端直接上传到 S3，无需通过服务器中转，大幅降低文件上传的服务器负载。URL 有时间限制并用 AWS 凭证签名。
// Presigned URLs — Direct Browser-to-S3 Upload
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { PutObjectCommand, GetObjectCommand } from '@aws-sdk/client-s3';
import { randomUUID } from 'crypto';
// Generate presigned upload URL (browser uploads directly to S3)
async function generateUploadUrl(
fileName: string,
fileType: string,
userId: string
) {
const key = `uploads/${userId}/${randomUUID()}-${fileName}`;
const command = new PutObjectCommand({
Bucket: BUCKET,
Key: key,
ContentType: fileType,
// Optional: limit file size with Content-Length condition
// This must be enforced server-side with a policy
});
const signedUrl = await getSignedUrl(s3, command, {
expiresIn: 15 * 60, // 15 minutes
});
return {
uploadUrl: signedUrl,
key, // Return key so client can reference the uploaded file
expiresIn: 900, // seconds
};
}
// Express.js endpoint
app.post('/api/upload-url', authenticate, async (req, res) => {
const { fileName, fileType, fileSize } = req.body;
// Validate file type and size on server
const allowedTypes = ['image/jpeg', 'image/png', 'image/webp', 'application/pdf'];
if (!allowedTypes.includes(fileType)) {
return res.status(400).json({ error: 'File type not allowed' });
}
if (fileSize > 10 * 1024 * 1024) { // 10 MB limit
return res.status(400).json({ error: 'File too large' });
}
const { uploadUrl, key } = await generateUploadUrl(fileName, fileType, req.user.id);
res.json({ uploadUrl, key });
});
// Client-side: use the presigned URL to upload
async function uploadToS3(file, presignedUrl) {
const response = await fetch(presignedUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': file.type,
},
});
if (!response.ok) throw new Error('Upload failed');
return response;
}CloudFront CDN 集成
CloudFront 是 AWS 的 CDN，在全球边缘节点缓存 S3 内容，降低延迟和数据传输成本。
// CloudFront + S3 Setup
// 1. Bucket policy to allow CloudFront (Origin Access Control)
const bucketPolicy = {
Version: '2012-10-17',
Statement: [
{
Sid: 'AllowCloudFrontServicePrincipal',
Effect: 'Allow',
Principal: {
Service: 'cloudfront.amazonaws.com',
},
Action: 's3:GetObject',
Resource: `arn:aws:s3:::my-bucket/*`,
Condition: {
StringEquals: {
'AWS:SourceArn': 'arn:aws:cloudfront::123456789:distribution/ABCDEF123456',
},
},
},
],
};
// 2. Generate signed URLs for private CloudFront content
import { getSignedUrl } from '@aws-sdk/cloudfront-signer';
function generateCloudFrontSignedUrl(key: string, expirySeconds = 3600) {
const url = `https://${process.env.CLOUDFRONT_DOMAIN}/${key}`;
const expiryDate = new Date();
expiryDate.setSeconds(expiryDate.getSeconds() + expirySeconds);
return getSignedUrl({
url,
keyPairId: process.env.CLOUDFRONT_KEY_PAIR_ID!,
privateKey: process.env.CLOUDFRONT_PRIVATE_KEY!,
dateLessThan: expiryDate.toISOString(),
});
}
// 3. Invalidate CloudFront cache when S3 objects change
import { CloudFrontClient, CreateInvalidationCommand } from '@aws-sdk/client-cloudfront';
const cloudfront = new CloudFrontClient({ region: 'us-east-1' });
async function invalidateCache(paths: string[]) {
const command = new CreateInvalidationCommand({
DistributionId: process.env.CLOUDFRONT_DISTRIBUTION_ID!,
InvalidationBatch: {
CallerReference: Date.now().toString(),
Paths: {
Quantity: paths.length,
Items: paths.map(p => `/${p}`),
},
},
});
return cloudfront.send(command);
}
// Usage: invalidate a specific file after update
await invalidateCache(['uploads/profile-pictures/user-123.jpg']);
// Wildcard: invalidate all files in a folder
await invalidateCache(['uploads/*']);常见问题
应该允许公开访问 S3 存储桶吗？
仅对静态网站托管或真正公开的资产开放。用户上传、敏感文件等应保持存储桶私有，使用预签名 URL 提供访问。
预签名 URL 应该有效多长时间？
上传 URL（PUT）通常为 15-30 分钟。最大过期时间为 7 天（604800 秒）。
如何降低 S3 存储成本？
使用 S3 智能分层（Intelligent-Tiering）和生命周期规则自动转换或删除旧对象。
什么是分段上传（Multipart Upload）？
分段上传将大文件拆分为并行上传的部分，然后由 S3 合并。AWS 建议文件超过 100MB 使用分段上传，5GB 以上必须使用。