AWS S3 Commands: The Complete CLI Guide

AWS S3 commands are the command-line instructions used with the AWS Command Line Interface (CLI) to manage objects and buckets in Amazon Simple Storage Service (S3). You enter these commands in a terminal after installing and configuring the AWS CLI, and they let you create buckets, upload and download files, sync directories, set permissions, and more. This guide explains the most useful S3 commands, how they work, and how to avoid common mistakes.

What Is the AWS CLI and Why Use It?

The AWS CLI is a unified tool that lets you interact with AWS services from your terminal. Instead of clicking through the S3 web console, you can run commands that are repeatable, scriptable, and easy to automate. The S3 commands all start with aws s3, followed by a subcommand like ls, cp, or sync. For example, aws s3 ls lists your buckets.

You can also use higher-level commands like aws s3 cp that handle large files and multi-part uploads automatically, making them more reliable than raw API calls. The CLI is available for Windows, macOS, and Linux, and it is the standard way to manage S3 from scripts and CI/CD pipelines.

Installing and Configuring the AWS CLI

Before running any S3 command, you need the AWS CLI installed and configured with your credentials. The installation method depends on your operating system:

• Windows: Download the MSI installer from the official AWS site, or use winget install Amazon.AWSCLI.
• macOS: Use Homebrew: brew install awscli.
• Linux: Use the bundled installer or your package manager, e.g., sudo apt install awscli on Debian/Ubuntu.

After installation, run aws configure. It will ask for your Access Key ID, Secret Access Key, default region (e.g., us-east-1), and output format (e.g., json). These keys are created in the AWS Management Console under IAM. Never share them or commit them to source control.

Core S3 Commands You Will Use Daily

These are the commands you will use most often. Each has many options, but these examples cover the basics.

• aws s3 ls – List your buckets. Add s3://bucket-name to list objects inside a bucket.
• aws s3 mb s3://my-bucket – Make a new bucket. Bucket names must be globally unique and follow DNS naming rules.
• aws s3 rb s3://my-bucket – Remove an empty bucket. Use --force to delete a non-empty bucket (use with caution).
• aws s3 cp file.txt s3://my-bucket/file.txt – Upload a file. Reverse the order to download.
• aws s3 mv file.txt s3://my-bucket/file.txt – Move a file (copy then delete).
• aws s3 rm s3://my-bucket/file.txt – Delete an object.
• aws s3 sync local-folder s3://my-bucket/folder – Sync a local directory with S3, uploading only changed files.

For example, to download a file from S3 to your current directory, run aws s3 cp s3://my-bucket/file.txt .. The dot means current directory.

How the Commands Work Under the Hood

When you run aws s3 cp, the CLI translates it into S3 API calls. For small files, it uses a single PUT or GET request. For files larger than 8 MB (or when you use --multipart-upload), it splits the file into parts and uploads them in parallel, then assembles them. This is why the CLI is more robust than a simple HTTP PUT for large files.

The sync command compares the last modified timestamps and sizes of local files with the objects in S3. It uploads only files that are new or changed, and optionally deletes remote files that no longer exist locally if you pass --delete. This makes it ideal for backups and static website deployments.

All commands support a --dryrun flag that shows what would happen without actually doing it. This is a safe way to test a command before running it for real.

Common Options and Flags

Every S3 command accepts options that modify its behavior. Here are the ones you will use most often:

• --recursive – Apply the command to all files under a directory or prefix. For example, aws s3 cp my-folder s3://bucket/my-folder --recursive uploads the entire folder.
• --exclude and --include – Filter files by pattern. For example, --exclude "*.tmp" skips temporary files.
• --acl – Set a canned access control list, like public-read or private.
• --storage-class – Set the storage class, e.g., STANDARD_IA or GLACIER.
• --sse – Enable server-side encryption, e.g., --sse AES256.
• --quiet – Suppress output, useful in scripts.

For instance, to upload only .jpg files from a folder and set them to public-read, you could run:

aws s3 cp my-folder s3://my-bucket/photos --recursive --exclude "*" --include "*.jpg" --acl public-read

Comparing s3api vs s3 Commands

The AWS CLI has two sets of S3 commands: aws s3 and aws s3api. The aws s3 commands are high-level and user-friendly; they handle recursion, multi-part uploads, and automatic retries. The aws s3api commands map directly to the S3 REST API and give you fine-grained control, but they require you to specify every parameter.

For example, to list objects with s3api, you would run aws s3api list-objects --bucket my-bucket. The output is JSON, which is great for scripting. To set a bucket policy, you must use aws s3api put-bucket-policy because the high-level aws s3 command does not have a direct equivalent. In general, use aws s3 for everyday file operations, and aws s3api for advanced configuration.

Common Mistakes and How to Avoid Them

Even experienced users make these errors:

• Forgetting --recursive – Running aws s3 cp my-folder s3://bucket without --recursive will fail with an error, because the CLI expects a file, not a directory.
• Using the wrong path format – S3 paths must start with s3://, and local paths must be absolute or relative to your current directory. Mixing them up causes “invalid path” errors.
• Not using --dryrun – A dryrun can prevent accidental mass deletions. Always test with --dryrun before running sync --delete.
• Ignoring region – If your bucket is in a different region than your CLI default, you may get a 301 redirect error. Specify the region with --region or set it in your config.
• Exposing credentials – Hard-coding keys in scripts is a security risk. Use IAM roles for EC2 instances, or environment variables.

Frequently Asked Questions

How do I list all files in an S3 bucket?

Run aws s3 ls s3://my-bucket --recursive. Without --recursive, it only shows the top-level prefixes. The output includes the file size and last modified date.

How do I upload a folder to S3?

Use aws s3 cp local-folder s3://my-bucket/folder --recursive. This uploads all files and subfolders. To only upload new or changed files, use aws s3 sync local-folder s3://my-bucket/folder.

How do I make an S3 bucket public?

You cannot make a bucket public with a single command; you must set a bucket policy. Use aws s3api put-bucket-policy --bucket my-bucket --policy file://policy.json, where policy.json contains a JSON policy granting s3:GetObject to "Principal": "*". Be careful: this makes all objects public.

What is the difference between cp and sync?

cp always copies the specified files, even if they already exist. sync only copies files that are new or have a different size/last-modified time. Use sync for backups and deployments to save time and bandwidth.

How do I delete all objects in a bucket?

Run aws s3 rm s3://my-bucket --recursive. This deletes every object. To delete the bucket itself, run aws s3 rb s3://my-bucket --force, which removes the bucket after deleting its contents.

Why do I get a “NoSuchBucket” error?

This usually means the bucket name is misspelled, or the bucket is in a different region than your CLI default. Check the name and add --region to your command. It can also mean the bucket was deleted.

Related

• Indian Ocean Dipole vs El Niño: Forecast Differences
• Jujutsu Infinite Weapons Tier List and Best Picks
• AI and the Cybersecurity Talent Gap: What to Know