Welcome back! In the previous lesson, you learned how to securely access AWS services from an EC2 instance using IAM roles and temporary credentials. Now, we are moving on to one of the most popular AWS services: Amazon S3.

Amazon S3 (Simple Storage Service) is a service that lets you store and retrieve any amount of data, at any time, from anywhere on the web. It is widely used for storing files, backups, images, and much more.

In this lesson, you will learn how to:

• Make sure an S3 bucket exists (and create one if it doesn't)
• Upload a file to S3
• List files in your S3 bucket
• Download a file from S3

By the end of this lesson, you will be able to interact with S3 using Python code, which is a key skill for many cloud development tasks.

Before we start working with S3, let's quickly remind ourselves how to connect to AWS services using the boto3 library in Python. You learned about this in the previous lesson, but here's a short reminder.

First, make sure you have boto3 installed. If you're working on your own machine, you can install it using pip:

On CodeSignal, boto3 is usually pre-installed for you, so you can skip this step.

To use boto3, you need to have AWS credentials set up. On CodeSignal, these are usually pre-configured for you. On your own machine, you would set up credentials using the AWS CLI or environment variables.

To connect to S3, you create a client like this:

Here, boto3.client('s3') creates a client object that lets you interact with S3. You will use this s3 object for all your S3 operations in this lesson.

Before you can upload or download files, you need a bucket. A bucket is like a folder in S3 where your files (called "objects") are stored. Bucket names must be globally unique across all AWS users.

Let's see how to check if a bucket exists and create it if it doesn't.

First, you need to decide on a bucket name. In the example code, we use a unique name with a random part to avoid conflicts:

• os.environ.get("BUCKET") checks if there is a BUCKET environment variable set.
• If not, it creates a new name like aws-s3-practice-bucket-1a2b3c4d using a random string.

Now, let's check if the bucket exists and create it if needed:

Explanation:

• s3.head_bucket(Bucket=bucket) checks if the bucket exists and you have access.
• If the bucket does not exist (404), it creates the bucket in your region.

Now that you have a bucket, let's upload a file to it. First, you need a file to upload. Let's create a simple text file and then upload it.

Explanation:

• with open(LOCAL_UPLOAD, "w") as f: creates a new file called sample.txt and opens it for writing.
• f.write("Hello from boto3 S3!\n") writes a simple message to the file.
• s3.upload_file(LOCAL_UPLOAD, bucket, KEY) uploads the file to your S3 bucket under the key (path) examples/sample.txt.
• The KEY is like the file path inside your bucket.

Example output:

After uploading, you might want to see what files are in your bucket. You can list objects in a specific folder (called a "prefix" in S3).

Explanation:

• s3.list_objects_v2(Bucket=bucket, Prefix="examples/") lists all objects in the examples/ folder of your bucket.
• The response contains a list of objects under the "Contents" key.
• For each object, we print its key (file path) and size in bytes.

Example output:

This tells you that there is a file called examples/sample.txt and its size is 20 bytes.

Finally, let's download the file you uploaded back to your local machine. We'll save it in a folder called downloads.

Explanation:

• os.makedirs(DOWNLOAD_DIR, exist_ok=True) creates the downloads directory if it doesn't already exist.
• out = os.path.join(DOWNLOAD_DIR, "downloaded.txt") sets the path for the downloaded file.
• s3.download_file(bucket, KEY, out) downloads the file from S3 to your local directory.

Example output:

In this lesson, you learned how to:

• Make sure an S3 bucket exists (and create one if needed)
• Upload a file to S3
• List files in your S3 bucket
• Download a file from S3

These are the basic building blocks for working with files in AWS S3 using Python. Up next, you'll get to practice these steps yourself in hands-on exercises. This will help you get comfortable with S3 operations and prepare you for more advanced AWS development tasks. Good luck!