Boto3 Python Crash Course with AWS S3

• What is Boto3?
• Installation
• To create a new IAM user follow bellow steps
• Creating new bucket
• Upload file’s into s3 bucket
• Get all files from s3 bucket
• Get all prefix file’s from s3 bucket
• Deleting files from s3 bucket

What is Boto3?

Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3.

Installation:

Once you set up your environment then install Boto3 on your computer by using below command

pip install boto3

To create a new IAM user follow bellow steps

• Log into your AWS Management Console and select the IAM service.
• In the side navigation menu, select Access management | Users, and then select Add user.
• In the User name field, enter the name of the new user. In the Access type field, check the Programmatic access option to allow the user only programmatic access.
• Select Attach existing policies directly, and then select policy Name.
• Select Next: Tags  and then Select Next: Review.
• Download and save the credentials of the new user (Access key iD and Secret access key).

AWS Client Vs Resource

Client and Resource are two different abstractions within the boto3 SDK for making AWS service requests.

Creating new bucket:

You can create new Bucket name Programmatically also. Remember this bucket name must be unique throughout the whole AWS platform. If you try to create a bucket name, but that bucket name has already created then you will receive the following error: botocore.errorfactory.BucketAlreadyExists.

import boto3
def check_bucket_name():
    s3 = boto3.resource('s3',aws_access_key_id='',aws_secret_access_key='')
    data = s3.create_bucket(Bucket="botopy")
    if data:
        print("successfully bucket is created")
check_bucket_name()

Upload file’s into s3 bucket: You can upload different file format(txt, image, csv, xls, xlsx and json) into s3 bucket.

import boto3
def upload():
    s3 = boto3.resource('s3', aws_access_key_id="", aws_secret_access_key="")
    data = s3.Object('botopy', 'data/test.txt').put(Body=open('E:\projects\portfolio/aws\s3/test.txt', 'rb'))
    if data:
        print("files is stored into botopy bucket")
upload()

Get all files from s3 bucket:

You can get all files from specific folder. Here bototype is bucket name and prefix is subfolder’s.

import boto3
def get_files():
    s3 = boto3.resource('s3', aws_access_key_id="", aws_secret_access_key="")
    my_bucket = s3.Bucket('botopy')
    r = [obj.key for obj in my_bucket.objects.filter(Delimiter='/', Prefix='data/')]
    print("list of all files", r)
get_files()

Get all prefix file’s from s3 bucket:

You can get list of all files starting with given string/prefix.

import boto3, os
def get_files():
    s3 = boto3.resource('s3', aws_access_key_id="", aws_secret_access_key="")
    my_bucket = s3.Bucket('botopy')
    r = [obj.key for obj in my_bucket.objects.filter(Delimiter='/', Prefix='data/')]
    final_data = []
    for x in range(len(r)):
        d = os.path.basename(r[x])
        if d.startswith("test"):
            final_data.append(d)
    return final_data
print(get_files())

Deleting files from s3 bucket:

You can delete the files from particular s3 path.

import boto3
def delete_files():
    s3 = boto3.resource('s3', aws_access_key_id="", aws_secret_access_key="")
    data = s3.Object("botopy", 'data/test444.txt')
    data.delete()
delete_files()