Get ACLs for All S3 Buckets

I have a scheduled task to review the access control list (ACL) with each of the S3 buckets in my account. At first, I had a few buckets and doing the task manually was not an issue. However, as the number of buckets grow, it has become more and more time consuming to perform by hand.

So, I decided to develop a script. The script is using bash syntax and the AWS CLI. My first attempt looked like this:

#!/bin/bash

for bucket in $(aws s3api list-buckets  --output text --query 'Buckets[*].Name'); do
    echo Getting ACL for $bucket
    aws s3api get-bucket-acl --bucket $bucket
done

The for loop includes the CLI command to list the buckets and all I want is the bucket name. We loop thru each of these and call the get-bucket-acl command. This worked for many buckets but some displayed the following error:

A client error (InvalidRequest) occurred when calling the GetBucketAcl operation: You are attempting to operate on a bucket in a region that requires Signature Version 4.  You can fix this issue by explicitly providing the correct region location using the --region argument, the AWS_DEFAULT_REGION environment variable, or the region variable in the AWS CLI configuration file.  You can get the bucket's location by running "aws s3api get-bucket-location --bucket BUCKET".

So, I modified the script as suggested in the error message:

#!/bin/bash


for bucket in $(aws s3api list-buckets  --output text --query 'Buckets[*].Name'); do
    echo Getting ACL for $bucket
    region=`aws s3api get-bucket-location --bucket $bucket --output text`
    aws s3api get-bucket-acl --region=$region --bucket $bucket
done

This improved but I got a couple of invalid region references. For some of the buckets the region returned was “None”. Looking at the CLI reference, due to legacy related items, this occurs when the region is us-east-1. Also, for two older buckets the region returned “EU”. I did not find any reasons for this but I know that the buckets which return this are in eu-west-1.

So my final script is as follows:

#!/bin/bash

for bucket in $(aws s3api list-buckets  --output text --query 'Buckets[*].Name'); do
    echo Getting ACL for $bucket
    region=`aws s3api get-bucket-location --bucket $bucket --output text`
    if [ $region == 'None' ]; then
	region=us-east-1
    fi
    if [ $region == 'EU' ]; then
	region=eu-west-1
    fi
    aws s3api get-bucket-acl --region=$region --bucket $bucket
done