Prerequisite Amazon S3 bucket setup

Before you use the Amazon S3 collector or forwarder functions, you must set up the required Amazon S3 bucket. You must also assign an IAM policy to the user who will access the bucket. The IAM policy provides the necessary permissions for the collector or forwarder to function properly.

The IAM policy must include the permissions shown in the code snippet:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowGetPutDelete",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject*",
                "s3:PutObject*",
                "s3:DeleteObject*",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": "arn:aws:s3:::<bucket-name>/*"
        },
        {
            "Sid": "AllowListingBucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": "arn:aws:s3:::<bucket-name>"
        }
    ]
}

The "Resource": "arn:aws:s3:::<bucket-name>/*" entry means that the actions in the statement apply to all objects in the S3 bucket identified by arn:aws:s3:::<bucket-name>. This allows access to every file and folder within the specified bucket.

If you want to grant access to only a specific folder (key) in the S3 bucket, you can specify the folder path after the bucket name. Use the following format in the Resource section:

"arn:aws:s3:::<bucket-name>/<directory-to-share>/*"

This restricts access to all objects within the specified folder. You can use this approach only for the AllowGetPutDelete statement.

Caution!
You must not use the AWS-managed policy AmazonS3FullAccess, because this policy allows all actions on all buckets and objects. This can create security risks by granting access that is broader than necessary.

For more information about setting up IAM policies, see the Amazon S3 Documentation.

To connect to your bucket, you need to provide your Access Key and Secret Access Key. For details on how to find these credentials, see https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds.html. You must also specify the folder from which you collect data or to which you send data.

For an example of how to use the Amazon S3 collector and forwarder in a stream, see https://infozone.atlassian.net/wiki/spaces/DAZ/pages/872480769.