Page Comparison

The Amazon S3 Function allows data to be sent or received from/to Amazon S3 buckets. To use this function, it is necessary to have prepared an Amazon S3 bucket in advance. 

For information on how to do this, see https://docs.aws.amazon.com/s3/.

To connect to your bucket, you need to have your Access Key and Secret Access Key at hand. For information on how to find this information, see https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html. You also need to enter the folder to which you are collecting data or sending data.

...

1. In Amazon Credentials, specify the Access Key and the Secret Key. This information is available on your Amazon S3 account. You also have the option to use the Secrets Wallet option to enter the S3 account credentials. 

2. In File Location, specify the Bucket and the path to the folder in Folder. The folder path cannot begin or end with a ' / '.

   Note
   title Note!
   If the path to the folder is not specified, the root folder of the S3 bucket is selected by default.

   
   

3. In Output file Information, specify how you want to handle the output file(s) in Filename options. You can select from the following options:

   Filename option	Action
   Collector filename	Select Collector filename if you want to keep the same filename as your input file(s). If a collector does not have a filename, for example, Counter, the system generates a filename based on the function.
   Custom filename	Select Custom filename to define a new filename for all the output files. If you require more flexibility in defining file names refer to Configuring Dynamic Naming in Fields.

   
   
   

4. Select Append timestamp to append the timestamp to the name of the output file. For example, the output filename for a CSV file will look like <myfile>_<timestamp>.csv.

   Note
   title Note!
   If Append timestamp is not selected, the existing file at the destination can be overwritten by the output file.

   
   

5. In File format, select the format of the output file(s) from the following options:

   File format	Description
   CSV	Select to send the output file in CSV format. Select Include table header to include the table header in the output file(s). Specify a Delimiter for the CSV file format. The default value is ' , '. Compression can be toggled by enabling the Compress file option. A dropdown menu is used to select the format – Zip and GZIP are supported.  An additional option toggles the Bucket owner full control.
   Excel	Select to send the output file in Excel format. You can also specify the Sheet name. The default sheet name is Sheet 1. Compression can be toggled by enabling the Compress file option. A dropdown menu is used to select the format – Zip and GZIP are supported.  An additional option toggles the Bucket owner full control.
   Buffer	Select to send the output file in the Buffer format.  If you are reading or processing files containing binary data (Buffer format), for example for performance or other reasons, you can write these files through the AWS S3 Forwarder. Compression can be toggled by enabling the Compress file option. A dropdown menu is used to select the format – Zip and GZIP are supported.  An additional option toggles the Bucket owner full control.
   JSON	Select to send the output file in JSON format. Select the preferred output format, Action on records: one file with All in one array, one file with All in one array with key, or One file per record. Note title Note! If One file per record is selected, you must also select Append timestamp, otherwise, the files will be overwritten. Compression can be toggled by enabling the Compress file option. A dropdown menu is used to select the format – Zip and GZIP are supported.  An additional option toggles the Bucket owner full control.  JSON files can be formatted for easier reading by selecting the  Output file in pretty print option.

   
   

6. To output JSON files in a more compact form, deselect the Output file in pretty print checkbox. By default, pretty print is on.

   Note
   title Note!
   Selecting pretty print increases the size of the output file.

   
   

7. To compress the output file, select the Compress file checkbox and specify the Compression format. The supported formats are:
   • Zip
   • GZip
8. You have the option to provide the S3 bucket owners full access to the object that are written by other S3 account holders. Select the checkbox, Bucket owner full control, to give this permission. To be able to do this Usage Engine uses an option called as Access Control List (ACL) and is enabled as ACL = bucket-owner-full-control.

...