Log Forwarding
- Kevin Wu
You can send MediationZone log files to an external log collector by configuring the properties described in this section. All log files generated in MediationZone will be forwarded when log forwarding is enabled, including System, Platform, and EC logs.
The log structure, fields, and severity follow the JSON and RFC5424 specifications and it is a prerequisite to know one of these specifications to use the log forwarding functionality.
The following properties should be added to the platform configuration file to forward the logs to the external log collector:
Properties | Description | Default Value |
|---|---|---|
| This property is used to enable or disable the logging. | false |
| This property defines the protocol to use, either UDP or TCP. | UDP |
| This property defines the hostname or IP address of the target log collector. | localhost |
| This property defines the port of the target log collector. | 514 |
| This property defines the layout type based on the supported specifications. You can select either RFC5424 or JSON. | RFC5424 |
| This property is used to include a full stack trace in your log messages. | false |
| This property is used to generate a debug log file, useful when you need to determine any issues when configuring the log forwarding. | false |
| This property defines the name of the debug log file. | syslog-debug.log |
| This property defines the location of the debug log file. | /tmp/syslog |
| This property defines the number of debug log file to be generated. | 5 |
Platform Configuration
To enable syslog logging to connect to external log collector.
mzsh topo set topo://container:main1/pico:platform/val:config.properties.mz.syslog.logging.enable trueTo set external log collector connection port. For instance, port 5140 is connection port for Fluent-bit
mzsh topo set topo://container:main1/pico:platform/val:config.properties.mz.syslog.port 5140The rest of the properties that are not set explicitly will follow their default value in the tables.
Changes made to the properties above only take effect after platform restarted.
mzsh restart platformLog Forwarding With Fluent-bit:
Fluent-bit is one of the external log collector that is widely used by major cloud providers, banks, and companies world-wide. Fluent-bit can read from local files and network devices, and can scrape metrics in the Prometheus format from your server. Furthermore, Fluent Bit supports a vendor-neutral approach, seamlessly integrating with other ecosystems such as Prometheus and OpenTelemetry.
Click here to install Fluent-bit for your environment if you have not.
Fluent-bit Configuration File
Fluent-bit configuration file allows user to configure what kind of data source (input) to read from and where to forward the data to, i.e., data destination (output).
Edit Fluent-bit configuration file with the following contents.
then startup Fluent-bit with configuration file manually
on Linux environment with Systemd, startup Fluent-bit with systemctl
then do a status check, you should see a similar output like this:
Fluent-bit started up manually with these output on the screen
Fluent-bit Outputs
Fluent-bit can be configured to route collected data to multiples destinations. The fluent-bit configuration file listed in previous section routes collected data to standard output, local file and remote rsyslog server.
Refer to official documentation to know more about output plugins supported by Fluent-bit.