Input plugins define the source from which Fluent Bit collects logs and processes the logs to give them structure through a parser. In Konvoy, the tail plugin is configured to read each container log at /var/log/containers*.log and by default, the tail plugin is configured to use the CRI parser.
When the REGEX and JSON patterns defined in the built-in parsers are not a good fit to capture the events logged and it is not possible to change the application’s logging structure, the operator could define a custom parser.
In Konvoy, it is possible to define custom parsers and input plugins by defining the following stanza under the fluentbit kubeaddons block:
- name: fluentbit
enabled: true
values: |
config:
inputs: |
[INPUT]
Name tail
Alias kubernetes_cluster
Path /var/log/containers/*.log
Parser my_custom_parser
DB /tail-db/kube.db
Tag kube.*
Refresh_Interval 60
Rotate_Wait 5
Mem_Buf_Limit 5MB
Skip_Long_Lines On
customParsers: |
[PARSER]
Name docker_no_time
Format json
Time_Keep Off
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
[PARSER]
Name my_custom_parser
Format regex
Regex ^(?
In the above example, a custom parser named “my_custom_parser” is defined and then referred to in the tail input plugin stanza. Once the parser and the input plugin are defined in the cluster.yaml, the operator should execute:
./konvoy deploy addons
To verify that the changes were applied, the content of the configmap fluentbit-kubeaddons-fluent-bit can be inspected with the following command:
kubectl get cm fluentbit-kubeaddons-fluent-bit -n kubeaddons -ojsonpath='{.data.fluent-bit\.conf}'
kubectl get cm fluentbit-kubeaddons-fluent-bit -n kubeaddons -ojsonpath='{.data.custom_parsers\.conf}'