Fluentbit is shipped (along with kibana and elasticsearch) as one of the Konvoy logging stack components. By default, the “es” fluentbit’s output plugin is set to send logs to the local Elasticsearch service. However, the “es” output plugin can be configured to use a remote elasticsearch backend by modifying the default “es” plugin output configuration and removing the dependency on the elasticsearch addon.
The elasticsearch addon should be disabled in the cluster.yaml:
- name: elasticsearch
enabled: false
- name: elasticsearch-curator
enabled: false
- name: elasticsearchexporter
enabled: false
Then fluentbit’s output configuration should be specified as follows in the file cluster.yaml:
- name: fluentbit
enabled: true
values: |
config:
outputs: |
[OUTPUT]
Name es
Match *
Host
Port
HTTP_User
HTTP_Passwd
tls On
Time_Key @ts
Logstash_Format On
Logstash_Prefix kubernetes_cluster
Retry_Limit 10
Buffer_Size 512KB
For more details on the "es" output plugin configuration parameters, please refer to this link.
As elasticsearch is configured as a dependency, the fluentbit addon must be patched to eliminate that requirement. To remove the requirement, please execute:
kubectl patch addon.kubeaddons.mesosphere.io/fluentbit -n kubeaddons --patch='{"spec":{"requires":null}}' --type merge
Without removing the requirement, konvoy won’t be able to deploy the fluentbit addon:
kubectl get addons fluentbit -n kubeaddons
fluentbit false waitingOnDependency 1.5.7-2
kubectl describe addons fluentbit -n kubeaddons
Name: fluentbit
Namespace: kubeaddons
Labels: kubeaddons.mesosphere.io/name=fluentbit
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning MissingDeps 3m35s (x79 over 18m) kubeaddons-controller no addons found matching required dependency label: {MatchLabels:map[kubeaddons.mesosphere.io/name:elasticsearch] MatchExpressions:[]}