Update documentation

Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
This commit is contained in:
Michel Hollands 2023-08-16 11:02:16 +01:00
parent 6d091d564e
commit d9c3b60659
2 changed files with 30 additions and 11 deletions

View File

@ -3,6 +3,8 @@
This is a meta-monitoring chart for GEL, GEM and GET. It should be installed in a This is a meta-monitoring chart for GEL, GEM and GET. It should be installed in a
separate namespace next to GEM, GEL or GET installations. separate namespace next to GEM, GEL or GET installations.
Note that this is pre-production software at the moment.
## Preparation ## Preparation
Create a values.yaml file based on the [default one](../charts/meta-monitoring/values.yaml). Create a values.yaml file based on the [default one](../charts/meta-monitoring/values.yaml).
@ -15,18 +17,18 @@ Create a values.yaml file based on the [default one](../charts/meta-monitoring/v
## Local and cloud modes ## Local and cloud modes
The chart has 2 modes: local and cloud. In the local mode logs, metrics and traces are sent The chart has 2 modes: local and cloud. In the local mode logs, metrics and/or traces are sent
to small Loki, Mimir and Tempo installations running in the meta-monitoring namespace. to small Loki, Mimir and Tempo installations running in the meta-monitoring namespace.
![local mode](docs/images/Meta%20monitoring%20local.png) ![local mode](docs/images/Meta%20monitoring%20local.png)
To enable local mode set `local.enabled` to true. To enable local mode set `local.<logs|metrics|traces>.enabled` to true.
In the cloud mode the logs, metrics and traces are sent to In the cloud mode the logs, metrics and/or traces are sent to Grafana Cloud.
![cloud mode](docs/images/Meta%20monitoring%20cloud.png) ![cloud mode](docs/images/Meta%20monitoring%20cloud.png)
To enable cloud mode set `cloud.enabled` to true. The `endpoint`, `username` and `password` settings for your Grafana Cloud logs, metrics and traces instances have to be filled in as well. To enable cloud mode set `cloud.<logs|metrics|traces>.enabled` to true. The `endpoint`, `username` and `password` settings for your Grafana Cloud logs, metrics and traces instances have to be filled in as well.
Both modes can be enabled at the same time. Both modes can be enabled at the same time.
@ -46,9 +48,22 @@ For more instructions including how to update the chart go to the [installation]
- Specify if logs, metrics or traces should be enabled for cloud or local - Specify if logs, metrics or traces should be enabled for cloud or local
- Specify the cluster name used for the logs, metrics and traces - Specify the cluster name used for the logs, metrics and traces
- Specify PII regexes that are applied to logs before they are sent to Loki (cloud or local). The capture group in the regex is replaced with *****. - Specify PII regexes that are applied to logs before they are sent to Loki (cloud or local). The capture group in the regex is replaced with *****.
- a Grafana instance is installed (when local mode is used) with the relevant datasources and dashboards installed. - a Grafana instance is installed (when local mode is used) with the relevant datasources installed. The following dashboards are installed:
- TBD: add dashboard for metrics and traces - logs dashboards
- TBD: update the logs dashboards to be the SSD ones - metrics dashboards
- traces dashboards
- agent dashboards
- Retention is set to 24 hours
Most of these features are enabled by default. See the values.yaml file for how to enable/disable them.
## Caveats
- The [loki.source.kubernetes](https://grafana.com/docs/agent/latest/flow/reference/components/loki.source.kubernetes/) component of the Grafana Agent is used to scrape Kubernetes log files. This component is marked experimental at the moment.
- This has not been tested on Openshift yet.
- The underlying Loki, Mimir and Tempo are at the default size installed by the Helm chart. This might need changing when monitoring bigger Loki, Mimir or Tempo installations.
- MinIO is used as storage at the moment with a limited retention. At the moment this chart cannot be used for monitoring over longer periods.
- Agent self monitoring is not done at the moment.
## Developer help topics ## Developer help topics

View File

@ -1,9 +1,12 @@
# Specify the namespaces to monitor here
namespacesToMonitor: namespacesToMonitor:
- loki - loki
- mimir - mimir
- tempo - tempo
clusterName: "meta-monitoring" # TODO check if this can be derived # The name of the cluster where this will be installed
clusterName: "meta-monitoring"
# Set to true for a local version of logs, metrics or traces
local: local:
logs: logs:
enabled: false enabled: false
@ -14,6 +17,7 @@ local:
minio: minio:
enabled: false # This should be set to true if any of the previous is enabled enabled: false # This should be set to true if any of the previous is enabled
# Set to true to write logs, metrics or traces to Grafana Cloud
cloud: cloud:
logs: logs:
enabled: true enabled: true
@ -31,7 +35,7 @@ cloud:
username: username:
password: password:
# Adding regexes here will add a stage.replace block. For more information see # Adding regexes here will add a stage.replace block for logs. For more information see
# https://grafana.com/docs/agent/latest/flow/reference/components/loki.process/#stagereplace-block # https://grafana.com/docs/agent/latest/flow/reference/components/loki.process/#stagereplace-block
logs: logs:
piiRegexes: piiRegexes:
@ -63,7 +67,7 @@ kubeStateMetrics:
endpoint: kube-state-metrics.kube-state-metrics.svc.cluster.local:8080 endpoint: kube-state-metrics.kube-state-metrics.svc.cluster.local:8080
# The following are configuration for the dependencies. # The following are configuration for the dependencies.
# These should not be changed. # These should usually not be changed.
loki: loki:
loki: loki: