Public/grafana-meta-monitoring-chart
2
0
Fork 0
forked from RemoteSync/grafana-meta-monitoring-chart
Code Pull Requests Activity

Compare commits

merge into: Public:add_ci_install
Branches Tags
Public:main Public:modified Public:chore/update-dependencies Public:chore/update-minio Public:slim-bean-patch-1 Public:update-metrics-port Public:add_loki_team_to_prs Public:test_branch Public:update_version Public:update_main_page Public:cleanup_ci Public:add_ci_install Public:add_ksm_docs Public:fix_cluster_name Public:fix_dashboards Public:fix_loki_write_endpoint Public:fix_traces_forwarding Public:change_default_sampling_type Public:update_ingress_documentation Public:create_new_version Public:add_more_metrics Public:fix_rules Public:remove_rules_files_mimir_tempo Public:split_up_grafana_template Public:remove_redundant_variables Public:fix_dependency_check Public:chore/update-tempo-distributed Public:fix_alloy_config_for_traces Public:fix_minio_secret_name Public:combine_ci Public:update_grafana Public:chore/update-loki Public:use_secret_for_minio Public:set_step_time_to_1_minute_in_dashboards Public:add_extra_metrics Public:more_dashboard_cleanup Public:remove_non_loki_dashboards Public:open_extra_ports_for_otlp Public:get_logs_from_all_namespaces Public:fix_correct_version Public:fix_logic_for_secrets Public:use_updated_dashboards2 Public:filter_cadvisor_kubelet_metrics_on_namespace Public:add_creation_of_dashboard Public:more_docs_fixes Public:add_more_docs Public:update_installation_instructions Public:fix_release_flow Public:add_ci_to_release_helm_chart Public:remove_reviewers Public:chore/update-grafana-alloy Public:chore/update-mimir-distributed Public:fix_action Public:add_helm_chart_updater Public:add_agent_metrics Public:filter_out_errors_local_loki Public:filter_log_lines Public:filter_only_important_log_lines Public:remove_helm_test_step Public:add_create_kind_cluster_to_ci Public:add_lint_to_ci Public:add_ci Public:add_skip_cdrs_to_installation_step Public:add_retention Public:fix_tempo_dashboards Public:fix_mimir_dashboards Public:add_agent_dashboards Public:add_ruler Public:add_tempo_dashboards.yaml Public:add_scraping_of_cadvisor Public:add_mimir_dashboards Public:use_loki_ssd_dashboards Public:add_kube_state_metrics_scraping Public:add_regexes_to_filter_log_lines Public:add_validation Public:enable_dbs_individually Public:add_minio
Public:v1.3.0 Public:v1.2.0 Public:v1.1.0 Public:v1.0.0 Public:v0.0.3 Public:v0.0.2 Public:v0.0.1
...
pull from: Public:update_version
Branches Tags
Public:modified Public:chore/update-dependencies Public:main Public:chore/update-minio Public:slim-bean-patch-1 Public:update-metrics-port Public:add_loki_team_to_prs Public:test_branch Public:update_version Public:update_main_page Public:cleanup_ci Public:add_ci_install Public:add_ksm_docs Public:fix_cluster_name Public:fix_dashboards Public:fix_loki_write_endpoint Public:fix_traces_forwarding Public:change_default_sampling_type Public:update_ingress_documentation Public:create_new_version Public:add_more_metrics Public:fix_rules Public:remove_rules_files_mimir_tempo Public:split_up_grafana_template Public:remove_redundant_variables Public:fix_dependency_check Public:chore/update-tempo-distributed Public:fix_alloy_config_for_traces Public:fix_minio_secret_name Public:combine_ci Public:update_grafana Public:chore/update-loki Public:use_secret_for_minio Public:set_step_time_to_1_minute_in_dashboards Public:add_extra_metrics Public:more_dashboard_cleanup Public:remove_non_loki_dashboards Public:open_extra_ports_for_otlp Public:get_logs_from_all_namespaces Public:fix_correct_version Public:fix_logic_for_secrets Public:use_updated_dashboards2 Public:filter_cadvisor_kubelet_metrics_on_namespace Public:add_creation_of_dashboard Public:more_docs_fixes Public:add_more_docs Public:update_installation_instructions Public:fix_release_flow Public:add_ci_to_release_helm_chart Public:remove_reviewers Public:chore/update-grafana-alloy Public:chore/update-mimir-distributed Public:fix_action Public:add_helm_chart_updater Public:add_agent_metrics Public:filter_out_errors_local_loki Public:filter_log_lines Public:filter_only_important_log_lines Public:remove_helm_test_step Public:add_create_kind_cluster_to_ci Public:add_lint_to_ci Public:add_ci Public:add_skip_cdrs_to_installation_step Public:add_retention Public:fix_tempo_dashboards Public:fix_mimir_dashboards Public:add_agent_dashboards Public:add_ruler Public:add_tempo_dashboards.yaml Public:add_scraping_of_cadvisor Public:add_mimir_dashboards Public:use_loki_ssd_dashboards Public:add_kube_state_metrics_scraping Public:add_regexes_to_filter_log_lines Public:add_validation Public:enable_dbs_individually Public:add_minio
Public:v1.3.0 Public:v1.2.0 Public:v1.1.0 Public:v1.0.0 Public:v0.0.3 Public:v0.0.2 Public:v0.0.1

6 Commits
add_ci_ins ... update_ver

Author SHA1 Message Date
Michel Hollands
4b01214225 Update to version 1.0 
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
 2024-05-14 14:09:41 +01:00
Michel Hollands
0e63a86fe5 Merge pull request #124 from grafana/update_main_page 
Update the README and docs
 2024-05-14 14:08:00 +01:00
Michel Hollands
4e8b2be044 Update the README and docs 
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
 2024-05-14 14:06:49 +01:00
Michel Hollands
df12d96f9c Merge pull request #123 from grafana/cleanup_ci 
Comment out installation in CI for now
 2024-05-14 11:51:24 +01:00
Michel Hollands
fcb5de6793 Comment out installation in CI for now 
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
 2024-05-14 11:50:57 +01:00
Michel Hollands
661662caec Merge pull request #121 from grafana/add_ci_install 
Add CI install step
 2024-05-14 10:52:01 +01:00
5 changed files with 52 additions and 56 deletions
Expand all files Collapse all files

76
.github/workflows/helm-ci.yml vendored
View File

@@ -20,48 +20,48 @@ jobs:
- name: Lint Yaml - name: Lint Yaml
run: make helm-lint run: make helm-lint
call-test: # call-test:
name: Test Helm Chart # name: Test Helm Chart
runs-on: ubuntu-latest # runs-on: ubuntu-latest
steps: # steps:
- name: Checkout # - name: Checkout
uses: actions/checkout@v4 # uses: actions/checkout@v4
with: # with:
fetch-depth: 0 # fetch-depth: 0
- name: Set up Helm # - name: Set up Helm
uses: azure/setup-helm@v3 # uses: azure/setup-helm@v3
with: # with:
version: v3.8.2 # version: v3.8.2
# Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and # # Python is required because `ct lint` runs Yamale (https://github.com/23andMe/Yamale) and
# yamllint (https://github.com/adrienverge/yamllint) which require Python # # yamllint (https://github.com/adrienverge/yamllint) which require Python
- name: Set up Python # - name: Set up Python
uses: actions/setup-python@v4 # uses: actions/setup-python@v4
with: # with:
python-version: 3.9 # python-version: 3.9
- name: Set up chart-testing # - name: Set up chart-testing
uses: helm/chart-testing-action@v2 # uses: helm/chart-testing-action@v2
- name: Run chart-testing (list-changed) # - name: Run chart-testing (list-changed)
id: list-changed # id: list-changed
run: | # run: |
changed=$(ct list-changed --config "${CT_CONFIGFILE}") # changed=$(ct list-changed --config "${CT_CONFIGFILE}")
if [[ -n "$changed" ]]; then # if [[ -n "$changed" ]]; then
echo "changed=true" >> $GITHUB_OUTPUT # echo "changed=true" >> $GITHUB_OUTPUT
fi # fi
- name: Run chart-testing (lint) # - name: Run chart-testing (lint)
run: ct lint --config "${CT_CONFIGFILE}" --check-version-increment=false # run: ct lint --config "${CT_CONFIGFILE}" --check-version-increment=false
- name: Create kind cluster # - name: Create kind cluster
uses: helm/kind-action@v1 # uses: helm/kind-action@v1
if: steps.list-changed.outputs.changed == 'true' # if: steps.list-changed.outputs.changed == 'true'
with: # with:
config: "${{ github.workspace }}/.github/configs/cluster-config.yaml" # config: "${{ github.workspace }}/.github/configs/cluster-config.yaml"
- name: Run chart-testing (install) # - name: Run chart-testing (install)
run: | # run: |
changed=$(ct list-changed --config "${CT_CONFIGFILE}") # changed=$(ct list-changed --config "${CT_CONFIGFILE}")
ct install --config "${CT_CONFIGFILE}" # ct install --config "${CT_CONFIGFILE}"

16
README.md
View File

@@ -1,8 +1,6 @@
# meta-monitoring-chart # meta-monitoring-chart
This is a meta-monitoring chart for Loki. This is a meta-monitoring chart for Loki, specifically Loki installed via the Loki helm chart.
Note that this is pre-production software at the moment.
## Local and cloud modes ## Local and cloud modes
@@ -11,19 +9,15 @@ to small Loki, Mimir and Tempo installations running in the meta-monitoring name
![local mode](docs/images/Meta%20monitoring%20local.png) ![local mode](docs/images/Meta%20monitoring%20local.png)
To enable local mode set `local.<logs|metrics|traces>.enabled` to true.
In the cloud mode the logs, metrics and/or traces are sent to Grafana Cloud. In the cloud mode the logs, metrics and/or traces are sent to Grafana Cloud.
![cloud mode](docs/images/Meta%20monitoring%20cloud.png) ![cloud mode](docs/images/Meta%20monitoring%20cloud.png)
To enable cloud mode set `cloud.<logs|metrics|traces>.enabled` to true. The `endpoint`, `username` and `password` settings for your Grafana Cloud logs, metrics and traces instances have to be filled in as well.
Both modes can be enabled at the same time. Cloud mode is preferred. Both modes can be enabled at the same time. Cloud mode is preferred.
## Installation ## Installation
For more instructions including how to update the chart go to the [installation](docs/installation.md) page. For more instructions including how to install the chart go to the [installation](docs/installation.md) page.
## Supported features ## Supported features
@@ -33,8 +27,7 @@ For more instructions including how to update the chart go to the [installation]
- Specify PII regexes that are applied to logs before they are sent to Loki (cloud or local). The capture group in the regex is replaced with *****. - Specify PII regexes that are applied to logs before they are sent to Loki (cloud or local). The capture group in the regex is replaced with *****.
- a Grafana instance is installed (when local mode is used) with the relevant datasources installed. The following dashboards are installed: - a Grafana instance is installed (when local mode is used) with the relevant datasources installed. The following dashboards are installed:
- logs dashboards - logs dashboards
- agent dashboards - Alloy dashboards
- Retention is set to 24 hours
Most of these features are enabled by default. See the values.yaml file for how to enable/disable them. Most of these features are enabled by default. See the values.yaml file for how to enable/disable them.
@@ -42,8 +35,7 @@ Most of these features are enabled by default. See the values.yaml file for how
- This has not been tested on Openshift yet. - This has not been tested on Openshift yet.
- The underlying Loki, Mimir and Tempo are at the default size installed by the Helm chart. This might need changing when monitoring bigger Loki, Mimir or Tempo installations. - The underlying Loki, Mimir and Tempo are at the default size installed by the Helm chart. This might need changing when monitoring bigger Loki, Mimir or Tempo installations.
- MinIO is used as storage at the moment with a limited retention. At the moment this chart cannot be used for monitoring over longer periods. - MinIO is used as storage for the local mode at the moment with a limited retention. At the moment this chart cannot be used for monitoring over longer periods.
- Agent self monitoring is not done at the moment.
## Developer help topics ## Developer help topics

2
charts/meta-monitoring/Chart.yaml
View File

@@ -13,7 +13,7 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes # This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version. # to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/) # Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.0.3 version: 1.0.0
# This is the version number of the application being deployed. This version number should be # This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to # incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using. # follow Semantic Versioning. They should reflect the version the application is using.

6
docs/dev_update_dependencies.md
View File

@@ -1,8 +1,12 @@
# Update the dependencies # Update the dependencies
The dependencies are the version of Loki, Mimir, Agent and so on that are included in this chart. The dependencies are the versions of Loki, Mimir, Agent and so on that are included in this chart.
The current versions can be found in the [Chart.yaml](../charts/meta-monitoring/Chart.yaml) file. The current versions can be found in the [Chart.yaml](../charts/meta-monitoring/Chart.yaml) file.
A Github action runs daily to see if updated versions are available. A PR will be created.
The manual steps are as follows:
Run this in the charts/meta-monitoring directory after updating a dependency: Run this in the charts/meta-monitoring directory after updating a dependency:
``` ```

8
docs/installation.md
View File

@@ -4,7 +4,7 @@
1. Use an existing Grafana Cloud account or setup a new one. Then create an access token: 1. Use an existing Grafana Cloud account or setup a new one. Then create an access token:
1. In Grafana go to Administration -> Users and Access -> Cloud access policies. 1. In a Grafana instance on Grafana Cloud go to Administration -> Users and Access -> Cloud access policies.
1. Click `Create access policy`. 1. Click `Create access policy`.
@@ -39,7 +39,7 @@
--from-literal=endpoint='https://otlp-gateway-prod-us-east-0.grafana.net/otlp' --from-literal=endpoint='https://otlp-gateway-prod-us-east-0.grafana.net/otlp'
``` ```
The logs, metrics and traces usernames are the `User / Username / Instance IDs` of the Loki, Prometheus/Mimir and OpenTelemetry instances in Grafana Cloud. From `Home` in Grafana click on `Stacks`. Then go to the `Details` pages of Loki and Prometheus/Mimir. For OpenTelemetry go to the `Configure` page. The logs, metrics and traces usernames are the `User / Username / Instance IDs` of the Loki, Prometheus/Mimir and OpenTelemetry instances in Grafana Cloud. From `Home` in Grafana click on `Stacks`. Then go to the `Details` pages of Loki and Prometheus/Mimir. For OpenTelemetry go to the `Configure` page. The endpoints will also have to be changed to match your settings.
1. Create a values.yaml file based on the [default one](../charts/meta-monitoring/values.yaml). Fill in the names of the secrets created above as needed. An example minimal values.yaml looks like this: 1. Create a values.yaml file based on the [default one](../charts/meta-monitoring/values.yaml). Fill in the names of the secrets created above as needed. An example minimal values.yaml looks like this:
@@ -102,7 +102,7 @@
enabled: true enabled: true
``` ```
## Installing the chart ## Installing, updating and deleting the chart
1. Add the repo 1. Add the repo
@@ -175,7 +175,7 @@ For each of the dashboard files in charts/meta-monitoring/src/dashboards folder
## Configure Loki to send traces ## Configure Loki to send traces
1. In the Loki config enable tracing: 1. In the Loki that is being monitored enable tracing in the config:
``` ```
loki: loki: