Compare commits

..

45 Commits

Author SHA1 Message Date
Michel Hollands
34545e15b4 Merge pull request #66 from grafana/add_ci_to_release_helm_chart
Add CI to publish helm chart
2024-04-22 10:17:37 +01:00
Michel Hollands
33b8e37bed Update helm release and add required files
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-22 09:53:29 +01:00
Michel Hollands
b1975505e5 Add CI to publish helm chart
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-16 14:44:41 +01:00
Michel Hollands
c282bf352d Merge pull request #65 from grafana/update_loki_config_for_3_0
Update Loki config to match 3.0
2024-04-16 11:01:17 +01:00
Michel Hollands
60af0b4d19 Update Loki config to match 3.0
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-16 10:00:20 +01:00
Michel Hollands
0980cb2ede Merge pull request #63 from grafana/chore/update-loki
[dependency] Update the Loki subchart
2024-04-16 09:17:42 +01:00
Michel Hollands
75ab1f0d97 Merge pull request #64 from grafana/remove_reviewers
Remove reviewers
2024-04-16 09:16:07 +01:00
Michel Hollands
dd49623508 Remove reviewers
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-16 09:15:45 +01:00
MichelHollands
095fb09d26 Update loki 2024-04-16 08:10:07 +00:00
Michel Hollands
0fc5e2f847 Merge pull request #60 from grafana/chore/update-tempo-distributed
[dependency] Update the Tempo Distributed subchart
2024-04-16 09:08:02 +01:00
Michel Hollands
d04d74cc26 Merge pull request #62 from grafana/use_loki_squad
Use loki-squad, run later
2024-04-16 09:07:26 +01:00
Michel Hollands
b840555522 Use loki-squad, run later
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-16 09:06:59 +01:00
MichelHollands
e1a8495227 Update Tempo Distributed 2024-04-16 08:02:28 +00:00
Michel Hollands
a812b4f63a Merge pull request #56 from grafana/add_reviewers
Add team-reviewers and fix text
2024-04-16 09:01:18 +01:00
Michel Hollands
8dde9642c9 Add team-reviewers and fix text
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-16 08:57:30 +01:00
Michel Hollands
0832bc8e8f Merge pull request #49 from grafana/chore/update-tempo-distributed
[dependency] Update tempo Distributed
2024-04-15 16:57:21 +01:00
Michel Hollands
cbae75acb8 Merge branch 'main' into chore/update-tempo-distributed 2024-04-15 16:57:09 +01:00
Michel Hollands
f5a5472b95 Merge pull request #50 from grafana/chore/update-minio
[dependency] Update minio
2024-04-15 16:53:24 +01:00
MichelHollands
58a4696a6b Update Tempo Distributed 2024-04-15 15:51:05 +00:00
MichelHollands
e31f6b0906 Update minio 2024-04-15 15:51:04 +00:00
Michel Hollands
351f50e238 Merge pull request #47 from grafana/fix_action
Use correct action file
2024-04-15 16:50:23 +01:00
Michel Hollands
170c17b721 Use correct action file
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 16:49:47 +01:00
Michel Hollands
b2d06ab8e3 Merge pull request #46 from grafana/fix_tempo_update_filename
Add extension
2024-04-15 16:47:04 +01:00
Michel Hollands
9d63c32d4f Add extension
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 16:46:32 +01:00
Michel Hollands
a201cef34c Merge pull request #44 from grafana/update_versions
Update versions
2024-04-15 16:43:25 +01:00
Michel Hollands
ea4d5e278a Add update github actions
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 16:43:00 +01:00
Michel Hollands
e3c3f6a094 Add updates for Loki, Tempo and Minio
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 16:40:57 +01:00
Michel Hollands
d6da6fec35 Merge pull request #42 from grafana/update_mimir_dependency
Update Mimir subschart
2024-04-15 16:19:07 +01:00
Michel Hollands
5e2ffb222b Merge pull request #43 from grafana/chore/update-mimir-distributed
[dependency] Update Mimir Distributed
2024-04-15 16:18:42 +01:00
MichelHollands
9b12bad16c Update Mimir Distributed 2024-04-15 15:17:35 +00:00
Michel Hollands
ed31bcf345 Update Mimir subschart
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 16:15:50 +01:00
Michel Hollands
a0184e27d0 Merge pull request #40 from grafana/fix_workflow
Use correct path
2024-04-15 14:58:50 +01:00
Michel Hollands
3491886311 Merge pull request #41 from grafana/chore/update-grafana-alloy
[dependency] Update Grafana Alloy
2024-04-15 14:58:18 +01:00
MichelHollands
c58b76cfc7 Update Grafana Alloy 2024-04-15 13:56:36 +00:00
Michel Hollands
a72e64327f Use correct path
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 14:55:42 +01:00
Michel Hollands
cfdc6b95eb Merge pull request #39 from grafana/add_helm_chart_updater
Add Github workflow to update Alloy chart version
2024-04-15 14:50:56 +01:00
Michel Hollands
b1ccef91cb Add Github workflow to update Alloy chart version
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 14:44:54 +01:00
Michel Hollands
f3f970d783 Merge pull request #38 from grafana/upgrade_to_alloy
Use Alloy instead of Grafana Agent
2024-04-15 11:33:24 +01:00
Michel Hollands
b78571dfdc Use alloy Helm chart 0.1.1
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 10:16:45 +01:00
Michel Hollands
1859c3a82c Use alloy instead of Grafana Agent
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-15 10:08:42 +01:00
Michel Hollands
f275b2d1b6 Merge pull request #36 from grafana/add_more_metrics
Use new Loki dashboards
2024-04-11 10:09:50 +01:00
Michel Hollands
fd1aadc099 Merge pull request #37 from grafana/add_agent_metrics
Scrape agent metrics as well
2024-04-10 15:29:13 +01:00
Michel Hollands
690cda9eb5 Add ruler metrics
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-10 14:43:32 +01:00
Michel Hollands
00cad594f4 Add more metrics
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-10 14:34:09 +01:00
Michel Hollands
e74ec96349 Use new loki dashboards
Signed-off-by: Michel Hollands <michel.hollands@gmail.com>
2024-04-10 14:20:48 +01:00
32 changed files with 4134 additions and 3687 deletions

3
.github/configs/cr.yaml vendored Normal file
View File

@@ -0,0 +1,3 @@
owner: grafana
git-repo: helm-charts
skip-existing: true

15
.github/configs/ct.yaml vendored Normal file
View File

@@ -0,0 +1,15 @@
## Reference: https://github.com/helm/chart-testing/blob/master/doc/ct_lint-and-install.md
remote: origin
target-branch: main
chart-dirs:
- charts
chart-repos:
- grafana=https://grafana.github.io/helm-charts
- minio=https://charts.min.io
validate-chart-schema: true
validate-maintainers: true
validate-yaml: true
exclude-deprecated: true
excluded-charts: []
namespace: meta-monitoring # Need to set the namespace because we create the secret there
release-label: app.kubernetes.io/instance

30
.github/configs/updatecli.d/alloy.yaml vendored Normal file
View File

@@ -0,0 +1,30 @@
name: Bump dependency "alloy" for Helm chart "meta-monitoring"
sources:
alloy:
name: Get latest "alloy" Helm chart version
kind: helmchart
spec:
name: alloy
url: https://grafana.github.io/helm-charts
versionfilter:
kind: semver
pattern: '*'
conditions:
alloy:
name: Ensure Helm chart dependency "alloy" is specified
kind: yaml
spec:
file: charts/meta-monitoring/Chart.yaml
key: $.dependencies[1].name
value: alloy
disablesourceinput: true
targets:
alloy:
name: Bump Helm chart dependency "alloy" for Helm chart "meta-monitoring"
kind: helmchart
spec:
file: Chart.yaml
key: $.dependencies[1].version
name: charts/meta-monitoring
versionincrement: none
sourceid: alloy

30
.github/configs/updatecli.d/loki.yaml vendored Normal file
View File

@@ -0,0 +1,30 @@
name: Bump dependency "loki" for Helm chart "meta-monitoring"
sources:
loki:
name: Get latest "loki" Helm chart version
kind: helmchart
spec:
name: loki
url: https://grafana.github.io/helm-charts
versionfilter:
kind: semver
pattern: '*'
conditions:
loki:
name: Ensure Helm chart dependency "loki" is specified
kind: yaml
spec:
file: charts/meta-monitoring/Chart.yaml
key: $.dependencies[0].name
value: loki
disablesourceinput: true
targets:
loki:
name: Bump Helm chart dependency "loki" for Helm chart "meta-monitoring"
kind: helmchart
spec:
file: Chart.yaml
key: $.dependencies[0].version
name: charts/meta-monitoring
versionincrement: none
sourceid: loki

View File

@@ -0,0 +1,30 @@
name: Bump dependency "mimir-distributed" for Helm chart "meta-monitoring"
sources:
mimir-distributed:
name: Get latest "mimir-distributed" Helm chart version
kind: helmchart
spec:
name: mimir-distributed
url: https://grafana.github.io/helm-charts
versionfilter:
kind: semver
pattern: '*'
conditions:
mimir-distributed:
name: Ensure Helm chart dependency "mimir-distributed" is specified
kind: yaml
spec:
file: charts/meta-monitoring/Chart.yaml
key: $.dependencies[2].name
value: mimir-distributed
disablesourceinput: true
targets:
mimir-distributed:
name: Bump Helm chart dependency "mimir-distributed" for Helm chart "meta-monitoring"
kind: helmchart
spec:
file: Chart.yaml
key: $.dependencies[2].version
name: charts/meta-monitoring
versionincrement: none
sourceid: mimir-distributed

30
.github/configs/updatecli.d/minio.yaml vendored Normal file
View File

@@ -0,0 +1,30 @@
name: Bump dependency "minio" for Helm chart "meta-monitoring"
sources:
minio:
name: Get latest "minio" Helm chart version
kind: helmchart
spec:
name: minio
url: https://charts.min.io
versionfilter:
kind: semver
pattern: '*'
conditions:
minio:
name: Ensure Helm chart dependency "minio" is specified
kind: yaml
spec:
file: charts/meta-monitoring/Chart.yaml
key: $.dependencies[4].name
value: minio
disablesourceinput: true
targets:
minio:
name: Bump Helm chart dependency "minio" for Helm chart "meta-monitoring"
kind: helmchart
spec:
file: Chart.yaml
key: $.dependencies[4].version
name: charts/meta-monitoring
versionincrement: none
sourceid: minio

View File

@@ -0,0 +1,30 @@
name: Bump dependency "tempo-distributed" for Helm chart "meta-monitoring"
sources:
tempo-distributed:
name: Get latest "tempo-distributed" Helm chart version
kind: helmchart
spec:
name: tempo-distributed
url: https://grafana.github.io/helm-charts
versionfilter:
kind: semver
pattern: '*'
conditions:
tempo-distributed:
name: Ensure Helm chart dependency "tempo-distributed" is specified
kind: yaml
spec:
file: charts/meta-monitoring/Chart.yaml
key: $.dependencies[3].name
value: tempo-distributed
disablesourceinput: true
targets:
tempo-distributed:
name: Bump Helm chart dependency "tempo-distributed" for Helm chart "meta-monitoring"
kind: helmchart
spec:
file: Chart.yaml
key: $.dependencies[3].version
name: charts/meta-monitoring
versionincrement: none
sourceid: tempo-distributed

View File

@@ -0,0 +1,177 @@
---
name: Check for dependency updates
on:
workflow_dispatch:
schedule:
# Run once a day
- cron: '0 7 * * *'
permissions:
contents: "write"
pull-requests: "write"
env:
UPDATECLI_CONFIG_DIR: "${{ github.workspace }}/.github/configs/updatecli.d"
UPDATECLI_GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
jobs:
updateLoki:
name: Update the Loki subchart
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Updatecli
uses: updatecli/updatecli-action@v2
- name: Run Updatecli
id: update-loki
run: |
updatecli apply --config ${UPDATECLI_CONFIG_DIR}/loki.yaml
if ! git diff --exit-code > /dev/null; then
echo "changed=true" >> "${GITHUB_OUTPUT}"
fi
- name: Create pull request
if: steps.update-loki.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
title: "[dependency] Update the Loki subchart"
body: "Updates the Loki subchart"
base: main
author: "${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>"
committer: "GitHub <noreply@github.com>"
commit-message: Update loki
labels: dependencies
branch: chore/update-loki
delete-branch: true
updateGrafanaAlloy:
name: Update the Grafana Alloy subchart
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Updatecli
uses: updatecli/updatecli-action@v2
- name: Run Updatecli
id: update-grafana-alloy
run: |
updatecli apply --config ${UPDATECLI_CONFIG_DIR}/alloy.yaml
if ! git diff --exit-code > /dev/null; then
echo "changed=true" >> "${GITHUB_OUTPUT}"
fi
- name: Create pull request
if: steps.update-grafana-alloy.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
title: "[dependency] Update the Grafana Alloy subchart"
body: "Updates the Grafana Alloy subchart"
base: main
author: "${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>"
committer: "GitHub <noreply@github.com>"
commit-message: Update Grafana Alloy
labels: dependencies
branch: chore/update-grafana-alloy
delete-branch: true
updateMimirDistributed:
name: Update the Mimir Distributed subchart
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Updatecli
uses: updatecli/updatecli-action@v2
- name: Run Updatecli
id: update-mimir-distributed
run: |
updatecli apply --config ${UPDATECLI_CONFIG_DIR}/mimir-distributed.yaml
if ! git diff --exit-code > /dev/null; then
echo "changed=true" >> "${GITHUB_OUTPUT}"
fi
- name: Create pull request
if: steps.update-mimir-distributed.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
title: "[dependency] Update the Mimir Distributed subchart"
body: "Updates the Mimir Distributed subchart"
base: main
author: "${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>"
committer: "GitHub <noreply@github.com>"
commit-message: Update Mimir Distributed
labels: dependencies
branch: chore/update-mimir-distributed
delete-branch: true
updateTempoDistributed:
name: Update the Tempo Distributed subchart
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Updatecli
uses: updatecli/updatecli-action@v2
- name: Run Updatecli
id: update-tempo-distributed
run: |
updatecli apply --config ${UPDATECLI_CONFIG_DIR}/tempo-distributed.yaml
if ! git diff --exit-code > /dev/null; then
echo "changed=true" >> "${GITHUB_OUTPUT}"
fi
- name: Create pull request
if: steps.update-tempo-distributed.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
title: "[dependency] Update the Tempo Distributed subchart"
body: "Updates the tempo Distributed subchart"
base: main
author: "${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>"
committer: "GitHub <noreply@github.com>"
commit-message: Update Tempo Distributed
labels: dependencies
branch: chore/update-tempo-distributed
delete-branch: true
updateMinio:
name: Update the Minio subchart
runs-on: "ubuntu-latest"
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Install Updatecli
uses: updatecli/updatecli-action@v2
- name: Run Updatecli
id: update-minio
run: |
updatecli apply --config ${UPDATECLI_CONFIG_DIR}/minio.yaml
if ! git diff --exit-code > /dev/null; then
echo "changed=true" >> "${GITHUB_OUTPUT}"
fi
- name: Create pull request
if: steps.update-minio.outputs.changed == 'true'
uses: peter-evans/create-pull-request@v5
with:
title: "[dependency] Update the Minio subchart"
body: "Updates the Minio subchart"
base: main
author: "${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>"
committer: "GitHub <noreply@github.com>"
commit-message: Update minio
labels: dependencies
branch: chore/update-minio
delete-branch: true

167
.github/workflows/helm-release.yml vendored Normal file
View File

@@ -0,0 +1,167 @@
name: Release Helm chart
on:
workflow_dispatch:
env:
CR_CONFIGFILE: "${{ github.workspace }}/source/.github/configs/cr.yaml"
CT_CONFIGFILE: "${{ github.workspace }}/source/.github/configs/ct.yaml"
CR_INDEX_PATH: "${{ github.workspace }}/.cr-index"
CR_PACKAGE_PATH: "${{ github.workspace }}/.cr-release-packages"
CR_TOOL_PATH: "${{ github.workspace }}/.cr-tool"
CR_VERSION: "1.5.0"
permissions:
contents: read
id-token: write
jobs:
setup:
runs-on: ubuntu-latest
outputs:
changed: ${{ steps.list-changed.outputs.changed }}
chartpath: ${{ steps.list-changed.outputs.chartpath }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
path: source
- name: Install chart-testing
uses: helm/chart-testing-action@v2
- name: List changed charts
id: list-changed
run: |
cd source
latest_tag=$( if ! git describe --tags --abbrev=0 --match='helm-chart/*' 2> /dev/null ; then git rev-list --max-parents=0 --first-parent HEAD; fi )
echo "Running: ct list-changed --config ${CT_CONFIGFILE} --since ${latest_tag} --target-branch ${{ github.ref_name }}"
changed=$(ct list-changed --config "${CT_CONFIGFILE}" --since "${latest_tag}" --target-branch "${{ github.ref_name }}")
echo "${changed}"
num_changed=$(wc -l <<< ${changed})
if [[ "${num_changed}" -gt "1" ]] ; then
echo "More than one chart changed, exiting"
exit 1
fi
if [[ -n "${changed}" ]]; then
name=$(yq ".name" < ${changed}/Chart.yaml)
version=$(yq ".version" < ${changed}/Chart.yaml)
tagname="v${version}"
if [ $(git tag -l "${tagname}") ]; then
echo "Tag ${tagname} already exists, skipping release"
echo "changed=false" >> $GITHUB_OUTPUT
else
echo "Releasing ${changed}"
echo "changed=true" >> $GITHUB_OUTPUT
echo "chartpath=${changed}" >> $GITHUB_OUTPUT
fi
else
echo "No charts have changed, skipping release"
echo "changed=false" >> $GITHUB_OUTPUT
fi
release:
needs: [setup]
runs-on: ubuntu-latest
if: needs.setup.outputs.changed == 'true'
steps:
- uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.app-id }}
private-key: ${{ secrets.private-key }}
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
path: source
- name: Configure Git
run: |
cd source
git config user.name "$GITHUB_ACTOR"
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Checkout helm-charts
uses: actions/checkout@v4
with:
fetch-depth: 0
repository: grafana/helm-charts
path: helm-charts
token: "${{ steps.app-token.outputs.token }}"
- name: Configure Git for helm-charts
run: |
cd helm-charts
git config user.name "$GITHUB_ACTOR"
git config user.email "$GITHUB_ACTOR@users.noreply.github.com"
- name: Set up Helm
uses: azure/setup-helm@v4
- name: Parse Chart.yaml
id: parse-chart
run: |
cd source
changed="${{ needs.setup.outputs.chartpath }}"
description=$(yq ".description" < ${changed}/Chart.yaml)
name=$(yq ".name" < ${changed}/Chart.yaml)
version=$(yq ".version" < ${changed}/Chart.yaml)
echo "chartpath=${changed}" >> $GITHUB_OUTPUT
echo "desc=${description}" >> $GITHUB_OUTPUT
echo "tagname=v${version}" >> $GITHUB_OUTPUT
echo "packagename=${name}-${version}" >> $GITHUB_OUTPUT
- name: Install CR tool
run: |
mkdir "${CR_TOOL_PATH}"
mkdir "${CR_PACKAGE_PATH}"
mkdir "${CR_INDEX_PATH}"
curl -sSLo cr.tar.gz "https://github.com/helm/chart-releaser/releases/download/v${CR_VERSION}/chart-releaser_${CR_VERSION}_linux_amd64.tar.gz"
tar -xzf cr.tar.gz -C "${CR_TOOL_PATH}"
rm -f cr.tar.gz
- name: Create Helm package
run: |
cd source
helm repo add grafana https://grafana.github.io/helm-charts
helm repo add minio https://charts.min.io
"${CR_TOOL_PATH}/cr" package "${{ steps.parse-chart.outputs.chartpath }}" --config "${CR_CONFIGFILE}" --package-path "${CR_PACKAGE_PATH}"
- name: Make a release on this repo
uses: softprops/action-gh-release@v2
with:
name: ${{ steps.parse-chart.outputs.tagname }}
repository: grafana/meta-monitoring-chart
tag_name: ${{ steps.parse-chart.outputs.tagname }}
token: ${{ steps.app-token.outputs.token }}
generate_release_notes: true
files: |
${{ env.CR_PACKAGE_PATH }}/${{ steps.parse-chart.outputs.packagename }}.tgz
# Note that this creates a release in grafana/helm-charts with a new tag.
# The tag name in grafana/helm-charts is <package>-<version>, while the
# tag name for grafana/meta-monitoring-chart is <version>.
- name: Make release on Helm Charts
uses: softprops/action-gh-release@v2
with:
name: ${{ steps.parse-chart.outputs.packagename }}
repository: grafana/helm-charts
tag_name: ${{ steps.parse-chart.outputs.packagename }}
token: ${{ steps.app-token.outputs.token }}
body: |
${{ steps.parse-chart.outputs.desc }}
Source commit: https://github.com/${{ github.repository }}/commit/${{ github.sha }}
Tag on source: https://github.com/${{ github.repository }}/releases/tag/${{ steps.parse-chart.outputs.tagname }}
files: |
${{ env.CR_PACKAGE_PATH }}/${{ steps.parse-chart.outputs.packagename }}.tgz
- name: Update helm-charts index.yaml
run: |
cd helm-charts
"${CR_TOOL_PATH}/cr" index --config "${CR_CONFIGFILE}" --token "${{ steps.app-token.outputs.token }}" --index-path "${CR_INDEX_PATH}" --package-path "${CR_PACKAGE_PATH}" --push

View File

@@ -1,18 +1,18 @@
dependencies:
- name: loki
repository: https://grafana.github.io/helm-charts
version: 5.47.2
- name: grafana-agent
version: 6.2.0
- name: alloy
repository: https://grafana.github.io/helm-charts
version: 0.37.0
version: 0.1.1
- name: mimir-distributed
repository: https://grafana.github.io/helm-charts
version: 5.2.0
version: 5.3.0
- name: tempo-distributed
repository: https://grafana.github.io/helm-charts
version: 1.9.1
version: 1.9.2
- name: minio
repository: https://charts.min.io
version: 5.0.11
digest: sha256:7b7e62e08d9a56e63fdb12ce3fd4d1fda4887545546ac3e98c7886be714fd763
generated: "2024-04-02T15:09:13.121195+01:00"
version: 5.1.0
digest: sha256:f9a79bfc30df65ba2ff94a097844f6b3aa41970318d5fb1708b3aeecebbe68d1
generated: "2024-04-16T08:10:03.998180905Z"

View File

@@ -1,7 +1,6 @@
apiVersion: v2
name: meta-monitoring
description: A Helm chart for meta monitoring Grafana Loki, Mimir and Tempo
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
@@ -11,35 +10,32 @@ description: A Helm chart for meta monitoring Grafana Loki, Mimir and Tempo
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.0.1
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "0.0.1"
dependencies:
- name: loki
repository: https://grafana.github.io/helm-charts
version: "5.47.2"
version: 6.2.0
condition: local.logs.enabled
- name: grafana-agent
- name: alloy
repository: https://grafana.github.io/helm-charts
version: "0.37.0"
version: 0.1.1
- name: mimir-distributed
repository: https://grafana.github.io/helm-charts
version: "5.2.0"
version: 5.3.0
condition: local.metrics.enabled
- name: tempo-distributed
repository: https://grafana.github.io/helm-charts
version: "1.9.1"
version: 1.9.2
condition: local.traces.enabled
- name: minio
repository: https://charts.min.io
version: "5.0.11"
version: 5.1.0
condition: local.minio.enabled

Binary file not shown.

Binary file not shown.

Binary file not shown.

File diff suppressed because it is too large Load Diff

View File

@@ -62,7 +62,6 @@
"expr": "sum(loki_compactor_pending_delete_requests_count{cluster=~\"$cluster\", namespace=~\"$namespace\"})",
"format": "time_series",
"instant": true,
"intervalFactor": 2,
"refId": "A"
}
],
@@ -138,7 +137,6 @@
"expr": "max(loki_compactor_oldest_pending_delete_request_age_seconds{cluster=~\"$cluster\", namespace=~\"$namespace\"})",
"format": "time_series",
"instant": true,
"intervalFactor": 2,
"refId": "A"
}
],
@@ -191,232 +189,145 @@
"height": "250px",
"panels": [
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 3,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 3,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "(loki_compactor_delete_requests_received_total{cluster=~\"$cluster\", namespace=~\"$namespace\"} or on() vector(0)) - on () (loki_compactor_delete_requests_processed_total{cluster=~\"$cluster\", namespace=~\"$namespace\"} or on () vector(0))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "in progress",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "# of Delete Requests (received - processed) ",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 4,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(increase(loki_compactor_delete_requests_received_total{cluster=~\"$cluster\", namespace=~\"$namespace\"}[1d]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "received",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Delete Requests Received / Day",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 5,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 5,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(increase(loki_compactor_delete_requests_processed_total{cluster=~\"$cluster\", namespace=~\"$namespace\"}[1d]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "processed",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Delete Requests Processed / Day",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
}
],
"repeat": null,
@@ -431,232 +342,145 @@
"height": "250px",
"panels": [
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 6,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 6,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "node_namespace_pod_container:container_cpu_usage_seconds_total:sum_irate{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"compactor\"}",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Compactor CPU usage",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 7,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 7,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "go_memstats_heap_inuse_bytes{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"compactor\"} / 1024 / 1024 ",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": " {{pod}} ",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Compactor memory usage (MiB)",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 8,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 8,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "loki_boltdb_shipper_compact_tables_operation_duration_seconds{cluster=~\"$cluster\", namespace=~\"$namespace\"}",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Compaction run duration (seconds)",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
}
],
"repeat": null,
@@ -671,156 +495,98 @@
"height": "250px",
"panels": [
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 9,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 9,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(increase(loki_compactor_load_pending_requests_attempts_total{status=\"fail\", cluster=~\"$cluster\", namespace=~\"$namespace\"}[1h]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "failures",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Failures in Loading Delete Requests / Hour",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 10,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 10,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(loki_compactor_deleted_lines{cluster=~\"$cluster\",job=~\"$namespace/(loki|enterprise-logs)-read\"}[$__rate_interval])) by (user)",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{user}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Lines Deleted / Sec",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
}
],
"repeat": null,
@@ -882,7 +648,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",

View File

@@ -77,7 +77,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -164,7 +164,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -250,7 +250,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -336,7 +336,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -422,7 +422,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -508,7 +508,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -595,7 +595,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -682,7 +682,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -718,7 +718,7 @@
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$logs",
"datasource": "$loki_datasource",
"fill": 1,
"fillGradient": 0,
"gridPos": {
@@ -787,7 +787,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -819,7 +819,7 @@
}
},
{
"datasource": "$logs",
"datasource": "$loki_datasource",
"gridPos": {
"h": 19,
"w": 24,
@@ -861,7 +861,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",
@@ -918,7 +918,7 @@
{
"hide": 0,
"label": null,
"name": "logs",
"name": "loki_datasource",
"options": [ ],
"query": "loki",
"refresh": 1,
@@ -1069,4 +1069,4 @@
"title": "Loki / Logs",
"uid": "logs",
"version": 0
}
}

View File

@@ -600,7 +600,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",

View File

@@ -102,7 +102,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -198,7 +198,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -293,7 +293,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -389,7 +389,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -485,7 +485,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -591,7 +591,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -697,7 +697,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -794,7 +794,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -903,7 +903,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1000,7 +1000,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1109,7 +1109,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1215,7 +1215,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1312,7 +1312,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1420,7 +1420,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1517,7 +1517,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1618,7 +1618,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1799,9 +1799,9 @@
"steppedLine": false,
"targets": [
{
"expr": "node_namespace_pod_container:container_cpu_usage_seconds_total:sum_irate{cluster=\"$cluster\", namespace=\"$namespace\", pod=~\"(loki|enterprise-logs)-write.*\"}",
"expr": "sum by(pod) (rate(container_cpu_usage_seconds_total{cluster=~\"$cluster\", namespace=~\"$namespace\", pod=~\"(loki|enterprise-logs)-write.*\"}[$__rate_interval]))",
"intervalFactor": 3,
"legendFormat": "{{pod}}-{{container}}",
"legendFormat": "{{pod}}",
"refId": "A"
}
],
@@ -1815,7 +1815,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1907,7 +1907,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -1943,7 +1943,7 @@
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "$logs",
"datasource": "$loki_datasource",
"fill": 1,
"fillGradient": 0,
"gridPos": {
@@ -1999,7 +1999,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2031,7 +2031,7 @@
}
},
{
"datasource": "$logs",
"datasource": "$loki_datasource",
"gridPos": {
"h": 18,
"w": 12,
@@ -2116,7 +2116,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2205,7 +2205,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2294,7 +2294,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2383,7 +2383,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2490,7 +2490,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2581,7 +2581,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2697,7 +2697,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -2850,7 +2850,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3006,9 +3006,9 @@
"steppedLine": false,
"targets": [
{
"expr": "node_namespace_pod_container:container_cpu_usage_seconds_total:sum_irate{cluster=\"$cluster\", namespace=\"$namespace\", pod=~\"(loki|enterprise-logs)-read.*\"}",
"expr": "sum by(pod) (rate(container_cpu_usage_seconds_total{cluster=~\"$cluster\", namespace=~\"$namespace\", pod=~\"(loki|enterprise-logs)-read.*\"}[$__rate_interval]))",
"intervalFactor": 3,
"legendFormat": "{{pod}}-{{container}}",
"legendFormat": "{{pod}}",
"refId": "A"
}
],
@@ -3022,7 +3022,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3114,7 +3114,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3150,7 +3150,7 @@
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "$logs",
"datasource": "$loki_datasource",
"fill": 1,
"fillGradient": 0,
"gridPos": {
@@ -3206,7 +3206,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3238,7 +3238,7 @@
}
},
{
"datasource": "$logs",
"datasource": "$loki_datasource",
"gridPos": {
"h": 18,
"w": 18,
@@ -3323,7 +3323,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3443,7 +3443,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3535,7 +3535,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3655,7 +3655,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3747,7 +3747,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3861,7 +3861,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -3964,7 +3964,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4067,7 +4067,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4165,7 +4165,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4255,7 +4255,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4345,7 +4345,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4435,7 +4435,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4525,7 +4525,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4645,7 +4645,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4737,7 +4737,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4825,7 +4825,7 @@
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(cortex_dynamo_failures_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"expr": "sum(rate(loki_dynamo_failures_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"refId": "A"
}
],
@@ -4839,7 +4839,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4911,7 +4911,7 @@
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(cortex_dynamo_consumed_capacity_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"expr": "sum(rate(loki_dynamo_consumed_capacity_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"refId": "A"
}
],
@@ -4925,7 +4925,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -4997,7 +4997,7 @@
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(cortex_dynamo_throttled_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"expr": "sum(rate(loki_dynamo_throttled_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"refId": "A"
}
],
@@ -5011,7 +5011,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5083,7 +5083,7 @@
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(cortex_dynamo_dropped_requests_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"expr": "sum(rate(loki_dynamo_dropped_requests_total{cluster=\"$cluster\", namespace=\"$namespace\"}[5m]))",
"refId": "A"
}
],
@@ -5097,7 +5097,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5169,17 +5169,17 @@
"steppedLine": false,
"targets": [
{
"expr": "histogram_quantile(.99, sum(rate(cortex_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"expr": "histogram_quantile(.99, sum(rate(loki_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"legendFormat": ".99",
"refId": "A"
},
{
"expr": "histogram_quantile(.9, sum(rate(cortex_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"expr": "histogram_quantile(.9, sum(rate(loki_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"legendFormat": ".9",
"refId": "B"
},
{
"expr": "histogram_quantile(.5, sum(rate(cortex_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"expr": "histogram_quantile(.5, sum(rate(loki_dynamo_query_pages_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])))",
"legendFormat": ".5",
"refId": "C"
}
@@ -5194,7 +5194,7 @@
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5269,19 +5269,19 @@
"steppedLine": false,
"targets": [
{
"expr": "histogram_quantile(.99, sum(rate(cortex_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"expr": "histogram_quantile(.99, sum(rate(loki_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"intervalFactor": 1,
"legendFormat": ".99-{{operation}}",
"refId": "A"
},
{
"expr": "histogram_quantile(.9, sum(rate(cortex_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"expr": "histogram_quantile(.9, sum(rate(loki_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"hide": false,
"legendFormat": ".9-{{operation}}",
"refId": "B"
},
{
"expr": "histogram_quantile(.5, sum(rate(cortex_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"expr": "histogram_quantile(.5, sum(rate(loki_dynamo_request_duration_seconds_bucket{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (operation, le))",
"hide": false,
"legendFormat": ".5-{{operation}}",
"refId": "C"
@@ -5297,7 +5297,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5372,7 +5372,7 @@
"steppedLine": false,
"targets": [
{
"expr": "sum(rate(cortex_dynamo_request_duration_seconds_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (status_code, operation)",
"expr": "sum(rate(loki_dynamo_request_duration_seconds_count{cluster=\"$cluster\", namespace=\"$namespace\"}[5m])) by (status_code, operation)",
"intervalFactor": 1,
"legendFormat": "{{status_code}}-{{operation}}",
"refId": "A"
@@ -5388,7 +5388,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5507,7 +5507,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5598,7 +5598,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5717,7 +5717,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5808,7 +5808,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -5927,7 +5927,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -6018,7 +6018,7 @@
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"type": "timeseries",
"xaxis": {
"buckets": null,
"mode": "time",
@@ -6070,7 +6070,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",
@@ -6081,7 +6081,7 @@
{
"hide": 0,
"label": null,
"name": "logs",
"name": "loki_datasource",
"options": [ ],
"query": "loki",
"refresh": 1,
@@ -6169,4 +6169,4 @@
"title": "Loki / Operational",
"uid": "operational",
"version": 0
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -33,113 +33,234 @@
"3xx": "#6ED0E0",
"4xx": "#EF843C",
"5xx": "#E24D42",
"OK": "#7EB26D",
"cancel": "#A9A9A9",
"error": "#E24D42",
"success": "#7EB26D"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "1xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EAB839",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "2xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "3xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#6ED0E0",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "4xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EF843C",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "5xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "OK"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "cancel"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#A9A9A9",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "error"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "success"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
}
]
},
"fill": 10,
"id": 1,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 0,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", route=~\"loki_api_v1_series|api_prom_series|api_prom_query|api_prom_label|api_prom_label_name_values|loki_api_v1_query|loki_api_v1_query_range|loki_api_v1_labels|loki_api_v1_label_name_values\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-z]+)\"))\n",
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", route=~\"loki_api_v1_series|api_prom_series|api_prom_query|api_prom_label|api_prom_label_name_values|loki_api_v1_query|loki_api_v1_query_range|loki_api_v1_labels|loki_api_v1_label_name_values\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-zA-Z]+)\"))\n",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{status}}",
"refId": "A",
"step": 10
"refId": "A"
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "QPS",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 2,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"targets": [
{
"expr": "histogram_quantile(0.99, sum by (le,route) (cluster_job_route:loki_request_duration_seconds_bucket:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-read\", route=~\"loki_api_v1_series|api_prom_series|api_prom_query|api_prom_label|api_prom_label_name_values|loki_api_v1_query|loki_api_v1_query_range|loki_api_v1_labels|loki_api_v1_label_name_values\"})) * 1e3",
@@ -166,23 +287,8 @@
"step": 10
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Latency",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"type": "timeseries",
"yaxes": [
{
"format": "ms",
@@ -201,6 +307,57 @@
"show": false
}
]
},
{
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"id": 3,
"links": [ ],
"nullPointMode": "null as zero",
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"targets": [
{
"expr": "histogram_quantile(0.99, sum(rate(loki_request_duration_seconds_bucket{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-read\", route=~\"loki_api_v1_series|api_prom_series|api_prom_query|api_prom_label|api_prom_label_name_values|loki_api_v1_query|loki_api_v1_query_range|loki_api_v1_labels|loki_api_v1_label_name_values\"}[$__rate_interval])) by (le,pod)) * 1e3",
"format": "time_series",
"interval": "1m",
"intervalFactor": 2,
"legendFormat": "__auto",
"refId": "A",
"step": 10
}
],
"title": "Per Pod Latency (p99)",
"type": "timeseries"
}
],
"repeat": null,
@@ -221,156 +378,256 @@
"3xx": "#6ED0E0",
"4xx": "#EF843C",
"5xx": "#E24D42",
"OK": "#7EB26D",
"cancel": "#A9A9A9",
"error": "#E24D42",
"success": "#7EB26D"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 10,
"id": 3,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "1xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EAB839",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "2xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "3xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#6ED0E0",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "4xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EF843C",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "5xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "OK"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "cancel"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#A9A9A9",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "error"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "success"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
}
]
},
"lines": true,
"fill": 10,
"id": 4,
"linewidth": 0,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-z]+)\"))\n",
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-zA-Z]+)\"))\n",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{status}}",
"refId": "A",
"step": 10
"refId": "A"
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "QPS",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 5,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"targets": [
{
"expr": "histogram_quantile(0.99, sum(rate(loki_boltdb_shipper_request_duration_seconds_bucket{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval])) by (le)) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "99th Percentile",
"refId": "A",
"step": 10
"refId": "A"
},
{
"expr": "histogram_quantile(0.50, sum(rate(loki_boltdb_shipper_request_duration_seconds_bucket{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval])) by (le)) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "50th Percentile",
"refId": "B",
"step": 10
"refId": "B"
},
{
"expr": "sum(rate(loki_boltdb_shipper_request_duration_seconds_sum{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval])) * 1e3 / sum(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "Average",
"refId": "C",
"step": 10
"refId": "C"
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Latency",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"type": "timeseries",
"yaxes": [
{
"format": "ms",
@@ -389,6 +646,57 @@
"show": false
}
]
},
{
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"id": 6,
"links": [ ],
"nullPointMode": "null as zero",
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 4,
"targets": [
{
"expr": "histogram_quantile(0.99, sum(rate(loki_boltdb_shipper_request_duration_seconds_bucket{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-read\", operation=\"Shipper.Query\"}[$__rate_interval])) by (le,pod)) * 1e3",
"format": "time_series",
"interval": "1m",
"intervalFactor": 2,
"legendFormat": "__auto",
"refId": "A",
"step": 10
}
],
"title": "Per Pod Latency (p99)",
"type": "timeseries"
}
],
"repeat": null,
@@ -412,7 +720,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",

File diff suppressed because it is too large Load Diff

View File

@@ -27,589 +27,447 @@
"collapsed": false,
"panels": [
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"gridPos": { },
"id": 1,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"expr": "sum by(pod) (loki_ingester_memory_streams{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "In-memory streams",
"tooltip": {
"sort": 2
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "request"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#FFC000",
"mode": "fixed"
}
},
{
"id": "custom.fillOpacity",
"value": 0
}
]
},
{
"matcher": {
"id": "byName",
"options": "limit"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E02F44",
"mode": "fixed"
}
},
{
"id": "custom.fillOpacity",
"value": 0
}
]
}
]
},
"gridPos": { },
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [
{
"alias": "request",
"color": "#FFC000",
"fill": 0
"options": {
"legend": {
"showLegend": true
},
{
"alias": "limit",
"color": "#E02F44",
"fill": 0
"tooltip": {
"mode": "single",
"sort": "none"
}
],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
},
"targets": [
{
"expr": "sum by(pod) (rate(container_cpu_usage_seconds_total{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\"}[$__rate_interval]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
},
{
"expr": "min(kube_pod_container_resource_requests{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\", resource=\"cpu\"} > 0)",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "request",
"legendLink": null,
"step": 10
"legendLink": null
},
{
"expr": "min(container_spec_cpu_quota{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\"} / container_spec_cpu_period{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "limit",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "CPU",
"tooltip": {
"sort": 2
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "bytes"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "request"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#FFC000",
"mode": "fixed"
}
},
{
"id": "custom.fillOpacity",
"value": 0
}
]
},
{
"matcher": {
"id": "byName",
"options": "limit"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E02F44",
"mode": "fixed"
}
},
{
"id": "custom.fillOpacity",
"value": 0
}
]
}
]
},
"gridPos": { },
"id": 3,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [
{
"alias": "request",
"color": "#FFC000",
"fill": 0
"options": {
"legend": {
"showLegend": true
},
{
"alias": "limit",
"color": "#E02F44",
"fill": 0
"tooltip": {
"mode": "single",
"sort": "none"
}
],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
},
"targets": [
{
"expr": "max by(pod) (container_memory_working_set_bytes{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
},
{
"expr": "min(kube_pod_container_resource_requests{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\", resource=\"memory\"} > 0)",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "request",
"legendLink": null,
"step": 10
"legendLink": null
},
{
"expr": "min(container_spec_memory_limit_bytes{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\"} > 0)",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "limit",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Memory (workingset)",
"tooltip": {
"sort": 2
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "bytes"
},
"overrides": [ ]
},
"gridPos": { },
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"expr": "sum by(pod) (go_memstats_heap_inuse_bytes{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Memory (go heap inuse)",
"tooltip": {
"sort": 2
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 10,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "Bps"
},
"overrides": [ ]
},
"gridPos": { },
"id": 5,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 0,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": true,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"expr": "sum by(instance, pod, device) (rate(node_disk_written_bytes_total[$__rate_interval])) + ignoring(pod) group_right() (label_replace(count by(instance, pod, device) (container_fs_writes_bytes_total{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\", device!~\".*sda.*\"}), \"device\", \"$1\", \"device\", \"/dev/(.*)\") * 0)\n",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}} - {{device}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Disk Writes",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 10,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "Bps"
},
"overrides": [ ]
},
"gridPos": { },
"id": 6,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 0,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": true,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"expr": "sum by(instance, pod, device) (rate(node_disk_read_bytes_total[$__rate_interval])) + ignoring(pod) group_right() (label_replace(count by(instance, pod, device) (container_fs_writes_bytes_total{cluster=~\"$cluster\", namespace=~\"$namespace\", container=\"loki\", pod=~\"(loki|enterprise-logs)-write.*\", device!~\".*sda.*\"}), \"device\", \"$1\", \"device\", \"/dev/(.*)\") * 0)\n",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{pod}} - {{device}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Disk Reads",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "Bps",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "percentunit"
},
"overrides": [ ]
},
"gridPos": { },
"id": 7,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"expr": "max by(persistentvolumeclaim) (kubelet_volume_stats_used_bytes{cluster=~\"$cluster\", namespace=~\"$namespace\"} / kubelet_volume_stats_capacity_bytes{cluster=~\"$cluster\", namespace=~\"$namespace\"}) and count by(persistentvolumeclaim) (kube_persistentvolumeclaim_labels{cluster=~\"$cluster\", namespace=~\"$namespace\",label_name=~\"(loki|enterprise-logs)-write.*\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{persistentvolumeclaim}}",
"legendLink": null,
"step": 10
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Disk Space Utilization",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "percentunit",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
}
],
"repeat": null,
@@ -634,7 +492,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",

View File

@@ -33,116 +33,237 @@
"3xx": "#6ED0E0",
"4xx": "#EF843C",
"5xx": "#E24D42",
"OK": "#7EB26D",
"cancel": "#A9A9A9",
"error": "#E24D42",
"success": "#7EB26D"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "1xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EAB839",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "2xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "3xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#6ED0E0",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "4xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EF843C",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "5xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "OK"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "cancel"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#A9A9A9",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "error"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "success"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
}
]
},
"fill": 10,
"id": 1,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 0,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-z]+)\"))\n",
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-zA-Z]+)\"))\n",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{status}}",
"refId": "A",
"step": 10
"refId": "A"
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "QPS",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": false
}
]
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 2,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "histogram_quantile(0.99, sum by (le) (cluster_job:loki_request_duration_seconds_bucket:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"})) * 1e3",
"expr": "histogram_quantile(0.99, sum by (le) (cluster_job_route:loki_request_duration_seconds_bucket:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"})) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "99th Percentile",
@@ -150,7 +271,7 @@
"step": 10
},
{
"expr": "histogram_quantile(0.50, sum by (le) (cluster_job:loki_request_duration_seconds_bucket:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"})) * 1e3",
"expr": "histogram_quantile(0.50, sum by (le) (cluster_job_route:loki_request_duration_seconds_bucket:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"})) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "50th Percentile",
@@ -158,7 +279,7 @@
"step": 10
},
{
"expr": "1e3 * sum(cluster_job:loki_request_duration_seconds_sum:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"}) / sum(cluster_job:loki_request_duration_seconds_count:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\"})",
"expr": "1e3 * sum(cluster_job_route:loki_request_duration_seconds_sum:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"}) / sum(cluster_job_route:loki_request_duration_seconds_count:sum_rate{cluster=~\"$cluster\", job=~\"($namespace)/(loki|enterprise-logs)-write\", route=~\"api_prom_push|loki_api_v1_push|/httpgrpc.HTTP/Handle\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "Average",
@@ -166,23 +287,8 @@
"step": 10
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Latency",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"type": "timeseries",
"yaxes": [
{
"format": "ms",
@@ -215,76 +321,105 @@
"height": "250px",
"panels": [
{
"aliasColors": {
"1xx": "#EAB839",
"2xx": "#7EB26D",
"3xx": "#6ED0E0",
"4xx": "#EF843C",
"5xx": "#E24D42",
"error": "#E24D42",
"success": "#7EB26D"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 10,
"id": 3,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 0,
"id": 3,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": true,
"steppedLine": false,
"targets": [
{
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-z]+)\"))\n",
"expr": "sum (rate(loki_distributor_structured_metadata_bytes_received_total{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\",}[$__rate_interval])) / sum(rate(loki_distributor_bytes_received_total{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\",}[$__rate_interval]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "{{status}}",
"refId": "A",
"step": 10
"legendFormat": "bytes",
"legendLink": null
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "QPS",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
"title": "Per Total Received Bytes",
"type": "timeseries"
},
{
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [ ]
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
"id": 4,
"links": [ ],
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": true,
"targets": [
{
"expr": "sum by (tenant) (rate(loki_distributor_structured_metadata_bytes_received_total{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\",}[$__rate_interval])) / ignoring(tenant) group_left sum(rate(loki_distributor_structured_metadata_bytes_received_total{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\",}[$__rate_interval]))",
"format": "time_series",
"legendFormat": "{{tenant}}",
"legendLink": null
}
],
"title": "Per Tenant",
"type": "timeseries",
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"max": 1,
"min": 0,
"show": true
},
@@ -292,85 +427,281 @@
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"max": 1,
"min": null,
"show": false
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": true,
"title": "Write Path",
"titleSize": "h6"
},
{
"collapse": false,
"height": "250px",
"panels": [
{
"aliasColors": {
"1xx": "#EAB839",
"2xx": "#7EB26D",
"3xx": "#6ED0E0",
"4xx": "#EF843C",
"5xx": "#E24D42",
"OK": "#7EB26D",
"cancel": "#A9A9A9",
"error": "#E24D42",
"success": "#7EB26D"
},
"datasource": "$datasource",
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 100,
"lineWidth": 0,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "normal"
}
},
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "short"
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "1xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EAB839",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "2xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "3xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#6ED0E0",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "4xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#EF843C",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "5xx"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "OK"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "cancel"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#A9A9A9",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "error"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#E24D42",
"mode": "fixed"
}
}
]
},
{
"matcher": {
"id": "byName",
"options": "success"
},
"properties": [
{
"id": "color",
"value": {
"fixedColor": "#7EB26D",
"mode": "fixed"
}
}
]
}
]
},
"fill": 10,
"id": 5,
"linewidth": 0,
"links": [ ],
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": true,
"targets": [
{
"expr": "sum by (status) (\n label_replace(label_replace(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval]),\n \"status\", \"${1}xx\", \"status_code\", \"([0-9])..\"),\n \"status\", \"${1}\", \"status_code\", \"([a-zA-Z]+)\"))\n",
"format": "time_series",
"legendFormat": "{{status}}",
"refId": "A"
}
],
"title": "QPS",
"type": "timeseries"
},
{
"aliasColors": { },
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"id": 4,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
"fieldConfig": {
"defaults": {
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 1,
"pointSize": 5,
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
}
},
"thresholds": {
"mode": "absolute",
"steps": [ ]
},
"unit": "ms"
},
"overrides": [ ]
},
"lines": true,
"linewidth": 1,
"id": 6,
"links": [ ],
"nullPointMode": "null as zero",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [ ],
"spaceLength": 10,
"options": {
"legend": {
"showLegend": true
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "histogram_quantile(0.99, sum(rate(loki_boltdb_shipper_request_duration_seconds_bucket{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval])) by (le)) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "99th Percentile",
"refId": "A",
"step": 10
"refId": "A"
},
{
"expr": "histogram_quantile(0.50, sum(rate(loki_boltdb_shipper_request_duration_seconds_bucket{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval])) by (le)) * 1e3",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "50th Percentile",
"refId": "B",
"step": 10
"refId": "B"
},
{
"expr": "sum(rate(loki_boltdb_shipper_request_duration_seconds_sum{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval])) * 1e3 / sum(rate(loki_boltdb_shipper_request_duration_seconds_count{cluster=~\"$cluster\",job=~\"($namespace)/(loki|enterprise-logs)-write\", operation=\"WRITE\"}[$__rate_interval]))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "Average",
"refId": "C",
"step": 10
"refId": "C"
}
],
"thresholds": [ ],
"timeFrom": null,
"timeShift": null,
"title": "Latency",
"tooltip": {
"shared": true,
"sort": 2,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [ ]
},
"type": "timeseries",
"yaxes": [
{
"format": "ms",
@@ -412,7 +743,7 @@
"value": "default"
},
"hide": 0,
"label": "Data Source",
"label": "Data source",
"name": "datasource",
"options": [ ],
"query": "prometheus",

View File

@@ -93,6 +93,12 @@ metrics:
- container_spec_cpu_quota
- container_spec_memory_limit_bytes
- cortex_ingester_flush_queue_length
- cortex_prometheus_rule_group_iterations_total
- cortex_prometheus_rule_evaluation_failures_total
- cortex_prometheus_rule_group_rules
- cortex_prometheus_rule_group_last_duration_seconds
- cortex_prometheus_rule_group_last_evaluation_timestamp_seconds
- cortex_prometheus_rule_group_iterations_missed_total
- go_gc_duration_seconds
- go_goroutines
- go_memstats_heap_inuse_bytes
@@ -111,11 +117,15 @@ metrics:
- loki_boltdb_shipper_retention_marker_table_processed_total
- loki_boltdb_shipper_request_duration_seconds_bucket
- loki_boltdb_shipper_request_duration_seconds_count
- loki_boltdb_shipper_request_duration_seconds_sum
- loki_boltdb_shipper_retention_sweeper_chunk_deleted_duration_seconds_bucket
- loki_boltdb_shipper_retention_sweeper_chunk_deleted_duration_seconds_count
- loki_boltdb_shipper_retention_sweeper_chunk_deleted_duration_seconds_sum
- loki_boltdb_shipper_retention_sweeper_marker_files_current
- loki_boltdb_shipper_retention_sweeper_marker_file_processing_current_time
- loki_build_info
- loki_chunk_store_deduped_chunks_total
- loki_chunk_store_index_entries_per_chunk_bucket
- loki_chunk_store_index_entries_per_chunk_count
- loki_chunk_store_index_entries_per_chunk_sum
- loki_compactor_delete_requests_processed_total
@@ -123,7 +133,10 @@ metrics:
- loki_compactor_deleted_lines
- loki_compactor_oldest_pending_delete_request_age_seconds
- loki_compactor_pending_delete_requests_count
- loki_discarded_samples_total
- loki_distributor_bytes_received_total
- loki_distributor_lines_received_total
- loki_distributor_structured_metadata_bytes_received_total
- loki_ingester_chunk_age_seconds_bucket
- loki_ingester_chunk_age_seconds_count
- loki_ingester_chunk_age_seconds_sum
@@ -137,9 +150,13 @@ metrics:
- loki_ingester_chunk_utilization_bucket
- loki_ingester_chunk_utilization_sum
- loki_ingester_chunks_flushed_total
- loki_ingester_flush_queue_length
- loki_ingester_memory_chunks
- loki_ingester_memory_streams
- loki_ingester_streams_created_total
- loki_request_duration_seconds_bucket
- loki_request_duration_seconds_count
- loki_request_duration_seconds_sum
- loki_ruler_wal_appender_ready
- loki_ruler_wal_disk_size
- loki_ruler_wal_prometheus_remote_storage_highest_timestamp_in_seconds
@@ -185,6 +202,15 @@ kubeStateMetrics:
loki:
loki:
auth_enabled: false
schemaConfig:
configs:
- from: 2024-03-29
store: tsdb
object_store: s3
schema: v13
index:
prefix: index_
period: 24h
storage:
type: "s3"
s3:
@@ -202,8 +228,13 @@ loki:
secret_access_key: "{{ .Values.global.minio.rootPassword }}"
compactor:
retention_enabled: true
delete_request_store: s3
limits_config:
retention_period: 30d
lokiCanary:
enabled: false
test:
enabled: false
monitoring:
dashboards:
enabled: false
@@ -220,8 +251,8 @@ loki:
test:
enabled: false
grafana-agent:
agent:
alloy:
alloy:
clustering:
enabled: true
configMap: