grafana.wikimedia.org
grafana.wikimedia.org is a frontend for creating queries and storing dashboards using data from Graphite.
Service
Currently hosted on and served from krypton.
Viewing
Most viewing features can be discovered naturally, but here's a few features you may not immediately realise exist:
- Dynamic time range, you can zoom in and focus on any portion of a plot by selecting and dragging within the graph.
- Metrics, you can click on metric names in the graph legend to isolate a single metric, or ctrl/cmd click to exclude a metric.
- Annotations, such as for deployments, can be toggled by clicking the lightning icon on the top left.
Template
- Go to https://grafana-admin.wikimedia.org/dashboard/db/template-dashboard (or import from grafana.wikimedia.org/template-dashboard.json in case it was accidentally overwritten or deleted)
- Click on the gear icon on the top left and use "Save As..." to create a copy under a different name.
- Make your own graphs!
This template incorporates features we usually want but would otherwise have to be configured manually. Such as:
- The "Show deployments" annotation.
- Auto-refresh every minute.
- Timepicker to include a "6 months" and "1 year" entry.
- UTC timezone.
- Shared crosshair, shared tooltip.
Features
To enable the shared crosshair (which draws the current target cursor in all graphs on the page), go to "Configure dashboard" (top right menu). Then tick the "Shared Crosshair" setting in the Features section.
By default each data point of each metric has its own tooltip, only shown when hovering the exact point. Consider enabling the "All series" tooltip. This will ensure the tooltip is always shown when inside the graph. All points on the vertical axis are shown in a single tooltip at the same same. Horizontally the closest data point will be shown in the tooltip.
- Click on the graph title and select Edit.
- In the Display Styles section, enable tooltip "All series".
- From the top navigation, go back to the dashboard.
Show deployments
Add MediaWiki deployment events as annotations to your dashboard:
- Enable the "Annotations" feature from the "Configure dashboard" panel (top right menu).
- From new settings menu on the top left, choose Annotations.
- Add a new annotation.
- Name: Show deployments
- Data source: graphite
- Graphite target expression:
exclude(aliasByNode(deploy.*.count,-2),"all")
- Click Update, and close the settings menu.
Input variables
See Grafana Templated dashboards.
Time correction
From "Configure dashboard" (top right menu) one can change the default ("browser time") to use "UTC" instead.
Negative axis
If you're plotting metrics that contain negative values, you can use the "Transform: negative-Y" series override to flip the values in the graph, whilst preserving the positive values for min/max legends. You can use this instead of modifications like scale(-1). For example, plotting download traffic in a graph about upload and download bandwidth.[1] Example: Server board - Network traffic.
Known issues
Month duration
Configuring a relative time duration as "month" doesn't work as expected. In Graphite the unit for months is mon[2]. In Grafana is it "M", e.g. use "7M" for the last 7 months.
movingAverage template
When using a template inside movingAverage, the default mode wrongly expands the variable (it adds quotes around it, instead of leaving it as a number). These have to be removed manually by editing the metric directly (click on the pencil). Whenever the metric is changed, it has to be fixed again.
Color inspector broken
The UI is unable to save changes to color values. This can be worked around by editing the Panel JSON directly.
Recovering from toggling dashboard to not be editable
Delete the dashboard via the API, then restore it without the toggle.
No history of dashboard changes
While the sqlite db grafana uses as storage for dashboards is backed up, there is no real history of changes. Some people use Grafana.wikimedia.org/ as a dump for manual backups of dashboard JSON files. One could run grafcli or something using the grafana HTTP API with git in a cron to have a better history. There is grafana::dashboard to define dashboards in puppet.
External links
- https://grafana.wikimedia.org
- http://grafana.org/
- http://docs.grafana.org/guides/whats-new-in-v2/
- http://docs.grafana.org/guides/whats-new-in-v2-1/
- ↑ Negative y-tranform, What's new in Grafana v2.1
- ↑ Render API, Graphite documentation