Ingest relevant OBS and annotation data to generate insightful metrics.
The overall structure is to loop over all requests to extract points of change and calculate metrics. Many of the points of change are treated as deltas, meaning to add one to this bucket or minus one from another. After all requests are ingested the points are walked to evaluate the deltas and recreate the state at that point in time.
OBS provides incomplete and inconsistent data which causes odd results like negative backlog counts. The ingest tool implements a number of workarounds to figure out the correct values when possible, but when not possible the provided values are used rather than excluding a fairly large chunk of data. The main issues are documented in:
grafana.ini:
[dashboards.json].enabled = true to use the dashboards provided by rpmopenSUSE:Factory)See help information for InfluxDB connection flags.
./metrics -p openSUSE:Factory
Once completed the Grafana dashboard should make pretty graphs.
Grafana provides an export to JSON option which can be used when the dashboards are modified to export them and version control the changes in git. Ensure you do not unintentionally change the default project, annotation state, or time period by saving the dashboard with different defaults.
Use the --debug option and inspect individual request XML dumps by looking in
~/.cache/openSUSE-release-tools/request/metrics or:
osc api '/request/$reqid?withfullhistory=1'
When adding new delta based metrics it may be necessary to add key logic in
walk_points() to handle proper grouping for evaluation of deltas.