Webperf
webperf is a set of scripts that aggregate/extract (typically EventLogging) data and send it to statsd.
Setup
Sources are directly in puppet in the webperf module.
The corresponding role is in manifests/role/webperf.pp.
Code gets deployed directly through puppet to hafnium.
Each script is a separate Upstart service.
Metrics
asset-check
The asset-check
service loads the some pages (currently 9) from production wikis every 5 minutes and extracts information about loaded CSS, JavaScript, HTML, ... resources through PhantomJS.
The extracted information gets pumped into frontend.assets
in statsd. A key that currently (2014-12-22) always has hits, and can hence used to see if it is up and running is frontend.assets.html.bytes.enwiki.rate
. (Note that it is inherently spike-y at low time resolutions, due to getting the data every 5 minutes.)
The navtiming
service extracts information for the NavigationTiming and SaveTiming schemas from EventLogging.
NavigationTiming is reported to frontend.navtiming
in statsd. To see if it's running properly, the frontend.navtiming.dnsLookup.overall.rate
key always has hits (as of 2014-12-22).
SaveTiming get reported to mw.performance.save
in statsd. To see if it's running properly, the mw.performance.save.HHVM.rate
key always has hits (as of 2014-12-22).
mw-js-deprecate
The statsd-mw-js-deprecate
service extracts information for the DeprecatedUsage schema from EventLogging. It is then reported to mw.js.deprecate
in statsd. A subkey that currently always has hits (as of 2015-06-12) is addOnloadHook
.
ve
The ve
service extracts information for the TimingData schema from EventLogging.
Data gets reported to whatever the event's points
field holds, but typically underneath ve.mwtiming.
in statsd. A key that currently (2014-12-22) always has hits, and can hence used to see if it is up and running is ve.mwtiming.performance.system.domLoad
.