Webperf

From Wikitech

webperf is a set of scripts that aggregate/extract (typically EventLogging) data and send it to statsd.

Setup

Sources are directly in puppet in the webperf module.

The corresponding role is in manifests/role/webperf.pp.

Code gets deployed directly through puppet to hafnium.

Each script is a separate Upstart service.

Metrics

asset-check

The asset-check service loads the some pages (currently 9) from production wikis every 5 minutes and extracts information about loaded CSS, JavaScript, HTML, ... resources through PhantomJS.

The extracted information gets pumped into frontend.assets in statsd. A key that currently (2014-12-22) always has hits, and can hence used to see if it is up and running is frontend.assets.html.bytes.enwiki.rate. (Note that it is inherently spike-y at low time resolutions, due to getting the data every 5 minutes.)

navtiming

The navtiming service extracts information for the NavigationTiming and SaveTiming schemas from EventLogging.

NavigationTiming is reported to frontend.navtiming in statsd. To see if it's running properly, the frontend.navtiming.dnsLookup.overall.rate key always has hits (as of 2014-12-22).

SaveTiming get reported to mw.performance.save in statsd. To see if it's running properly, the mw.performance.save.HHVM.rate key always has hits (as of 2014-12-22).

mw-js-deprecate

The statsd-mw-js-deprecate service extracts information for the DeprecatedUsage schema from EventLogging. It is then reported to mw.js.deprecate in statsd. A subkey that currently always has hits (as of 2015-06-12) is addOnloadHook.

ve

The ve service extracts information for the TimingData schema from EventLogging.

Data gets reported to whatever the event's points field holds, but typically underneath ve.mwtiming. in statsd. A key that currently (2014-12-22) always has hits, and can hence used to see if it is up and running is ve.mwtiming.performance.system.domLoad.

See also