Nova Resource:Deployment-prep/Log Files

From Wikitech

When there are issues with Beta Cluster, looking through the log files that are produced can be very useful. The log files are located in /srv/mw-log on deployment-fluorine.eqiad.wmflabs.

In the log directory you will find different log files, most of them are generated by MediaWiki sending message over UDP to a receiver (udp2log) on deployment-bastion.eqiad.wmflabs instance. Additionally we log PHP fatal errors (under fatal.log) and Apache errors (under apache2.log) and access logs (under apache-access.log).

Wikimedia staff can access logstash and/or a https://logstash-beta.wmflabs.org (credentials are on the office wiki here).

Note bug 72275 that fatals on the front-end web server machines (deployment-mediawiki01, 02, ...) are not appearing in /data/project/logs and logstash. As a workaround, look at the local HHVM errors on these machines in /var/log/hhvm/error.log.