Nova Resource:Tools/Admin/emergency guides/single tool webservice

From Wikitech

During the june 2015 NFS outage, we did an emergency deployment of the demw wiki loves earth jury tool on toolsbeta. This is a step-for-step guide what we did at that point, although some intermediate steps might be missing.

1) Find a suitable host.

  • We chose a host on toollabs because it had on-line instances that could be deleted safely afterwards (spinning up new instances took an hour or more).
  • We chose a host with a suitable Security Group setup -- with port 8080 open to 10.0.0.0/8.
  • We chose a host with a running web service setup so we could have an environment akin to the existing toollabs one

Best host: toolsbeta-webgrid-04

2) Disable NFS and restart host (human intervention by Yuvi)

3) create basic structure in /data/project: 

     /data/project/jury
     /data/project/jury/public_html (containing salvaged code)
     /data/project/jury/replica.my.cnf

4) hack around the safeguards in lighttpd-starter

  • lighttpd-starter checks whether the server is allowed to start. It also tries to write in places where we don't have permission to write
--- /usr/local/bin/lighttpd-starter     2015-04-05 06:13:33.878601295 +0000
+++ lighttpd-starter   2015-07-03 21:45:21.945025980 +0000
@@ -23,14 +23,10 @@
 
 tool="$1"
 port="$2"
-prefix=$(/bin/cat /etc/wmflabs-project)
+prefix=tools #$(/bin/cat /etc/wmflabs-project)
 user="$prefix.$tool"
-home=$(getent passwd $user | cut -d : -f 6 | sed -e's/\/$//')
-if [ "$(getent group $user | cut -d : -f 1)" !="$user" ]; then
-  echo "$0: $1does not appear to be a tool" >&2
-  exit 1
-fi
-spool="/var/run/lighttpd"
+home=/data/project/jury #$(getent passwd $user | cut -d :-f 6 | sed -e 's/\/$//')
+spool="/data/project/jury/lighttpd"
 runbase="$spool/$tool"
 
 if ["$home" = "" -o ! -d "$home/public_html" ]; then

5) start the webserver in a screen 

sudo su tools.jury
script /dev/null
screen
./lighttpd-starter jury 8080

6) Test if the webservice is running (ssh tunnel)

7) Disable webservice on toollabs (requires root login)

  • stop webservice, make sure it stays stopped (no service.manifest, but that's hard without NFS)

8) set up proxy to redirect to temporary host

9) test on tools.wmflabs.org

Retrieved from "https://labtestwikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools/Admin/emergency_guides/single_tool_webservice&oldid=15824"