PyBal

PyBal is a LVS monitoring script. It's written in Python using the Twisted framework.

For more information about Wikimedia's LVS setup in general, see LVS.

At this moment, just a few features distinguish it from lvsmon:

It's using asynchronous communication, and thus runs all checks in parallel instead of sequentially
It has an extra monitoring method called IdleConnection, which keeps an idle connection open to all squids, and therefore notices immediately when the Squid processes are shut down / crashing
It can fetch server lists over HTTP as well as from the local filesystems

...but I intend to polish it more, and extend it with useful things.

The script is in Wikimedia's Git repo "operations/debs", subdirectory pybal.

-- Mark

Setup

PyBal is currently installed on our LVS hosts, in directory /usr/sbin. Start or stop it by

 /etc/init.d/pybal start|stop

Configuration is in /etc/pybal/. pybal.conf defines the LVS service parameters

The list of pooled hosts resides wherever the pybal::web class is installed via puppet, under the directory /srv/pybal-config and it will be reachable via the internal address http://configuration-master.$site.wmnet/pybal, with one file per LVS service. Attributes:

weight: a larger number means that more requests get sent to this server in comparison with others
enabled: either True or False, depending on whether you want requests to be sent to this server

The format should be fairly self explanatory; the files more or less use Python assignment / dictionary syntax.

PyBal supports multiple LVS services through a single instance and configuration file pybal.conf, e.g.:

[text]
protocol = tcp
ip = 145.97.39.155
port = 80
scheduler = wlc
config = file:///etc/pybal/text-squids

[images]
protocol = tcp
ip = 145.97.39.156
port = 80
scheduler = wlc
config = file:///etc/pybal/upload-squids

Beware, the code as checked out from git has DryRun = True set in ipvs.py, meaning that it will not modify any actual IPVS state but only show the commands for debugging. This should be changed to a command line option, but for now edit that file to DryRun = False.

The configuration files are generated via puppet.

How to

See LVS.

Setup

How to

See also