Refinery-source is the JVM software on the Analytics Cluster using spark or hive UDFs. The source code is in the analytics/refinery/source.

How to deploy

Warning:

You need archiva passwords in order to follow that procedure. Ask Andrew if needed.

Update the changelog.md file at the root of the repository whith changes that are going to be deployed.
Prepare deployment (change pom.xml files, push to git):

mvn -Duser.name=YOUR_WIKITECH_USERNAME release:prepare

cat release.properties

mvn -Duser.name=YOUR_WIKITECH_USERNAME release:perform

Download the new version of the jars from archiva (
Warning: git fat uses IDs to manage files, so it is important to use the correct jars.
) For convenience, the commonly updated jars are:
NOTE: to download, click on the new version you just created, then Artifacts, then copy the link to the jar and update 127.0.0.1:8080 with archiva.wikimedia.org and download that
Copy the updated jars to the correct refinery path (somewhere like <refinery>/artifacts/org/wikimedia/analytics/refinery/)
Update the symlinks in refinery/artifacts to the jars you just copied
Make sure git fat is installed and configured according to the instructions in the Refinery README
git add . && git commit (git fat will do some magic and replace the jar with a one-line id)
push for review

Please see the refinery page to deploy the jars and oozie code.