For phabricator admins

Updating the source from upstream

Updating phabricator (including arcanist) from upstream is a bit tricky, since we use 'fake repositories' to have phabricator in the internal-webserver repository.

Setup

Start by cloning the internal webserver repository from github to some spot on your local computer by running the following incantations that totally work.
git clone git@github.com:Khan/internal-webserver
cd internal-webserver
for dir in phabricator libphutil arcanist python-phabricator; do
  rm -rf $dir && git clone git@github.com:Khan/$dir && cd $dir && git remote add upstream http://github.com/facebook/$dir && cd -;
done

Now git status (in the internal-webserver directory) should show a clean client.

Updates

To update our version of phabricator with the latest changes from upstream, run ./update_phabricator.sh from the internal-webserver directory.
If the pull commands show merges or other problems, resolve them manually before continuing with this script! How will you know? The script will stop after  showing you a git status and then ask you

Does everything look ok? (y/N)

if you affirm this, the script will then proceed to push all the changes to phabricator.khanacademy.org and then it will stop all the phab daemons and _then_ it will restart them. the last sixty or so lines of output will let you know if anything has gone disastrously wrong as a result of the update process (i.e. lighty not waking up, the phab daemons going on strike, etc). In all likelihood, it looks like

.
.
.
Launching 'PhabricatorRepositoryPullLocalDaemon'...
Launching 'PhabricatorGarbageCollectorDaemon'...
Launching 'PhabricatorTaskmasterDaemon'...
Launching 'PhabricatorTaskmasterDaemon'...
Launching 'PhabricatorTaskmasterDaemon'...
Launching 'PhabricatorTaskmasterDaemon'...
Done.
DONE!
Already on 'master'

ssh to toby/phabricator, look at /var/tmp/phd/log/daemons.log for anything weird, and go to bed.

Handling error conditions

Errors in the daemon logs

The daemon logs live on the internal-webserver machine (toby) at /var/tmp/phd/log/daemons.log. If the logs show recent messages of the form

EXCEPTION: (Exception) Attempting to archive a task which hasn't been save()d! at [/home/ubuntu/internal-webserver/phabricator/src/infrastructure/daemon/workers/storage/PhabricatorWorkerActiveTask.php:75]
you can fix it by running:
sudo mysql phabricator_worker -e 'truncate table worker_archivetask;' && cd ~/internal-webserver && PHABRICATOR_ENV=khan phabricator/bin/phd restart


Common tasks

Deleting audits

Sometimes if you have to re-create a repo on phabricator, or create a branch, phabricator will re-create a bunch of audits associated with that repo.  You can delete them manually, though the tool is pretty crude:

e.g. Delete everything before Sept 5:
internal_webserver/phabricator$ ./bin/config set phabricator.base-uri http://phabricator.khanacademy.org/
internal_webserver/phabricator$ ./bin/audit delete --max-commit-date="5 sep 2013" --repositories WA --dry-run

Restarting the phabricator daemons

Sometimes a phabricator deamon gets stuck on a git pull -- this happens a lot when github is having trouble -- and you need to kick it to get it started again.  Here is the command you can run to do that (from your local machine no less!)  Note it is always safe to restart daemons; they were written to be interruptable.

ssh ubuntu@phabricator.khanacademy.org -i "$HOME/.ssh/internal_webserver.pem" "\
   cd internal-webserver; \
   git checkout master; \
   git pull; \
   PHABRICATOR_ENV=khan phabricator/bin/phd stop; \
   sudo /etc/init.d/lighttpd stop; \
   PHABRICATOR_ENV=khan phabricator/bin/storage upgrade --force; \
   sudo /etc/init.d/lighttpd start; \
   PHABRICATOR_ENV=khan phabricator/bin/phd start; \
   "

You may need to tweak this depending on what you called your .pem file.  Alternately, you can just log into the phabricator machine and run the stuff within the ssh.

This command also make sure the phabricator machine is up to date with code and storage.  If you want to skip those parts, just run the cd and the two phd commands.

This command is taken verbatim from the end of update_phabricator.sh.

Comments