Setting PYTHON_EGG_CACHE when deploying Python apps using FastCGI
Posted on
I recently sorted out an issue with the IMAPClient Trac instance that's been bugging me for a while.
The problem was that whenever the web server logs were rotated logrotate would restart Lighttpd. The web server restart would in turn restart the Trac (FastCGI) processes. Unfortunately, the Trac processes would fail to start with the following error.
pkg_resources.ExtractionError: Can't extract file(s) to egg cache
The following error occurred while trying to extract file(s) to the Python egg
cache:
[Errno 13] Permission denied: '/root/.python-eggs'
The Python egg cache directory is currently set to:
/root/.python-eggs
Bang, no IMAPClient web site (the
rest of the site was ok). To band-aid the problem when it happened (and
I noticed!) I issue a sudo /etc/init.d/lighttpd restart and everything
would be fine again.
After some investigation I found that running
/etc/init.d/lighttpd restart as root always triggered the problem
where-as restarting using sudo always worked. My guess is that
restarting when logged in as root was leaving $HOME at /root even after
Lighttpd had dropped to its unprivileged user account. The unprivileged
user isn't allowed to write to /root so Trac blows up. setuptools seems
to use $HOME instead of looking up the actual home directory of the
current user.
The fix for me was to set the PYTHON_EGG_CACHE environment variable for the FastCGI processes to somewhere they are allowed to write to. This is done with the bin-environment option if you're using Lighttpd like me.
I imagine similar problems can happen with any Python app deployed using FastCGI.