IMAPClient 0.6.1 released

I've just released IMAPClient 0.6.1.

The only functional change in the release is that it now automatically patches imaplib's IMAP4_SSL class to fix Python Issue 5949. This is a bug that's been fixed in later Python 2.6 versions and 2.7 but still exists in Python versions that are in common use. Without fix this you may experience hangs when using SSL.

The patch is only applied if the running Python version is known to be one of the affected versions. It is applied when IMAPClient is imported.

The only other change in this release is that I've now marked IMAPClient as "production ready" on PyPI and have updated the README to match. This was prompted by a request to clarify the current status of the project and seeing that all current functionality is solid and, I don't plan to change the existing APIs in backwards-incompatible ways, I've decided to indicate the project as suitable for production use.

As always, IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site. Feedback, bug reports and patches are most welcome.

IMAPClient 0.6 released

IMAPClient 0.6 is finally out! Highlights:

  • Completely new response parsing code. Complex response items like BODYSTRUCTURE and ENVELOPE are now handled properly and interaction with the Gmail and MS Exchange IMAP implementations works correctly.
  • Support for the COPY command
  • Support for the XLIST extension (used by Gmail)
  • select_folder() now returns a parsed response containing all reported details about the selected folder.
  • the return value from list_folders(), list_sub_folders() and xlist_folders() now include the IMAP folder flags and folder delimiter.
  • Handling of internationalised folder names has been cleaned up. Folder names now are always returned as unicode strings.
  • Many bug fixes.

Be aware that there have been several API changes with this release. See the NEWS file for further details.

IMAPClient can be installed from PyPI (pip install imapclient) or downloaded from the IMAPClient site. As always, feedback, bug reports and patches are most welcome.

Special thanks goes to Mark Hammond. He has contributed a significant amount of code and effort to this release. Incidentally, Mark is using IMAPClient as part of the backend for the Raindrop Mozilla Labs project.

TypeError: object.__init__() takes no parameters

At my employer we are in the process of migrating from Python 2.4 to 2.6. When running some existing code under Python 2.6 we started getting DeprecationWarnings about "object.__new__() takes no parameters" and "object.__init__() takes no parameters".

A simple example that triggers the warning:

class MyClass(object):

    def __new__(cls, a, b):
        print 'MyClass.__new__', a, b
        return super(MyClass, cls).__new__(cls, a, b)

    def __init__(self, a, b):
        print 'MyClass.__init__', a, b
        super(MyClass, self).__init__(a, b)

obj = MyClass(6, 7)

This gives:

$ python2.4
MyClass.__new__ 6 7
MyClass.__init__ 6 7

$ python2.6
MyClass.__new__ 6 7 DeprecationWarning: object.__new__() takes no parameters
  return super(MyClass, cls).__new__(cls, a, b)
MyClass.__init__ 6 7 DeprecationWarning: object.__init__() takes no parameters
  super(MyClass, self).__init__(a, b)

It turns out that a change to Python for 2.6 (and 3) means that object.__new__ and object.__init__ no longer take arguments - a TypeError is raised when arguments are passed. To avoid breaking too much pre-existing code, there is a special case that will cause a DeprecationWarning instead of TypeError if both __init__ and __new__ are overridden. This is the case we were running into with our code at work.

The reason for this change seems to make enough sense: object doesn't do anything with arguments to __init__ and __new__ so it shouldn't accept them. Raising an error when arguments are passed highlights code where the code might be doing the wrong thing.

Unfortunately this change also breaks Python's multiple inheritance in a fairly serious way when cooperative super calls are used. Looking at the ticket for this change, this issue was thought about but perhaps the implications were not fully understood. Given that using super with multiple inheritance is common and "correct" practice, it would seem that this change to Python is a step backwards.

Read more…

rst_break plugin for PyBlosxom

I just scratched an itch by writing a small plugin for PyBlosxom that allows the rst (reStructured Text) and readmore plugins to work together [1]. It defines a reST "break" directive which gets transformed into the breakpoint string the readmore plugin looks out for. This allows for "Read more..." breaks to be inserted in for reST based articles.

For further information see the Code page here and at the top of the plugin itself.

[1] Yes, the audience for this plugin is probably tiny!

A little thing about cron

Here's something I just learned the hard way.

If you edit a crontab with "crontab -e", cron won't reload the updated crontab immediately. Changes will be read at 1 second past the next minute boundary. For example, if you change the crontab at 10:54:32, cron will reload it at 10:55:01. This means if you're trying to test how something runs under cron and you're impatient so you set that thing to run at the next minute, you won't see it run!

I spent a good half hour chasing my tail on this one. Set the test entry to run 2 minutes ahead instead.

IMAPClient Trac instance now allows for user registrations

I've had several requests over the last few weeks to open up the IMAPClient Trac instance so that anyone can submit tickets. Initially I changed access levels so that anoymous users could create tickets. This turned out to be fairly inflexible: it doesn't allow people to add attachments or modify tickets later. This approach also resulted in one strange ticket being created where all fields were filled with random characters - a bot looking for buffer overruns?

Since then, I've disabled anonymous ticket creation and have set up the fantastic AccountManagerPlugin which allows people to register accounts for themselves. Once someone has created an account and logged in they can create and modify tickets. I have a feeling I'm going to have to turn on the optional CAPTCHA support, but I'm willing to see how it goes for a while first.

IMAPClient 0.5.1

I've just made a quick point release of IMAPClient. Mark Hammond is interested in using it for a project he's working on but the licenses (GPL and MPL) were incompatible. I was thinking about relaxing the license of IMAPClient anyway so this presented a good opportunity to make the switch.

Work on the 0.6 release is coming along. This version will fix a number issues with parsing of FETCH responses - the FETCH parsing code is being completely rewritten. This is the first time that IMAPClient will bypass most of imaplib for some functionality. It's looking like that at some point IMAPClient may not use imaplib at all.

IMAPClient can be installed from PyPI using pip install IMAPClient or easy_install IMAPClient. It can also be downloaded from the IMAPClient project page. As always, feedback and patches are most welcome.

Setting PYTHON_EGG_CACHE when deploying Python apps using FastCGI

I recently sorted out an issue with the IMAPClient Trac instance that's been bugging me for a while.

The problem was that whenever the web server logs were rotated logrotate would restart Lighttpd. The web server restart would in turn restart the Trac (FastCGI) processes. Unfortunately, the Trac processes would fail to start with the following error.

pkg_resources.ExtractionError: Can't extract file(s) to egg cache

The following error occurred while trying to extract file(s) to the Python egg

  [Errno 13] Permission denied: '/root/.python-eggs'

The Python egg cache directory is currently set to:


Bang, no IMAPClient web site (the rest of the site was ok). To band-aid the problem when it happened (and I noticed!) I issue a sudo /etc/init.d/lighttpd restart and everything would be fine again.

After some investigation I found that running /etc/init.d/lighttpd restart as root always triggered the problem where-as restarting using sudo always worked. My guess is that restarting when logged in as root was leaving $HOME at /root even after Lighttpd had dropped to its unprivileged user account. The unprivileged user isn't allowed to write to /root so Trac blows up. setuptools seems to use $HOME instead of looking up the actual home directory of the current user.

The fix for me was to set the PYTHON_EGG_CACHE environment variable for the FastCGI processes to somewhere they are allowed to write to. This is done with the bin-environment option if you're using Lighttpd like me.

I imagine similar problems can happen with any Python app deployed using FastCGI.