IMAPClient 2.1.0

IMAPClient 2.1.0 has just been released! Here's the main highlights:

  • Python 3.7 is now officially suppported
  • Testing against PyPy (version 2 and 3)
  • Added support for the QUOTA extension
  • Helper for locating special folders
  • Document usage for using self-signed TLS certificates
  • Document how to use the email package from the standard library to parse fetched emails
  • Handling NIL values for INTERNALDATE

As always, IMAPClient can be installed using pip (pip install imapclient). Full documentation is available at Read the Docs.

Enjoy!

Listing S3 objects with NodeJS

I recently had to write some NodeJS code which uses the AWS SDK to list all the objects in a S3 bucket which potentially contains many objects (currently over 80,000 in production). The S3 listObjects API will only return up to 1,000 keys at a time so you have to make multiple calls, setting the Marker field to page through all the keys.

It turns out there's a lot of sub-optimal examples out there for how to do this which often involve global state and complicated recursive callbacks. I'm also a fan of the clarity of JavaScript's newer async/await feature for handling asynchronous code so I was keen on a solution which uses that style.

Read more…

IMAPClient 1.1.0

IMAPClient 1.1.0 has just been released! Many thanks to the recent contributors and the project's other maintainers, Nicolas Le Manchet and Maxime Lorant. This release is full of great stuff because of them. Here's some highlights:

  • search now supports nested criteria so that more complex criteria can be expressed. IMAPClient will add parentheses in the right place. See
  • PLAIN authentication support
  • UNSELECT support
  • ENABLE support
  • UID EXPUNGE support
  • IMAP modified UTF-7 encoding/decoding now works correctly in all cases
  • the mock package is no longer installed by default (just as a test dependency)
  • many, many bug fixes

See the release notes for more details.

Much work has already gone into the 2.0 release and it won't be too far behind 1.1.0. The headline change there is reworking IMAPClient's handling of TLS.

UC Data Science talk

This week I had the pleasure of giving a talk to Data Science masters students at the University of Canterbury (NZ). It was my attempt at giving some real-world advice from many years working as a technology geek. I think it went well and I hope the audience got something useful out of it.

The slides are now available.

I also managed to promote my current gig as well. Expect an article about what I've been up to soon.

Showing all Go compiler errors

The Go compiler normally stops after it finds 10 errors, aborting with a too many errors message. For example:

$ go build
# sandbox/manyerrs
./errs.go:4: undefined: w
./errs.go:5: undefined: w
./errs.go:6: undefined: w
./errs.go:7: undefined: w
./errs.go:8: undefined: w
./errs.go:9: undefined: w
./errs.go:10: undefined: w
./errs.go:11: undefined: w
./errs.go:12: undefined: w
./errs.go:13: undefined: w
./errs.go:13: too many errors

This is useful default behaviour - if there's lots problems you usually don't care about seeing all of the issues. You just fix what you can see and try again.

Sometimes though you really want to see all the errors.

Read more…

Python async talk

I gave a talk at last night's Christchurch Python meetup about Python's relatively new asynchronous programming features. To be honest, I didn't know all that much about the topic and signed myself up for the talk to force myself to learn :)

I used Jupyter Notebook for the presentation. Its ability to mix together text and interactive Python snippets works really well for this kind of talk. I've published the slides on Github as it has native support for rendering Jupyter notebooks (thanks Github!).