My Python Toolbox

After a recent Christchurch Python meetup, I was asked to create a list of Python libraries and tools that I tend to gravitate towards when working on Python projects. The request was aimed at helping newer Pythonistas find a way through the massive Python ecosystem that exists today.

This is my attempt at such a list. It's highly subjective, being coloured by my personal journey. I've done a lot of Python in my career but that doesn't mean that I've necessarily picked the best tool for each task. Still, I hope it's useful as a starting point (I don't think there's any big surprises here).

Read more…

IMAPClient 2.1.0

IMAPClient 2.1.0 has just been released! Here's the main highlights:

  • Python 3.7 is now officially suppported
  • Testing against PyPy (version 2 and 3)
  • Added support for the QUOTA extension
  • Helper for locating special folders
  • Document usage for using self-signed TLS certificates
  • Document how to use the email package from the standard library to parse fetched emails
  • Handling NIL values for INTERNALDATE

As always, IMAPClient can be installed using pip (pip install imapclient). Full documentation is available at Read the Docs.

Enjoy!

Listing S3 objects with NodeJS

I recently had to write some NodeJS code which uses the AWS SDK to list all the objects in a S3 bucket which potentially contains many objects (currently over 80,000 in production). The S3 listObjects API will only return up to 1,000 keys at a time so you have to make multiple calls, setting the Marker field to page through all the keys.

It turns out there's a lot of sub-optimal examples out there for how to do this which often involve global state and complicated recursive callbacks. I'm also a fan of the clarity of JavaScript's newer async/await feature for handling asynchronous code so I was keen on a solution which uses that style.

Read more…