I recently had to write some NodeJS code which uses the AWS SDK to
list all the objects in a S3 bucket which potentially contains many
objects (currently over 80,000 in production). The S3 listObjects
API will only return up to 1,000 keys at a time so you have to make
multiple calls, setting the
Marker field to page through all the
I recently gave a talk at the Christchurch Python meetup which discussed the strengths and weaknesses of shell scripts, and why you might want to consider using Python instead of shell scripts. I went into some areas where Python is arguably worse than shell scripts, and then we dived into the excellent Plumbum package which nicely addresses most of those weaknesses.
The slides for the talk are available here.
IMAPClient 1.1.0 has just been released! Many thanks to the recent contributors and the project's other maintainers, Nicolas Le Manchet and Maxime Lorant. This release is full of great stuff because of them. Here's some highlights:
- search now supports nested criteria so that more complex criteria can be expressed. IMAPClient will add parentheses in the right place. See
- PLAIN authentication support
- UNSELECT support
- ENABLE support
- UID EXPUNGE support
- IMAP modified UTF-7 encoding/decoding now works correctly in all cases
- the mock package is no longer installed by default (just as a test dependency)
- many, many bug fixes
See the release notes for more details.
Much work has already gone into the 2.0 release and it won't be too far behind 1.1.0. The headline change there is reworking IMAPClient's handling of TLS.
This week I had the pleasure of giving a talk to Data Science masters students at the University of Canterbury (NZ). It was my attempt at giving some real-world advice from many years working as a technology geek. I think it went well and I hope the audience got something useful out of it.
The slides are now available.
I also managed to promote my current gig as well. Expect an article about what I've been up to soon.
The Go compiler normally stops after it finds 10 errors, aborting with a too many errors message. For example:
$ go build # sandbox/manyerrs ./errs.go:4: undefined: w ./errs.go:5: undefined: w ./errs.go:6: undefined: w ./errs.go:7: undefined: w ./errs.go:8: undefined: w ./errs.go:9: undefined: w ./errs.go:10: undefined: w ./errs.go:11: undefined: w ./errs.go:12: undefined: w ./errs.go:13: undefined: w ./errs.go:13: too many errors
This is useful default behaviour - if there's lots problems you usually don't care about seeing all of the issues. You just fix what you can see and try again.
Sometimes though you really want to see all the errors.
I gave a talk at last night's Christchurch Python meetup about Python's relatively new asynchronous programming features. To be honest, I didn't know all that much about the topic and signed myself up for the talk to force myself to learn :)
I used Jupyter Notebook for the presentation. Its ability to mix together text and interactive Python snippets works really well for this kind of talk. I've published the slides on Github as it has native support for rendering Jupyter notebooks (thanks Github!).
The code has been imported, tickets have migrated and the Bitbucket repository is now a redirect: the IMAPClient project is now using Git and Github.
The Github link is: https://github.com/mjs/imapclient
(http://imapclient.freshfoo.com redirects to the new location)
This change has been planned for a long time. I've noticed a that many contributors struggle with Mercurial and Bitbucket, and I greatly prefer Git myself nowadays.
IMAPClient 1.0.2 is out! This is release comes with a few small fixes and tweaks, as well as a some documentation improvements.
- There's now an explicit check that the pyOpenSSL version that IMAPClient is seeing is sufficient. This is to help with situations (typically on OS X) where the (old) system pyOpenSSL takes precedence over the version that IMAPClient needs. Use of virtualenvs is highly recommended.
- Python 3.5 is now officially supported and tested against.
- setup.py can now be used even if it's not in the current directory.
- Handling of RFC2822 group address syntax has been documented.
- The INI file format used by the live tests and interactive shell has finally been documented.
- Links to ReadTheDocs now go to readthedocs.io
- The project README has been arranged so that all the essentials are right at the top.
I announced that the project would be moving to Git and Github some time ago and this is finally happening. This release will be the last release where the project is on Bitbucket.