Never Write Shell Scripts Again

(use Python instead)

Menno Finlay-Smits <inbox@menno.io>

## The problem with shell scripts * Arcane syntax * Lack of high level programming constructs * Manual error handling * Lots of gotchas * Hard to organise larger amounts of code * Slow * Unit tests?

### Example: Conditional gotchas This works *unless* $filename is empty or contains a space: ``` if [ $filename = "foo" ]; ... ```

### Example: Error handling Scripts keep going by default: ``` mount /dev/foo /mnt/point # fails for some reason cp * /mnt/point # whoops! ``` There's a way of making this better but you have to remember to turn it on.

### Example: Numbers vs strings Numbers: ``` if [ $foo -eq 23 ]]; then ... ``` Strings: ``` if [ "$foo" = "bar" ]]; then ... ```

### Example: Arithmetic is weird ``` echo $((1 + 4)) ```

### Example: Functions are weird ``` function hello() { echo "Hello $1" } hello "Alice" ``` And forget about returning values...

## It's Not the Shell's Fault * Shells are old and have baggage * They were never meant to be complete programming languages * Some of my best friends are shell scripts

## What are shell scripts good at? * Effortlessly running other programs * Gluing different programs together * Globbing * String interpolation * Background execution

### Shell scripts: some good bits Witness the terse power: ``` ls /some/dir/*.jpg | grep -v foo | xargs rm -f echo "Hello $world, my name is $name." find / -name '*.jpg' & # run in background ```

## Python? * Concise & expressive * Easy to read * Great for keeping code organised * Powerful programming constructs * But ...

### Python: Gluing programs together sucks! ``` import subprocess p1 = subprocess.Popen("ls /some/dir/*.jpg", stdout=subprocess.PIPE, shell=True, ) p2 = subprocess.Popen(["grep", "-v", "foo"], stdin=p1.stdout) ... ``` Well out of the box anyway...

## Enter Plumbum... https://plumbum.readthedocs.io/

## Plumbum * The compactness of shell scripts in Python * Plumbum is latin for lead, which was used for pipes...

## Running Commands ``` >>> from plumbum.cmd import ls >>> >>> ls("/tmp") '/tmp/file0\n/tmp/file1\n' ```

## Dynamic Commands ``` >>> from plumbum.cmd import local >>> >>> ls = local["ls"] >>> ls("/tmp") '/tmp/file0\n/tmp/file1\n' ```

## Command failures raise exceptions ``` >>> ls('/does/not/exist') Traceback (most recent call last): ... plumbum.commands.processes.ProcessExecutionError: Command line: ['/bin/ls', '/does/not/exist'] Exit code: 2 Stderr: | /bin/ls: cannot access '/does/not/exist': No such file or directory ```

## Pipelining ``` >>> chain = ls['/some/dir/*.jpg'] | grep['-v', 'bar'] >>> print(chain()) /some/dir/bar01.jpg /some/dir/bar02.jpg ```

## I/O Redirection ``` >>> (cat < "setup.py") | head["-n", 1])() '#!/usr/bin/env python\n' ```

## Background execution ``` >>> from plumbum import BG >>> (ls["-a"] | grep["\\.py"]) & BG <Future ['/bin/grep', '\\.py'] (running)> ```

## Sending output to stdout Instead of output being returned... ``` >>> from plumbum import FG >>> ls['/tmp/foo'] & FG file0 file1 file2 ```

## Command nesting ``` >>> from plumbum.cmd import sudo >>> print(sudo["ifconfig", "-a"]) ```

## Remote execution (SSH) ``` >>> from plumbum import SshMachine >>> remote = Sshmachine("somehost", user="menno") >>> remote("ls") 'file1\nfile2\nfile3\n' ```

## Similar packages & tools * sh: https://amoffat.github.io/sh/ * sarge: https://bitbucket.org/vinay.sajip/sarge/ * shellypy: https://github.com/lamerman/shellpy

## No more excuses Write Python not Bash :)

## One more thing Python 3.6 f-strings are pretty great: ``` >>> your_name = 'Bob' >>> my_name = 'Grace' >>> f"Hello {your_name}, I'm {my_name}" "Hello Bob, I'm Grace" ``` Much like shell expansion, but way more powerful.

## More f-string examples ``` >>> f'Some arithmetic: {1 + 2 + 3}' 'Some arithmetic: 6' >>> f'Function calls: {max([1,2,3,2,1,2,99,3,4])}' 'Function calls: 99' ```