In my blog post The uselessness of bash I made a tool to improve pipes in shell, to assemble a better pipeline.
It solves the problem, but it’s a bit too different, with its own language.
While complaining with some people at work that one of the main features of shell (the pipe operator) is broken, someone joked that it should be replaced by a protobuf based protocol.
But on second thought it’s not really a joke.
How about instead of this:
$ goodpipe <<EOF [ ["gsutil", "cat", "gs://example/input-unsorted.txt"], ["sort", "-S300M", "-n"], ["gzip", "-9"], ["gsutil", "cp", "-", "gs://example/input-sorted-numerically.txt.gz"] ] EOF
how about this:
$ wp -o gsutil cat gs://example/input-unsorted.txt \ | wp -io sort -S300M -n \ | wp -io gzip -9 \ | wp -i gsutil cp - gs://example/input-sorted-numerically.txt.gz
First I hacked it together in Go, but I think the main implementation I’ll maintain is the one I made while porting it to Rust, as a way to learn Rust. The two implementations are compatible with each other.
I’ll write up my thoughts on Rust in a future blog post, but the summary is that it’s great, and ChatGPT was a helpful mentor in learning a new language.