Daniel Duan's blog

Kick-ass CLI Tools In Swift

As someone who lives in a terminal simulator, I’m pleasantly surprised by the new toys we get in recent years such as fzf, ripgrep, fd, etc. A great number of these are written in relatively young programming languages such as Go and Rust. But, noticibly, none of them are written in Swift.

In this post, I’ll try to explain why that is.

POSIX Ergonomics

Unix-like virtual file systems has been around for decades. API that manupulates such systems has standardized a long time ago and exists in most computers running Linux/BSD/macOS today (and, to a large extend, smart phones). To Swift users, Using these APIs is straight-forward (rmdir("path/to/dir")).

So Swift programmers are all happy campers (re-)inventeing all sorts of file system utilities, right?

Well, not quite.

Okay, I lied about POSIX APIs being “straight-forward” in Swift. Or rather, this is very subjective.

Continuing with the rmdir example, we must first import it from either Glibc or Darwin, depending on your OS. To know whether the operation is successful, we need to see whether it returned integer 0. To learn why 0 was not returned, we need to read the “magical” variable errno. errno could be written to by other APIs so we’d better capture it in time…

And that’s one of the simpler APIs in POSIX calls!

Programmers whine about ergonomics partially because we are previlidged and spoiled. But mostly because our attention is a limited resources. Mixing API conventions distracts us from solving the problem at hand. Bad ergonomics, therefore, drives away a good potion of users who cares about quality of their tools.

Culture and History

As of this writing, the release of Swift 5 is imminent. The vast majority of existing Swift code is written to run on iOS. The concept of a file, or the traditional virtal file system, is hidden to iOS users, and sandboxed for developers. I bet most Swift users rarely think about the fact that there’s a entire set of POSIX API at their disposal.

Foundation alleviates the need to deal with files and directories: Bundle locates the files; CoreData, UserDefaults or the keychain is your primary way to persist data; Data, String or NSCoding has methods to read and write to files. And finally, if you really need to deal with files, NSFileManager has everything you’ll ever need.

Why would a productive Swift programmer think about POSIX in this environment? Why would a tutor teach POSIX over the useful/practical/”native” alternatives?

We can trace “riding on the Apple platform” mentality back to the pre-iPhone days, where a very small Mac developer community labors on on a niche platform (compared to iOS today) and they loved it. However, I’m sure they used more POSIX stuff back then than the average iOS developers today.

Having a great library such as Foundation on the most popular developer platform where the language thrives means it’ll take longer for “subcultures” to emerge, if they do at all.

The Standard Library And Its Influence on New Users

File system APIs being in Foundation as opposed to the standard library is probably a temporary condition. Nevertheless, it has at least the following implications:

  1. Its quality of implementation is not held on the same standard that those APIs in the standard library. This is especially true for the separate, open-source Foundation implementation. Getting consistent and correct behaviors across macOS and Linux is hard.

  2. A person learning Swift won’t explore the language with a file system API. This I suspect, is the most important reason many of these great CLI utilites are written in other programming languages. Programmers seek instant gratification when they learn. And they usually stay in a limited domain (like iOS) at first. This is where the built-in library is special: no matter which domain is chosen, it’s always available. Languages such as Go and Rust include things like paths and files in their built-in library. Playing with these APIs while learning the lanugage plants a seed for future, serious, projects. There are less users of these languages compared to Swift, but there are more people thinking about projects that involves file systems in thoes communities. (Note I don’t have statistics here, just a guess.)

Conclusion

The next killer CLI tool is still more likely to be written in Go or Rust, than in Swift. Hopefully, somewhere in these speculations is a true cause of this phenomena. Maybe someone reading this will be inspired to accelerate change that will eventually revert the condition. (I’m trying).


Magical Interactions

I want to talk about this little function:

func interact(_ process: (String) -> String) {
    var input = ""
    while let line = readLine() {
        input += line
    }

    print(process(input))
}

Brief explanation: it reads all the input from stdin as a String, feeds it into a closure process, which it takes in as its only argument, and prints process’s output.

Here’s how one might use it:

// counts characters from stdin and prints result to stdout
interact { String($0.count) }

Got that? Well, now I’m going to rewrite it in a slightly less Swift-y way:

let count: (String) -> String {
    return String($0.count)
}

interact(count)

The argument for interact got defined with a name and an explicit type signature.

So, what’s so special about this interact function? Two words: side effects. More precisely, it took away the concern of side-effects from the user. count belongs in the realm of pure functions. It has no worries about file handles or operating systems. It’s (String) -> String. I wanted to emphasize this with the rewrite. Look at that empty line. Now you see a boundary between 2 worlds.

This may all seem contrived. But when I learned about this function in Haskell, I was blown away.

It’s like a great magic trick: you are presented a scenario, say, writing a little script. Maybe you need to process some CLI output and print out a CSV or JSON string (literally 90% of the script I write). A Haskell programmer would jump into the bottom level of the problem and start writing these little pure functions: one to split the string, one to convert some numbers, one to manipulate a list, one to match some patterns…gradually the broken-down absractions get built back up via function composition. You can see the light at the end of the tunnel, yes, yes! If you feed this list into that function that returns a string you’ll have the right value to print out! Okay, now the problem is solved in the pure functional world! The only thing left to do is…

Now, the setup of the magic is complete. Now, you are onboard with the solution, you thought the problem through with the magician…you are distracted. The ending came so…quickly, but unexpected. What? You just feed your solution to this interact function and…that’s it? I was expecting some readLines or prints (okay, at least 1 print statement)!

That’s the thing: interact deals with two side effects, the input and the output. But its user deals with zero. It’s as if the two effects “cancel” each other out! It’s a neat trick, really. Small, low-key, easy to miss. But I’m glad I noticed it and come to appreciate its power and simplicity.


🔗 [Accepted with Modifications] SE-0225

This announcement comes with some answers to “what should be in the Swift standard library?”

Notably:

Almost all library features can be composed from existing features with enough effort. To be considered for addition to the library, a proposed feature must satisfy two conditions: it must provide functionality that is useful to a substantial population of Swift programmers, and it must provide substantial advantages over the alternative ways of accomplishing that functionality. Potential advantages include:

It may be complex, challenging, or error-prone for users to implement themselves. It may have substantial performance advantages over a user implementation, either because it has access to library internals or just because the library implementation will likely be more carefully tuned. It may be substantially easier to work with because it composes better with other language or library features. It may be substantially more “fluent”: that is, more natural to discover, use, and read in code. The implementation may involve composing primitives in a subtle or tricky way, or the primitives may be unfamiliar to many programmers. This is a more subjective criterion than the others, and people may reasonably differ about how to apply it. For example, consider adding an method to rotate the elements of an array. This operation can be performed using a relatively simple composition of slicing and concatenation:

array = array[amount...] + array[..<amount]

However, a method to rotate an array in-place would still be welcome: it would have substantial performance advantages, it would avoid several potential off-by-one errors, and it would be significantly easier for a reader to recognize as a rotation.

In contrast, a property like isNotEmpty offers no substantial advantages over !isEmpty, and adding it would open the door to adding trivial negations of essentially every predicate.