exupero's blog

Adventures in notebook programming

Other than being aware of Jupyter, I don't recall doing any programming in notebooks until about 2016 when I used Gorilla to visualize the effects of some signal processing. It worked well for that project, but creating new visualizations was somewhat obscure, as the front-end was implemented (if I recall) in Knockout.js, which wasn't familiar to me. It also didn't seem to be actively developed at that point, so I moved on.

For fun I built a basic notebook system inspired by Observable but based on Clojure, which taught me a lot about CodeMirror and self-hosted ClojureScript. Ultimately the complexity of managing asynchrony and attempted over-generalization proved to be a distraction, as anytime I went to use it for some analysis I ended up trying to fix bugs or add features to the platform.

The next venture avoided as much distraction as possible by aiming for much greater simplicity. I recycled a lot of CodeMirror code but avoided self-hosted ClojureScript and just evaluated synchronous Clojure forms on the server. Individual notebooks were saved as separate EDN files, which overall worked well but did occasionally require some bulk updates when the data format changed. Because the notebook system was tightly coupled to a particular domain, it's remained closed source, but implementing it taught a lot of valuable lessons for a later iteration.

At some point I tried a divergent approach that focused less on evaluatable cells of code and more on documents as a whole. It provided a Clojure editor side-by-side with the rendered HTML content. It was the simplest system I built, but too confusing, as it was often difficult to identify which code produced which content. A side-by-side split was also limiting, as there was too little horizontal space for both Clojure code and most data visualizations.

Abstracting the closed source domain-specific system produced Margins. It returned to evaluating cells with self-hosted ClojureScript, but with a clearer implementation than my earlier attempt. The visual design was also cleaner, borrowing from Observable. Observable also inspired the addition of a #‘ dispatch macro for string interpolation, which I wrote about last week. Instead of saving notebooks as individual files, all cells and notebook metadata were persisted in a single datahike DB.

Even with CodeMirror's Vim mode, however, editing in the browser always felt a little unnatural due to the loss of language plugins and how top-level forms are broken into cells. When Clerk debuted, its approach of serving as renderer only and not renderer plus editor made perfect sense to me. I dabbled briefly with Clerk and NextJournal, but most of my play is exploration and documentation rather than data science or deep analysis, so I knew I'd want a system over which I had complete control. The result is this blog.

Given past experience, implementing a Clojure-driven blog wasn't particularly difficult, though naturally it helped to omit an in-browser editor and self-hosted ClojureScript. Form evaluation isn't as sophisticated as Clerk's, which analyzes code changes and only re-evaluates downstream dependencies, but most of the entries in this blog are small and quick to run from scratch. Only occasional tinkering with the platform has been necessary, allowing me to focus more on the content and playing with ideas.