A generic application for visualizing networked notes

A couple of years ago, zettelkastens and backlink-based note-taking applications that emphasize many interlinked notes became popular as the idea of zettelkastens became popular again. At least in the areas of the internet I frequent. I also jumped on the trend and started using one, which eventually became this blog: https://a-blog-with.relevant-information.com/posts/introducing_this_blog_thing/

I like plaintext stuff, partly because it makes everything very transparent. I can see the effect of an operation very clearly, and can do any manual intervention I like. For my slip-boxed note-taking software needs I use org-roam, which uses the plaintext format org-mode as the single source of truth, while using a sqlite database as a cache to speed things up.

Lately I’ve been eyeing another Emacs package called Denote, which focuses more on built-in Emacs functionality and seems to align better with its philosophy of extensibility. It is a system that is designed to be usable even with just basic Unix tools, which is supposed to make it more resilient to change. With a simple enough design it doesn’t hurt as much if Denote or even Emacs isn’t around in a couple of years, because it can be re-implemented with other tools. I’m attracted by the extensibility it offers because I’ve found that I wanted to change something about org-roam a couple of times, but the design of it doesn’t lend itself very well for extending or changing its functionality.

Another reason for my curiosity is, weirdly enough, performance. org-roam uses sqlite to improve performance, but the fundamental operation of searching by title is so slow that it takes multiple seconds to become responsive. Meanwhile, the normal file picker doesn’t even flinch. This isn’t because of a design flaw with org-roam, but a poor implementation. There exists an alternative implementation that is much faster, but the complexity of the solution makes me hesitate. (sidenote: I suppose one way to make it more useful is to apply some graph analysis algorithms on it, like clustering or something.¹ ) Is org-roam really as simple as it feels?

Because of this I’ve become unsure what the benefits of this database actually are. For org-roam itself, the speedup lies in the backlinks buffer I think. It allows org-roam to populate the backlinks buffer with neat metadata and context in real time. Using grep to find backlinks is also fast, but doesn’t give the same metadata and context. Another thing the database probably makes faster is the interactive graph view that org-roam-ui provides. The graph view shows all the notes in the database as nodes in a big graph, with edges representing a link between two notes. In order to generate this graph a program needs to go through all the files in the system and parse the links, so caching this process probably makes this a lot faster.

However, I’m not sure how much I care about this. The graph of my notes is so large and messy that looking at it doesn’t give any useful information. (sidenote: I suppose one way to make it more useful is to apply some graph analysis algorithms on it, like clustering or something.¹ ) What is more useful is the local graph, that shows a single node and its neighbours. You can add nodes (and their neighbours) through a command. This allows me to create an overview over a particular set of notes that I am interested in at the moment, which I think is pretty useful. The thing is though, that the sqlite database isn’t at all necessary for this. I suspect just grep ing for the current file’s filename or ID is enough in terms of performance.

This got me thinking, the hard part about displaying a small graph of notes seems to be to actually display the graph. Would it be feasible to create a generic application that displays a local graph for any set of interlinked files?

Here is what I’m envisioning. A simple client-server model, where the client is a web page and the server is a note-taking application. When the user wants to add note A to the graph, the editor collects the neighbouring nodes by parsing A for out-going links and greps the other files for in-going links. It then sends them along with some metadata (notably titles) to the client that displays them using one of the network graph visualisation libraries out there. When the user clicks on a node the page sends a message to the server, which responds with that node’s neighbours so that the page can add them to the graph.

Many graph visualisation programs try to be scalable to a large number of notes by making each node small. But since I’m not interested in large graphs I think it would be cool to have text boxes with the notes’ contents instead of the coloured balls that org-roam-ui has. In order to keep this format agnostic I’m thinking that the server is allowed to send arbitrary HTML for this, and let the client render it as-is. I don’t care about injections since both the client and the server is on the user’s machine.

It would be cool to have a link-less mode, where the server doesn’t send any information about the linked nodes. The web page just displays the text boxes without putting any edges, and lets the user rearrange them as they please. This could serve as a pin board where the user could organise their notes spatially. This could be saved in the browser storage so the user doesn’t have to recreate their setup all the time.

This kind of design would allow this application to be relatively format-agnostic, and let each note-taking application handle the bits about the format. That’s an interface, you’re describing an interface. With so many link-based note programs in the Emacs extension ecosystem alone, I think this could be pretty useful. Sadly, I haven’t created such a software, and don’t have any near-term plans of doing so. I’m just putting this out there in case anyone finds this useful enough to implement something similar themselves.