Embedding Neovim HTML into Obsidian

# Embedding Neovim HTML Into Obsidian What started off as just a crazy idea on America Day (fueled by the spirit of independence and freedom) has resulted in me yak-shaving on my blog for many hours (and don't get me started on the `Copy Code` button either). Below is the end result - a syntax highlighted code block, including highlighting of errors in exactly the way it looks inside of my editor (Neovim, btw) ```neovim something <pre> let%query (module UserName) = "SELECT User.idd, User.name FROM User" │ └──── Unbound type constructor User.Fields.idd │ Hint: Did you mean id? └──── Unbound value User.Fields.serialize_idd Hint: Did you mean serialize_id? </pre> ``` _The example is from my project about generating type-safe SQL for OCaml, but you can read more about that at the start of the series: [[2024-07-02 - SQL In OCaml (Part 1)|SQL In OCaml]]_ ## Problem I wanted to be able to create perfectly highlighted code snippets for this website - but without any screenshots. I've never really been a fan of screenshots in technical blogs, but I also find that most syntax highlighting solutions are a bit lackluster in the browser (this includes in Electron based editors, btw). ## Solution What if I: - Used Neovim's scripting capabilities to generate HTML with the correct highlight groups attached to each span in a `<pre>` block - Make it easy to copy the HTML from Neovim and/or edit text files and insert them back into Obsidian - Write a small plugin in Obsidian to override the display of `neovim` codeblocks, so the HTML is displayed correctly and styled. It's only a matter of time when you start thinking about writing your markdown-based "simple blog" that you come to such madness. But onward! Onward with our generated HTML, custom codeblocks, and a touch of dangerously-sett-inner-HTML (exactly like the web was designed for)! Onward to displaying the HTML directly and beautifully, with the final result being an over-engineered syntax highlighting replicating that of my beloved editor! (Onward, because we definitely don't want to look back now!) # Implementation ## Custom `:TOhtml` command I've long been a fan of the `:TOhtml` command in Vim and Neovim. In fact, quite a long time ago I attempted to add some HTML generation from the early days of tree-sitter and Neovim here (very good commits, as always in my personal projects): - [Start working on some 'to html' stuff](https://github.com/tjdevries/colorbuddy.nvim/commit/0e7ecae265301303d359df0336f5b2f92e2432db) - [Add Lua HTML Stuff](https://github.com/tjdevries/colorbuddy.nvim/commit/faf847d551eba6123f1997152a1959c951e1735f) Unfortunately I hit a dead end, partially the Lua integration in 2018 wasn't quite what it is today, but primarily because I didn't care about anything in a browser. That is, until Neovim 0.10 was released and the new [TOhtml](https://neovim.io/doc/user/lua.html#_lua-module:-tohtml) command was created! Now there was an incredibly easy way to get all of the Neovim specific highlights generated correctly. > [!note] Contact Me! > If you'd like to hear more about any of those differences and why Neovim needed specific handling for a feature that's been around in Vim for a long time, feel free reach out to me on [X](https://x.com/teej_dv) or similar and let me know. Happy to write more random Neovim tidbits down. However, the builtin `:TOhtml` didn't quite fit exactly what I was looking for: - It generated a complete HTML document, (`DOCTYPE`, `head` and all) - It always included styling information in a `style` tag, and only the styles used in the block - It couldn't be used to generate only a range of lines So I did what any self-respecting Neovimmer would do - copied the entire file into my Neovim configuration, removed the parts of the code that I didn't like and spent even more time writing custom code so it would be just the way I liked it. - You can see the initial commit [here](https://github.com/tjdevries/config.nvim/commit/892ec7ebbb2755d057e05e4934c320a162be2266) - And the followup commit [here](https://github.com/tjdevries/config.nvim/commit/49132b871c40d595f6e9e520c73cd9592c79544b) After this I had accomplished the three primary things I needed from `:TOhtml` - Generate a file of all the styles from any highlight set in my Neovim configuration - Generate only a range of the file as HTML - Generate a file/Copy to clipboard/Write to stderr the generated HTML of a file I also added a few "quirks" that I like including: - Stripping leading whitespace from the HTML (particularly useful when grabbing snippets) - Automatically placing the HTML into my clipboard register, so I can paste it whenever I want. ## Easily copy a file into Obsidian Now that we've gained the capability of generating HTML from a Neovim buffer, we need some way to actually place that HTML into Obsidian. Believe it or not, I actually enjoy writing non-technical things *outside of Neovim*. I think Obsidian is an elegant piece of technology and does it's job very well - and if I ever change my mind, at least Obsidian is [[File over App|File Over App]]. ![[Pasted image 20240706014217.png]] Using my custom `:TOhtml` command, I'm able to create a Neovim command that takes a range, so I can put the HTML directly into my clipboard and paste it into Obsidian. This method works really well for one-off scripts / references that do not need to get updated, or that have some custom highlights/diagnostics applied to the selection. ```neovim <pre> vim.api.nvim_create_user_command("ObsidianHTML", function(args) package.loaded["custom.to-obsidian-html"] = nil local opts = {} if args.range ~= 0 then opts.filter = function(row) return row >= args.line1 and row <= args.line2 end end require("custom.to-obsidian-html").tohtml(0, opts) end, { range = true })</pre> ``` Now I can select a range of code and easily paste the entire code block into Obsidian (since the `lines` includes the `neovim` codeblock demarcation) ## Custom Code Block Processor Obsidian's API for the local Obsidian Vault and Obsidian Publish contains a function: `registerMarkdownCodeBlockProcessor`. ```neovim <pre> /** * Register a special post processor that handles fenced code given a language and a handler. * This special post processor takes care of removing the <pre><code> and create a <div> that * will be passed to the handler, and is expected to be filled with custom elements. * @see {@link ...} * @public */ registerMarkdownCodeBlockProcessor( language: string, handler: ( source: string, el: HTMLElement, ctx: MarkdownPostProcessorContext) => Promise<any> | void, sortOrder?: number ): MarkdownPostProcessor;</pre> ``` This lets you control the handling of the `<language>` of code blocks inside of Obsidian. Since we have this exposed to us, we can do exactly the thing that React has told you is dangerous for many years: ```neovim <pre> this.registerMarkdownCodeBlockProcessor("neovim", async (content, el, _ctx) => { el.innerHTML = content; });</pre> ``` We just set the HTML to the content inside of the codeblock. I know that I'm generated valid HTML from the plugin, I already have the styles set up by placing them in my [publish.css](https://help.obsidian.md/Obsidian+Publish/Customize+your+site#Static+assets), so let's just tell Obsidian to display it as the embedded HTML! That's actually it! So inside of the code block it looks like: ![[Pasted image 20240707154722.png]] But the display is what you see above! I think this is a pretty fun strategy, particularly because in the end, if I ever want to switch off of Obsidian or publish somewhere besides Obsidian Publish, I have all the HTML for all of my codeblocks already generated and included in the documents! ## Update referenced files in Obsidian But now I'm asking myself: > What about for files that can change? Or is it possible to include files without copy and pasting? Fortunately, Obsidian has a great plugin API and with Typescript it's easy to write code that I'll regret later. You can make an Obsidian plugin locally, and the easiest way I found was to use the [Sample Plugin](https://github.com/obsidianmd/obsidian-sample-plugin). Once you've got that running, it's pretty easy to explore and iterate quickly on your plugin. You can also use the [Hot Reload Plugin](https://github.com/pjeby/hot-reload) to make it really quick to check different options within your running Obsidian session. Once I had my own Obsidian plugin, I created new commands. The first commands allows me to update `INCLUDE NEOVIM` blocks. ```neovim <pre> this.addCommand({ id: 'update-include-neovim-blocks', name: 'Update INCLUDE NEOVIM blocks', callback: () => { this.updateNeovimHighlightBlocksCommand(); } });</pre> ``` An `INCLUDE NEOVIM` block looks like this in plain text: ``` %% INCLUDE NEOVIM <filepath> %% '''neovim <contents of file as highlight HTML> ''' ``` The `%% INCLUDE NEOVIM <filepath> %%` is stripped out on Publish (it's basically a comment in Markdown), so it won't show up in the published site. The following codeblock is the `neovim` codeblock that we'll talk about later (it uses a custom `registerMarkdownCodeBlockProcessor`). The command just looks for lines like the `INCLUDE` line, parses out the `filepath` and then executes a Neovim process like so: ```neovim <pre> const absolutePath = this.fileToPath(resolved); const output = await captureStderr( 'nvim', ['--headless', '-c', `RunObsidian ${absolutePath}`, '-c', 'q'] );</pre> ``` Which captures the `stderr` from Neovim, executes it in a headless session, calls the `RunObsidian` command with the absolute path of the file, and then quits when it's done. That's it! Now I can use Obsidian to update the Neovim blocks whenever I want. A future improvement could be to run this automatically when file changes are detected, but I don't really think that's useful since I'm only going to be editing the files when I'm updating a blog post. ![[Pasted image 20240707151413.png]] ## Edit in Neovim Now that I have an easy way to both View and Update code snippets within Obsidian and get the corresponding Neovim highlights, we need a way to edit them quickly! First, we register a new command for Obsidian in our plugin. ```neovim <pre> this.addCommand({ id: 'edit-neovim-block', name: 'Edit a NEOVIM block', callback: () => { this.editNeovimHighlightBlockInNeovim(); } });</pre> ``` Then, we open up a terminal (for me, `ghostty` at the moment) and instruct Neovim to open the referenced file in the current Neovim block: ```neovim <pre> await captureStderr('ghostty', ['-e', 'nvim', '-c', `"EditObsidian ${absolutePath}"`]); let activeFile = this.app.workspace.getActiveFile()! let activeFileContents = await this.app.vault.cachedRead(activeFile) this.updateNeovimHighlightBlock(activeFile, cursor.line, activeFileContents.split("\n"))</pre> ``` We just await until Neovim is closed (so when I `:wq`) and then update the current Neovim highlight block (we re-use the code that we had done previously in [[#Custom Code Block Processor]]. One fun thing I did, was because I wanted to edit these files in Neovim, I made a new AwesomeWM rule that said that anything with the title `edit-obsidian` would become a floating window that pops up on top of the screen. So the result is that when I run my `Edit INCLUDE NEOVIM` block command, a brand-new Neovim instance floats above Obsidian for me. It looks like this: ![[Pasted image 20240708132648.png]] And when I `:wq` from Neovim, the text is instantly updated inside of my post. This makes it really easy to write code snippets when I want to do something more complicated without getting the writing groove cut short. Also it makes me smile every time it works :) # Various Code Block Examples Included OCaml highlighting Before: ```ocaml type t = | String of string | Int of int | Float of float | Bool of bool let string (s : string) -> t = String s let int (i : int) -> t = Int i let float (f : float) -> t = Float f let bool (b : bool) -> t = Bool b ``` After: ```neovim {"update":true,"path":"_/code/why-i-like-ocaml-int-or-string.ml","timestamp":1725484757472} <pre> type t = | String of string | Int of int | Float of float | Bool of bool let string (s : string) -> t = String s let int (i : int) -> t = Int i let float (f : float) -> t = Float f let bool (b : bool) -> t = Bool b </pre> ``` Scroll bars instead of wrapping: ```neovim {"update":true,"path":"_/code/1720205717935.ml","timestamp":1725484757200} <pre> let _ = print_endline "testing 1 2 3 4 5 this is longer and again wow!!!! A really long string won't wrap, it will scroll" module Something = struct type t = int let make (x : int) : int -> t = t end </pre> ``` Can do other languages (although why would you want to write anything besides OCaml) ```neovim {"update":true,"path":"_/code/1720205765886.c","timestamp":1725484756943} <pre> #include <stdio.h> int main() { printf("Hello %s!!\n", "world"); return 0; } </pre> ``` And for the cherry on top, we even get cool stuff like including errors and highlights from the code! I plan on also using this feature to be able to add custom messages to code, to easily illustrate what part of the code I am talking about, like below! ```neovim <pre> local opts = {} local lines = require("custom.to-obsidian-html").tohtml(0, opts) io.stderr:write("Trying to open " .. filename .. "\n") io.stderr:write("Current working directory: " .. vim.fn.getcwd() .. "\n") └──── Incredibly useful message io.stderr:write(table.concat(lines, "\n"))</pre> ``` I set this with a 'simple' command like this, but would be able to do this automatically with a script as well in the future. ```neovim <pre> vim.diagnostic.set(vim.api.nvim_create_namespace "Blog", 0, { { lnum = 20, col = 2, end_col = 11, severity = vim.diagnostic.severity.HINT, message = "Incredibly useful message", }, })</pre> ``` # Conclusion Sometimes, it's good to just do something for fun and because you like it. It doesn't matter if I've written more code than blog posts, I'm happy. See you in the next post (hopefully) ## Addendum I love [Berkeley Mono](https://berkeleygraphics.com/) the font for all the `pre` blocks (Neovim or otherwise) on my site: ```neovim {"update":true,"path":"_/code/1720807806302.css","timestamp":1725484756702} <pre> pre { font-family: "BerkeleyMono", monospace; } </pre> ``` ^1720807806302