#+TITLE: Static Blog Builder #+SUBTITLE: A few static blog rewrite experiences #+AUTHOR: Yann Esposito #+EMAIL: yann@esposito.host #+DATE: [2021-05-01 Sat] #+KEYWORDS: blog static #+DESCRIPTION: Minimal and fast static website builder with make. #+OPTIONS: auto-id:t As someone on the Internet said not so far ago. Building its own static building system is a rite of passage for many developers. It has a lot of nice features. It gives a goal with a feeling of accomplishment. It is simple enough so most developers could build their own system. It could also become complex when you go down the rabbit hole. Along the years I used different tools and used and wrote of few static website systems: - [[https://nanoc.app][nanoc]] (in Ruby), at that time it looked like this: [[https://web.archive.org/web/20081002071448/http://nanoc.stoneship.org/][old nanoc 2 website]] - [[https://jaspervdj.be/hakyll/][hakyll]] (haskell static website generator) - [[https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html][org-publish]] (emacs package in conjunction with org-mode) - [[https://shakebuild.com][shake]] (haskell again) So if you look at the progression, I first used nanoc because I used ruby and it was a new solution, the website looked really great. Also the main developer [[https://denisdefreyne.com][Denis Defreyne]] was really helpful. Ruby was really great at dealing with regular expressions for hacking my documents. Then I was interested in Haskell, and I switched to a Haskell-made solution. I used hakyll, and I wrote a bit about it in [[http://yannesposito.com/Scratch/en/blog/Hakyll-setup/][Hakyll Setup]]. As a side note, the author of Hakyll [[https://jaspervdj.be/hakyll/][Jasper Van der Jeugt]] is apparently a friend of the author of nanoc. They both wrote a static site generators with their preferred programming language. I added a lot of personal features to my own site builder. It was a nice toy project. Then, due to a major disruption in my professional and private life I stopped to take care of my website. And a few years ago, I wanted to start a new website from scratch. In the meantime I switched my editor of choice from vim to Emacs. I started to work in Clojure and emacs is generally a natural choice because you can configure it with LISP. I discovered [[https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html][org-mode]] (I don't think the homepage of org mode makes justice to how incredible it is). So org-mode comes with an export system. Thus I switched to org-publish. Again [[https://her.esy.fun/posts/0001-new-blog/index.html][I wrote a bit about it]]. It was nice but slow. I improved a few things like writing a short script to [[https://her.esy.fun/posts/0005-rss-gen/index.html][Generate RSS from a tree of html files.]] I still had the feeling it was too slow. Static site building is a specific usage of a build system. And as I knew I could use =pandoc= to build HTML out of org-mode files and still versed in the Haskell culture I decided to try [[https://shakebuild.com][shake]]. You can learn more by reading this excellent paper about it, I think all developer should read it: [[https://github.com/snowleopard/build-systems/releases/download/icfp-submission/build-systems.pdf][Build System à la carte]]. As a bonus, [[https://pandoc.org][pandoc]] is written in Haskell. I could then directly use the [[https://pandoc.org][pandoc]] library in my build program. It worked like a charm and it was *very fast* as compared to other solutions I tried. So really let me tell you shake is a great build system. Unfortunately it was not perfect. While it was very fast, and I was able to use pandoc API directly. It made me dependent on Haskell. The best way I found to have Haskell reproducible build environment is to use [[https://nixos.org/nix][nix]]. This was great until the Big Sur update. To keep it short, nix stopped working on my computers after I upgraded my to Big Sur. Gosh, it was painful to fix. Concurrently I discovered [[/posts/0016-gemini/index.html][gemini]] and wanted to duplicate my website into gemini sphere. So I tried to update my build system but my code was to oriented to use pandoc and it was painful to have gemini in the middle of it. Particularly, generating a gemini index file. My main goal was to have gemini file that could only be linked from withing gemini sphere. Because gemini is a lot smaller web where you could feel a bit more protected from what the Web has become along the years. Whatever, in the end, I just had two problems to tackles. 1. Haskell became difficult to trust as very stable tool. Stable in the sense that I would not have any support work to do in order to keep just using it and not fixing/tweaking it. 2. Simplify the overall system to have a simpler build description So a very stable tool that I am pretty sure will still work almost exactly as today in 10 years is *=make=* (more precisely gnumake). I expected a lot of people had already come to the same conclusion and wrote about it. To my great surprise, I found very few article about generating static website with make. I only found solutions a bit too specific for my need. This is why I would like to give you a more generic starting point solution. * The =Makefile= :PROPERTIES: :CUSTOM_ID: the--makefile- :END: Instead of copy/pasting my current =Makefile= entirely let me give you a more generic one. It should be a great start. The first part will be used to simply copy the files from =src/= to =_site/=. #+begin_src makefile all: website # directory containing my org files as well as my assets files SRC_DIR ?= src # directory where I will but the files for my website (HTML + assets) DST_DIR ?= _site # list all files in src # if you want to exclude .org files use the exclude from the find command SRC_RAW_FILES := $(shell find $(SRC_DIR) -type f) # generate all file that should be copied in the site # For my site, I want to publish my source files along the HTML files DST_RAW_FILES := $(patsubst $(SRC_DIR)/%,$(DST_DIR)/%,$(SRC_RAW_FILES)) ALL += $(DST_RAW_FILES) # COPY EVERYTHING (.org file included) $(DST_DIR)/% : $(SRC_DIR)/% mkdir -p "$(dir $@)" cp "$<" "$@" #+end_src This part is about running the =pandoc= command for all =org= files in =src/= so they generate a html file in =_site/=. #+begin_src makefile # ORG -> HTML, If you prefer markdown replace .org by .md EXT := .org # all source file we'll pass to pandoc SRC_PANDOC_FILES ?= $(shell find $(SRC_DIR) -type f -name "*$(EXT)") # all destination files we expect (replace the extension by .html) DST_PANDOC_FILES ?= $(subst $(EXT),.html, \ $(subst $(SRC_DIR),$(DST_DIR), \ $(SRC_PANDOC_FILES))) ALL += $(DST_PANDOC_FILES) # use a template (you should use one) TEMPLATE ?= templates/post.html # URL of the CSS put yours CSS = /css/y.css # The pandoc command to run to generate an html out of a source file PANDOC := pandoc \ -c $(CSS) \ --template=$(TEMPLATE) \ --from org \ --to html5 \ --standalone # Generate all html if the org file change or the template change $(DST_DIR)/%.html: $(SRC_DIR)/%.org $(TEMPLATE) mkdir -p $(dir $@) $(PANDOC) $< \ --output $@ #+end_src A missing part is often the part where you would like to generate an index page to list the latest posts. Here you are a bit alone, you need to make one yourself. There is not generic way to do this one. #+begin_src makefile # Generating an index page is not difficult but not trivial either HTML_INDEX := $(DST_DIR)/index.html MKINDEX := engine/mk-index.sh $(HTML_INDEX): $(DST_PANDOC_FILES) $(MKINDEX) mkdir -p $(DST_DIR) $(MKINDEX) ALL += $(HTML_INDEX) #+end_src Finally, a few useful make commands. =make clean= and =make deploy=. #+begin_src makefile # make deploy will deploy the files to my website write your own script deploy: $(ALL) engine/deploy.sh website: $(ALL) .PHONY: clean clean: -rm -rf $(DST_DIR)/* #+end_src Limitation: =make= is old. So it really does not support spaces in filenames. Take care of that. Let me tell you. While this is quite a minimalist approach (<100 lines) it is nevertheless *very fast*. It will only generate the minimal amount of work to generate your website. I have a nice watcher script that update the website every time I save a file. It is almost instantaneous. The only risky dependencies for my website now is =pandoc=. Perhaps, they will change how they generate an HTML from the same org file in the future. I still use =nix= to pin my pandoc version. The static site builder itself is very simple, very stable and still very efficient. As a conclusion, if you want to write your own static site builder that's great. There are plenty of things to learn along the way. Still if you want something stable for a long time, with a minimal amount of dependencies, I think this Makefile is really a great start.