229 lines
8.8 KiB
Org Mode
229 lines
8.8 KiB
Org Mode
#+TITLE: Static Blog Builder
|
|
#+SUBTITLE: A few static blog rewrite experiences
|
|
#+AUTHOR: Yann Esposito
|
|
#+EMAIL: yann@esposito.host
|
|
#+DATE: [2021-05-01 Sat]
|
|
#+KEYWORDS: blog static
|
|
#+DESCRIPTION: Minimal and fast static website builder with make.
|
|
#+OPTIONS: auto-id:t
|
|
|
|
|
|
|
|
As someone on the Internet said not so far ago.
|
|
Building its own static building system is a rite of passage for many developers.
|
|
It has a lot of nice features.
|
|
It gives a goal with a feeling of accomplishment.
|
|
It is simple enough so most developers could build their own system.
|
|
It could also become complex when you go down the rabbit hole.
|
|
|
|
Along the years I used different tools and used and wrote of few static
|
|
website systems:
|
|
|
|
- [[https://nanoc.app][nanoc]] (in Ruby), at that time it looked like this: [[https://web.archive.org/web/20081002071448/http://nanoc.stoneship.org/][old nanoc 2 website]]
|
|
- [[https://jaspervdj.be/hakyll/][hakyll]] (haskell static website generator)
|
|
- [[https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html][org-publish]] (emacs package in conjunction with org-mode)
|
|
- [[https://shakebuild.com][shake]] (haskell again)
|
|
|
|
So if you look at the progression, I first used nanoc because I used ruby
|
|
and it was a new solution, the website looked really great.
|
|
Also the main developer [[https://denisdefreyne.com][Denis Defreyne]] was really helpful.
|
|
Ruby was really great at dealing with regular expressions for hacking my
|
|
documents.
|
|
|
|
Then I was interested in Haskell, and I switched to a Haskell-made
|
|
solution.
|
|
I used hakyll, and I wrote a bit about it in [[http://yannesposito.com/Scratch/en/blog/Hakyll-setup/][Hakyll Setup]].
|
|
As a side note, the author of Hakyll [[https://jaspervdj.be/hakyll/][Jasper Van der Jeugt]] is apparently a
|
|
friend of the author of nanoc.
|
|
They both wrote a static site generators with their preferred programming
|
|
language.
|
|
I added a lot of personal features to my own site builder.
|
|
It was a nice toy project.
|
|
|
|
Then, due to a major disruption in my professional and private life I
|
|
stopped to take care of my website.
|
|
|
|
And a few years ago, I wanted to start a new website from scratch.
|
|
In the meantime I switched my editor of choice from vim to Emacs.
|
|
I started to work in Clojure and emacs is generally a natural choice
|
|
because you can configure it with LISP.
|
|
I discovered [[https://orgmode.org/worg/org-tutorials/org-publish-html-tutorial.html][org-mode]] (I don't think the homepage of org mode makes justice
|
|
to how incredible it is).
|
|
So org-mode comes with an export system.
|
|
Thus I switched to org-publish.
|
|
Again [[https://her.esy.fun/posts/0001-new-blog/index.html][I wrote a bit about it]].
|
|
|
|
It was nice but slow.
|
|
I improved a few things like writing a short script to
|
|
[[https://her.esy.fun/posts/0005-rss-gen/index.html][Generate RSS from a tree of html files.]]
|
|
I still had the feeling it was too slow.
|
|
|
|
Static site building is a specific usage of a build system.
|
|
And as I knew I could use =pandoc= to build HTML out of org-mode files
|
|
and still versed in the Haskell culture I decided to try [[https://shakebuild.com][shake]].
|
|
You can learn more by reading this excellent paper about it, I
|
|
think all developer should read it: [[https://github.com/snowleopard/build-systems/releases/download/icfp-submission/build-systems.pdf][Build System à la carte]].
|
|
|
|
As a bonus, [[https://pandoc.org][pandoc]] is written in Haskell.
|
|
I could then directly use the [[https://pandoc.org][pandoc]] library in my build program.
|
|
It worked like a charm and it was *very fast* as compared to other
|
|
solutions I tried.
|
|
So really let me tell you shake is a great build system.
|
|
|
|
Unfortunately it was not perfect.
|
|
While it was very fast, and I was able to use pandoc API directly.
|
|
It made me dependent on Haskell.
|
|
The best way I found to have Haskell reproducible build environment is to
|
|
use [[https://nixos.org/nix][nix]].
|
|
This was great until the Big Sur update.
|
|
To keep it short, nix stopped working on my computers after I upgraded my
|
|
to Big Sur.
|
|
Gosh, it was painful to fix.
|
|
|
|
Concurrently I discovered [[/posts/0016-gemini/index.html][gemini]] and wanted to duplicate my website into
|
|
gemini sphere.
|
|
So I tried to update my build system but my code was to oriented to use
|
|
pandoc and it was painful to have gemini in the middle of it.
|
|
Particularly, generating a gemini index file.
|
|
My main goal was to have gemini file that could only be linked from withing
|
|
gemini sphere.
|
|
Because gemini is a lot smaller web where you could feel a bit more
|
|
protected from what the Web has become along the years.
|
|
Whatever, in the end, I just had two problems to tackles.
|
|
|
|
1. Haskell became difficult to trust as very stable tool. Stable in the
|
|
sense that I would not have any support work to do in order to keep just
|
|
using it and not fixing/tweaking it.
|
|
2. Simplify the overall system to have a simpler build description
|
|
|
|
So a very stable tool that I am pretty sure will still work almost exactly
|
|
as today in 10 years is *=make=* (more precisely gnumake).
|
|
I expected a lot of people had already come to the same conclusion and
|
|
wrote about it.
|
|
To my great surprise, I found very few article about generating static
|
|
website with make.
|
|
I only found solutions a bit too specific for my need.
|
|
This is why I would like to give you a more generic starting point
|
|
solution.
|
|
|
|
* The =Makefile=
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: the--makefile-
|
|
:END:
|
|
|
|
Instead of copy/pasting my current =Makefile= entirely let me give you a
|
|
more generic one.
|
|
It should be a great start.
|
|
|
|
The first part will be used to simply copy the files from =src/= to
|
|
=_site/=.
|
|
|
|
#+begin_src makefile
|
|
all: website
|
|
|
|
# directory containing my org files as well as my assets files
|
|
SRC_DIR ?= src
|
|
# directory where I will but the files for my website (HTML + assets)
|
|
DST_DIR ?= _site
|
|
|
|
# list all files in src
|
|
# if you want to exclude .org files use the exclude from the find command
|
|
SRC_RAW_FILES := $(shell find $(SRC_DIR) -type f)
|
|
# generate all file that should be copied in the site
|
|
# For my site, I want to publish my source files along the HTML files
|
|
DST_RAW_FILES := $(patsubst $(SRC_DIR)/%,$(DST_DIR)/%,$(SRC_RAW_FILES))
|
|
ALL += $(DST_RAW_FILES)
|
|
|
|
# COPY EVERYTHING (.org file included)
|
|
$(DST_DIR)/% : $(SRC_DIR)/%
|
|
mkdir -p "$(dir $@)"
|
|
cp "$<" "$@"
|
|
#+end_src
|
|
|
|
This part is about running the =pandoc= command for all =org= files in =src/=
|
|
so they generate a html file in =_site/=.
|
|
|
|
#+begin_src makefile
|
|
# ORG -> HTML, If you prefer markdown replace .org by .md
|
|
EXT := .org
|
|
# all source file we'll pass to pandoc
|
|
SRC_PANDOC_FILES ?= $(shell find $(SRC_DIR) -type f -name "*$(EXT)")
|
|
# all destination files we expect (replace the extension by .html)
|
|
DST_PANDOC_FILES ?= $(subst $(EXT),.html, \
|
|
$(subst $(SRC_DIR),$(DST_DIR), \
|
|
$(SRC_PANDOC_FILES)))
|
|
ALL += $(DST_PANDOC_FILES)
|
|
|
|
# use a template (you should use one)
|
|
TEMPLATE ?= templates/post.html
|
|
# URL of the CSS put yours
|
|
CSS = /css/y.css
|
|
# The pandoc command to run to generate an html out of a source file
|
|
PANDOC := pandoc \
|
|
-c $(CSS) \
|
|
--template=$(TEMPLATE) \
|
|
--from org \
|
|
--to html5 \
|
|
--standalone
|
|
|
|
# Generate all html if the org file change or the template change
|
|
$(DST_DIR)/%.html: $(SRC_DIR)/%.org $(TEMPLATE)
|
|
mkdir -p $(dir $@)
|
|
$(PANDOC) $< \
|
|
--output $@
|
|
|
|
#+end_src
|
|
|
|
A missing part is often the part where you would like to generate
|
|
an index page to list the latest posts.
|
|
Here you are a bit alone, you need to make one yourself.
|
|
There is not generic way to do this one.
|
|
|
|
#+begin_src makefile
|
|
# Generating an index page is not difficult but not trivial either
|
|
HTML_INDEX := $(DST_DIR)/index.html
|
|
MKINDEX := engine/mk-index.sh
|
|
$(HTML_INDEX): $(DST_PANDOC_FILES) $(MKINDEX)
|
|
mkdir -p $(DST_DIR)
|
|
$(MKINDEX)
|
|
ALL += $(HTML_INDEX)
|
|
#+end_src
|
|
|
|
Finally, a few useful make commands. =make clean= and =make deploy=.
|
|
|
|
#+begin_src makefile
|
|
# make deploy will deploy the files to my website write your own script
|
|
deploy: $(ALL)
|
|
engine/deploy.sh
|
|
|
|
website: $(ALL)
|
|
|
|
.PHONY: clean
|
|
|
|
clean:
|
|
-rm -rf $(DST_DIR)/*
|
|
#+end_src
|
|
|
|
Limitation: =make= is old.
|
|
So it really does not support spaces in filenames.
|
|
Take care of that.
|
|
|
|
Let me tell you.
|
|
While this is quite a minimalist approach (<100 lines) it is nevertheless *very fast*.
|
|
It will only generate the minimal amount of work to generate your website.
|
|
I have a nice watcher script that update the website every time I save a
|
|
file.
|
|
It is almost instantaneous.
|
|
|
|
The only risky dependencies for my website now is =pandoc=.
|
|
Perhaps, they will change how they generate an HTML from the same org file
|
|
in the future.
|
|
I still use =nix= to pin my pandoc version.
|
|
The static site builder itself is very simple, very stable and still
|
|
very efficient.
|
|
|
|
As a conclusion, if you want to write your own static site builder that's great.
|
|
There are plenty of things to learn along the way.
|
|
Still if you want something stable for a long time, with a minimal amount
|
|
of dependencies, I think this Makefile is really a great start.
|