A bit of re-reading my article.

This commit is contained in:
Yann Esposito (Yogsototh) 2021-09-01 14:33:37 +02:00
parent 09b0af122a
commit 2acd6effce
Signed by untrusted user who does not match committer: yogsototh
GPG Key ID: 7B19A4C650D59646
1 changed files with 53 additions and 54 deletions

View File

@ -8,17 +8,17 @@
#+OPTIONS: auto-id:t #+OPTIONS: auto-id:t
#+STARTUP: showeverything #+STARTUP: showeverything
This article will dig a bit deeper about how I generate my static website. This article will dig a bit deeper about my =Makefile= based static website generator.
In a [[https://her.esy.fun/posts/0017-static-blog-builder/index.html][previous article]] I just gave the rationale and an overview to do it In a [[https://her.esy.fun/posts/0017-static-blog-builder/index.html][previous article]] I just gave the rationale and an overview to do it
yourself. yourself.
Mainly it is very fast and portable. Mainly it is very fast and portable.
A few goal reached by my current build system are: A few goals reached by my current build system are:
1. Be fast, try to make as few work as possible. 1. Be fast and make the minimal amount of work as possible.
I don't want to rebuild all the html pages if I only change one file. I don't want to rebuild all the html pages if I only change one file.
2. Source file format agnostic. You can use markdown, org-mode or even 2. Source file format agnostic. You can use markdown, org-mode or even
directly writing html. directly write html.
3. Support gemini 3. Support gemini
4. Optimize size: minify HTML, CSS, images 4. Optimize size: minify HTML, CSS, images
5. Generate an index page listing the posts 5. Generate an index page listing the posts
@ -38,7 +38,7 @@ To make those transformations I use very short a shell scripts.
:CUSTOM_ID: -makefile--overview :CUSTOM_ID: -makefile--overview
:END: :END:
A Makefile is constitued of rules. A Makefile is made out of rules.
The first rule of your Makefile will be the default rule. The first rule of your Makefile will be the default rule.
The first rule of my Makefile is called =all=. The first rule of my Makefile is called =all=.
@ -50,15 +50,14 @@ target: file1 file2
--output target --output target
#+end_src #+end_src
if =target= does not exists, then =make= will look at its if =target= does not exists, then =make= will look at its dependencies.
dependencies. If any of its dependencies need to be updated, it will run all the rules in
If any of its dependency need to be updated, it will run all the rules in the correct order to rebuild them and finally run the script to build
the correct order to rebuild them, and finally run the script to build
=target=. =target=.
A file need to be updated if one of its dependency need to be updated or is A file needs to be updated if one of its dependency needs to be updated or is
newer. newer.
The ususal case of =make= is about building a single binary out of many The usual use case of =make= is about building a single binary out of many
source files. source files.
But for a static website, we need to generate a lot of files from a lot of But for a static website, we need to generate a lot of files from a lot of
files. files.
@ -105,7 +104,7 @@ I have a block for:
The rules to copy assets will be a good first example. The rules to copy assets will be a good first example.
1. find all assets in =src/= directory 1. find all assets in =src/= directory
2. generate all assets from these file in =_site/= directory 2. generate all assets from these files in =_site/= directory
3. make this rule a dependency on the =all= rule. 3. make this rule a dependency on the =all= rule.
@ -123,44 +122,44 @@ ALL += assets
OK, this looks terrible. OK, this looks terrible.
But mainly: But mainly:
- ~SRC_ASSETS~ will contains the result of the command ~find~. - ~SRC_ASSETS~ will contain the result of the command ~find~.
- ~DST_ASSETS~ will contains the files of ~SRC_ASSETS~ but we replace the - ~DST_ASSETS~ will contain the files of ~SRC_ASSETS~ but we replace the
=src/= by =_site/=. =src/= by =_site/=.
- We create a generic rule; for all files matching the following pattern - We create a generic rule; for all files matching the following pattern
=_site/%=, look for the file =src/%= and if it is newer (in our case) =_site/%=, look for the file =src/%= and if it is newer (in our case)
then execute the following commmands: then execute the following commands:
- create the directory to put =_site/%= in - create the directory to put =_site/%= in
- copy the file - copy the file
About the line ~@mkdir -p "$(dir $@)"~: About the line ~@mkdir -p "$(dir $@)"~:
- the =@= at the start of the command simply means that we make this execution silent. - the =@= at the start of the command simply means that we make this execution silent.
- The =$@= is replaced by the target string. - The =$@= is replaced by the target string.
- And =$(dir $@)= will generate the dirname of =$@=. - And =$(dir $@)= will generate the folder name of =$@=.
For the line with ~cp~ you just need to know that =$<= will represent the For the line with ~cp~, you just need to know that =~$<~= will represent the
first dependency. first dependency.
So my Makefile is composed of similar blocks, where I replace the first My Makefile is composed of similar blocks, where I replace the first
find command to match specific files and where I use different building rule. find command to match specific files and where I use different building rules.
An important point, is that the rule must be the most specific possible An important point is that the rules must be the most specific possible.
because make will use the most specific rule in case of ambiguity. This is because =make= will use the most specific rule in case of ambiguity.
So for example, the matching rule =_site/%: src/%= will match all files in For example, the matching rule =_site/%: src/%= will match all files in
the =src/= dir. the =src/= dir.
But if we want to treat css file with another rule we could write: But if we want to treat =CSS= files with another rule we could write:
#+begin_src makefile #+begin_src makefile
_site/%.css: src/%.css _site/%.css: src/%.css
minify "$<" "$@" minify "$<" "$@"
#+end_src #+end_src
And if the selected file is a css file, this rule will be selected. And if the selected file is a =CSS= file, this rule will be selected.
** Prelude ** Prelude
:PROPERTIES: :PROPERTIES:
:CUSTOM_ID: prelude :CUSTOM_ID: prelude
:END: :END:
So to start I have a few predefined useful variables. I start with variables declarations:
#+begin_src makefile #+begin_src makefile
all: site all: site
@ -183,7 +182,7 @@ NO_SRC_FILE := ! -name '*.org'
:CUSTOM_ID: css :CUSTOM_ID: css
:END: :END:
So here we go, the same simple pattern for CSS files. Here we go; the same simple pattern for CSS files.
#+begin_src makefile #+begin_src makefile
# CSS # CSS
@ -280,16 +279,15 @@ or the file, it will generate the dependencies.
:CUSTOM_ID: indexes :CUSTOM_ID: indexes
:END: :END:
One of the goal I have is to be as agnostic as possible regarding format. We often need indexes to build a website.
I know that the main destination format will be html. Typically to list the latest articles, build the RSS file.
So as much as possible, I would like to use this format. So for sake of simplicity, I decided to build my index as a set of XML files.
So for every generated html file I will generate a clean XML file (via Of course, this could be optimizide, by using SQLite for example.
hxclean) so I will be able to get specific node of my HTML files. But this will already be really fast.
These XML files will constitute my "index".
Of course this is not the most optimized index (I could have used sqlite For every generated html file I will generate a clean XML file with
for example) but it will already be quite helpful as the same index files =hxclean=.
will be used to build the homepage with the list of articles, and the RSS Once cleaned, it will be easy to access a specific node of in these XML files.
file.
#+begin_src makefile #+begin_src makefile
# INDEXES # INDEXES
@ -308,24 +306,23 @@ indexcache: $(DST_XML_FILES)
ALL += indexcache ALL += indexcache
#+end_src #+end_src
So to resume this rule will generate for every file in =site/posts/*.html= This rule will generate for every file in =site/posts/*.html= a corresponding
a corresponding =xml= file (=hxclean= takes an HTML an try its best to make =xml= file (=hxclean= takes an HTML an try its best to make an XML out of it).
an XML out of it).
** HTML Index ** HTML Index
:PROPERTIES: :PROPERTIES:
:CUSTOM_ID: html-index :CUSTOM_ID: html-index
:END: :END:
So now we just want to generate the main =index.html= page at the root of Now we just want to generate the main =index.html= page at the root of
the site. the site.
This page should list all articles by date in reverse order. This page should list all articles by date in reverse order.
So the first step is to take advantage of the cache index. The first step is to take advantage of the cache index.
For every XML file I generated before I should generate the small HTML For every XML file I generated before I should generate the small HTML
block I want for every entry. block I want for every entry.
For this I use a script =mk-index-entry.sh=. For this I use a script =mk-index-entry.sh=.
He will use =hxclean= to retrieve the date and the title from the cached He will use =hxselect= to retrieve the date and the title from the cached
XML files. XML files.
Then generate a small file just containing the date and the link. Then generate a small file just containing the date and the link.
@ -340,7 +337,7 @@ $(INDEX_CACHE_DIR)/%.index: $(INDEX_CACHE_DIR)/%.xml $(MK_INDEX_ENTRY)
$(MK_INDEX_ENTRY) "$<" "$@" $(MK_INDEX_ENTRY) "$<" "$@"
#+end_src #+end_src
which reads, for every =.xml= file generate a =.index= file with It means: for every =.xml= file generate a =.index= file with
=mk-index-entry.sh=. =mk-index-entry.sh=.
#+begin_src sh #+begin_src sh
@ -378,7 +375,7 @@ printf ": %-55s" "$title ($keywords)"
echo " [${fg[green]}OK${reset_color}]" echo " [${fg[green]}OK${reset_color}]"
#+end_src #+end_src
Then I use these intermediate file to generate a single bigger index file. Then I use these intermediate files to generate a single bigger index file.
#+begin_src makefile #+begin_src makefile
HTML_INDEX := $(DST_DIR)/index.html HTML_INDEX := $(DST_DIR)/index.html
@ -393,13 +390,16 @@ ALL += index
#+end_src #+end_src
This script is a big one, but it is not that complex. This script is a big one, but it is not that complex.
For every file, I generate a new file =DATE-dirname=, I sort them in For every file, I generate a new file =DATE-dirname=.
reverse order and put their content in the middle of an HTML file. I sort them in reverse order and put their content in the middle of an HTML
file.
The important part is that it is only generated if the index change. Important note: this file updates only if the index change.
So first part of the script handle the creation of file using the date in
their file name which will help us sort them later.
The first part of the script creates files with the creation date in their
metadatas.
The created file name will contain the creation date, this will be helpful
later.
#+begin_src sh #+begin_src sh
#!/usr/bin/env zsh #!/usr/bin/env zsh
@ -480,11 +480,11 @@ echo "* HTML INDEX [done]"
:CUSTOM_ID: rss :CUSTOM_ID: rss
:END: :END:
So for my RSS generation this is quite similar to the system I use to My RSS generation is similar to the system I used to generate the index
generate my index file. file.
I just slightly improved the rules. I just slightly improved the rules.
The makefile blocks look like: The =Makefile= blocks look like:
#+begin_src makefile #+begin_src makefile
# RSS # RSS
@ -553,7 +553,7 @@ gemini: $(DST_GMI_FILES) $(GMI_INDEX) $(GEM_ATOM)
:CUSTOM_ID: images :CUSTOM_ID: images
:END: :END:
For images, I try to convert all of them with imagemagick to compress them. For images, I try to compress them all with imagemagick.
#+begin_src makefile #+begin_src makefile
# Images # Images
@ -586,7 +586,6 @@ ALL += $(DST_IMG_FILES)
:END: :END:
A nice bonus is that I also deploy my website using make. A nice bonus is that I also deploy my website using make.
And note I protect myself from Makefile temporary bugs for the =clean= rule.
#+begin_src makefile #+begin_src makefile
# DEPLOY # DEPLOY