Zola Performance degraded on Alpine Linux (musl)
Having recently switched my daily driver computers to Alpine Linux, and the performance and computing enjoyment gains being fantastic, I have found the performance on some applications, Zola specifically, can be quite drasticly poor. Here's what I did to solve it. (It's a very simple solution)
Zola
Zola is a Static Site Generator (SSG), similar to Hugo or Pelican, which I use for most websites I deal with, both professionally and for personal sites.
It has always been quick to generate a site, an example site I manage has 244 pages in 58 sections and generates in about 3 seconds on an Intel i7 Gen 6 (circa 2015/2016) and when running in server mode, rebuilds the site in about the same amount of time whenever a change is made, though zola serve can always be run in with the --fast option, which only rebuilds the minimum required on a change i.e. the page currently loaded in your web browser, which can be much quicker.
Either way, for me, 3 seconds is acceptable for development purposes.
Alpine Linux
Alpine Linux is a "lightweight Linux distribution based on musl libc and busybox" - I like it because it's lightweight, and doesn't use systemd (systemd is becoming bloatware, quite frankly). It's quick, robust, and secure, and allows you to build exactly the type of system you want as its barebones installation is bare.
Coming from an Arch Linux and FreeBSD background, it's exactly what I want and something I can depend on.
Performance
Despite both Zola and Alpine being brilliantly fast and efficient software, when combined I noticed a distinct speed reduction with Zola's build time.
The aforementioned 244 page, 58 section site is suddenly taking 30 seconds to build, and changes when run in serve mode about the same. With the --fast flag, changes still take about 10 seconds to update. Definitely not acceptable.
I checked some older versions of Zola as I hadn't actually built this site in some time; maybe there's been a change I need to cater for? Nope. Still poor performance.
I wondered if it might be my storage devices failing, but everything else is running fine. Still, I spun up an Arch Linux chroot in my running Alpine system, installed Zola in there and tried building the site in there - it built fine! Back to about 3 seconds build time.
Don't waste CPU cycles!
I did a bit of searching and discovered one thing related - Zola's get_section built-in has a metadata_only boolean flag which, when set to true, only pulls the metadata of the section and not the subpages within that section. Using this flag in places where getting the whole subpage list is not required brings the build time down from 30 seconds to 7.3 seconds.
That's a lot better, but still not ideal. It's a code efficiency change really, one I probably should have been doing anyway. There's still a performane issue somewhere.
Memory Allocator woes
Further research strongly points the finger as musl's memory allocator being much, much less efficient that glibc's (apologies but I cann0t find the article I read this on) and recommendations are to use mimalloc or scudo as the memory allocator.
I'm only really a code brawler - throwing stuff together to make things work, so looking at this scared me into thinking I'd have to learn some new things:
- Am I going to need to edit the Zola source code to use mimalloc or scudo and see if that actually makes a difference?
- If it works, what if Zola don't want this change? I'm going to have to either write some code that looks good, or ask one of the Zola community to do it for me, which feels a bit of an ask for an otherwise functional project.
Turns out I didn't need to be so concerned.
mimalloc is the choice I went for. Yes, it is a Microsoft project which brings many concerns for the ongoing support of it, but for now it can be useful.
All I need to do is install the mimalloc package and preload it before running zola:
# Install mimalloc
> apk add mimalloc
# Preload it before running Zola
> LD_PRELOAD=/usr/lib/libmimalloc.so.2.2 zola serve
And with that, the same 244 page, 58 section website that was building in 30 seconds, reduced to 7.3 seconds with some efficiency tweaks, now builds in 969ms - less than 1 second!
I am very happy with those numbers, and thus my woes were gone.
Although avoidable, this is why I like systems like Arch Linux, FreeBSD and Alpine. Yes you have to build your own system which can be troublesome sometimes, but once you've done it you've learned something you can use again and again, and possibly use in similar situations in the future.
[Update] Hardened-malloc
I mentioned my worries about using mimalloc with it being a Microsoft project, and as it happens I've also tested with Alpine's hardened-malloc package (available in Edge testing repository) which may actually be the better option.
Firstly, it's a security-focused memory allocator so provides substantial hardening against some vulnerabilities. Secondly, it appears to be from the GrapheneOS team who I trust a whole lot more than Redmond.
With hardened-malloc the build time isn't quite so good at 2.3 seconds, but satisfactory for what I need it for. I think I'm going to use hardened-malloc for now.
# Install Hardened Malloc (requires testing repository on Alpine Edge)
> apk add hardened-malloc@testing
# Preload before running Zola
> LD_PRELOAD=/usr/lib/libhardened_malloc.so zola serve
Even better, Alpine do suggest putting export LD_PRELOAD=/usr/lib/libhardened_malloc.so into your /etc/profile or other system startup configurations so it's available to you without having to enter it every time you run the application. You could do the same with mimalloc, but I understand it is not as safe and should done at your own risk.
[Update 2] Hardened-malloc-light
Ha! Just found the hardened-malloc package contains /usr/lib/libhardened_malloc-light.so which when used instead results in the same site building in 1.5 seconds.
> LD_PRELOAD=/usr/lib/libhardened_malloc-light.so zola serve
Computer now goes brrrrrrrrrrrrrr