unpacking squashfs

I've been thought experimenting different operations based on squashfs packaging this week, and while I've decidedly moved away from that implementation I wanted to capture some thoughts around it.

The idea of serving directly from an image file became a bit of focus. I didn't want to deal with mount points, nor did I want to unpack files, regardless of archive format. That led me to look more into a potential nginx plugin, but squashfs-tools doesn't expose a library so that'd be extra work. I found squashfs-tools-ng, which does offer a library, but it's not in the community managed Arch repos.

This brought me back to an idea I had ages ago to write a python wrapper for libext2fs. I'm authoring in Nikola, so I could theoretically tap into the build system to enumerate which files are meant to be served (using the logic that drives nikola check for instance). Then I could, in the same process, act as my own virtual filesystem, populating files in an ext2 image from what's on disk.

On the consumption side that opens up two possibilities. One is to stick with the nginx plugin example. libext2fs.so is generally available, so I don't have the same concerns about that as with libsquashfs.so. Alternatively, I could have a small independent web server that serves HTTP on a local domain socket, reading from some image file containing an ext2 filesystem. That frees up the nginx dependency, so the architecture would be "some proxy" serving requests to the domain name from that domain socket.

Development to deployment would start with checking the output in the local nikola serve dev server. Once good I'd pack an ext2 image file and upload to a production server. I think I'd have a system service unit template there for serving images on predictable socket names, so after upload I could do something like start serve@image1. That would let me ssh port forward from a local socket to that domain socket to do a staging preview without making the site publicly visible. Once that's acceptable I'd finally have to push an updated nginx config, reload that process, and enable serve@image1 so that he survives a reboot. The nginx config should be templatable, taking the name of an image that's mean to be production and producing the proxy pass directives to forward to a similarly named socket.

I think a future improvement from there would be to sign and push the image files to an object store rather than hosting them on the server itself. Instead I would verify the signature then use qemu-nbd to expose a domain socket on the server forwarding reads to the image in the object store, which could then theoretically even be qcow2 formatted. The "e2httpd" service would need a little upgrade to support an NBD client-side IO manager rather than reading directly from a raw image file.