ytree
Winter called for more children. This week's is named ytree! His heritage traces back through some earlier projects, and this week I felt a compulsion to give him new shape.
Arch packaging uses MTREE
as a programmable archiving format. I
built on the concept by switching from the bespoke line based format
to a YAML encoding with an archive entry per document. A complete
archive is serialized from a stream of independent documents.
--- !ytree/tar/dir/v1 name: ./foo --- !ytree/tar/file/v1 name: ./foo/bar contents: !ytree/res/file/v1 path: files/ytree/example.yaml --- !ytree/tar/file/v1 name: ./hello/world contents: !ytree/res/buffer/v1 contents: !!binary aGVsbG8sIHdvcmxk --- !ytree/tar/file/v1 name: ./hello/world contents: !ytree/res/string/v1 value: hello, world
The file resource type produces the most familiar behavior, populating archive contents from local files. I also added content providers that allow for specifying object data inline, rendered from a template, or fetched remotely.
All of the mapped types have public python APIs so they can be generated programmatically. A function like this could encapsulate a version number on the fly. That could be transformed directly into a tarchive or serialized to YAML and served elsewhere.
# version.yaml --- !ytree/tar/file/v1 name: version contents: !ytree/res/string/v1 value: decorating
I could build a sort of tree of archives with indirection. The first file could represent some set of metrics each named by a path and this second one a catalog with the particular "database" name given by the path for dispatching.
# status.yaml --- !ytree/tar/file/v1 name: version.yaml contents: !ytree/res/url/v1 url: https://yieldsfalsehood.com/version.yaml
That evaluates to a tarchive with one entry populated by a snapshot of
that URL. I can access a specific configuration item in there by
deferring to tar
to "extract" the database (version.yaml
)
first then the item stored as a file (version
) inside it.
#!/bin/bash set -euo pipefail domain="${DOMAIN:-https://yieldsfalsehood.com}" url="$domain"/status.yaml curl -s "$url" \ | python -m ytree.tar -c \ | tar -Oxf - version.yaml \ | python -m ytree.tar -c \ | tar -Oxf - version
There's currently unoccupied space to configure SSL trust for those types of resources. So far I do have runtime configurable support for most jinja template loaders. Template resources themselves bind context data to a template name, which are resolved at runtime. There's no telling what this might look like when rendered!
--- !ytree/tar/file/v1 name: foo contents: !ytree/res/template/v1 template: template context: name: world
There is effectively no loader by default, which is fine if you never reference a template. If you do then you need glue, like this ad hoc loader that embeds a template body into the command line so you can see the secret message.