expressions
I've been forging ahead with stabilizing this latest set of tools and
amending the examples in these posts as I go without stopping to think
about what changed. Most notably ursula now has a brother,
poseidon, so they inhabit a shared repository. That also means I'm
deleting the original ursula repository, a flicker of a primordial
hadron struggling to take shape before collapsing and bubbling back up
in some new form. So update your bookmarks!
I've pulled a large chunk of shared logic out of ytree and
execvm and moved it into metrologyf, where I define some
helper functions plus a small suite of abstract and base classes for
describing computations. This marks an occasion to think about what
computation even is.
In this ecosystem the value of a statement is determined by the execution of an expression, and intentionally or not our agreed use of the term "execution" seems appropriately violent for the purpose. A statement marks a pivot point of identity between one value and another, the membrane between a state transfer. But then the statement is itself an expression subject to execution, so its distinction as something different seems like a contrivance. Maybe we can resolve that with one more layer of abstraction! And if the layers of identity ever become insurmountable the easy way out is to introduce another god of identity in conflict with the other.
In most cases so far the point hasn't been all that interesting. Both
ytree.tar and execvm.ysh look like filter functions that
operate on streams of documents. For those tools, the "top level"
documents in the input stream are statements (expressions to be
executed) and their evaluation is the "typical thing", which for
execvm.ysh means deferring to execv. For ytree.tar that
means either "generate a tar-encoded stream of output" or "extract to
the filesystem". I previously only supported that first outcome, but
extraction turned out to be relatively painless to add (although it's
not entirely complete so I wouldn't trust it not to bonk your data).
I've amended the runtime arguments to look more like the typical tar. A combination of dry-run and verbosity settings also affect the operation. I don't really know file sizes ahead of time (not without executing something!) so I left room for uncertainty there.
$ python -m ytree.tar --create scripts/entries/string.yaml \ | tar -tvf - drwxr-x--- 0/0 0 1969-12-31 19:00 ./build/scripts/string -rw-r----- 0/0 12 1969-12-31 19:00 ./build/scripts/string/1 -rw-r----- 0/0 26 1969-12-31 19:00 ./build/scripts/string/2 $ python -m ytree.tar --create -nv scripts/entries/string.yaml drwxr-x--- 0/0 *************** 1969-12-31 19:00 ./build/scripts/string -rw-r----- 0/0 *************** 1969-12-31 19:00 ./build/scripts/string/1 -rw-r----- 0/0 *************** 1969-12-31 19:00 ./build/scripts/string/2 $ python -m ytree.tar --create -nvv scripts/entries/string.yaml drwxr-x--- 0/0 0 1969-12-31 19:00 ./build/scripts/string -rw-r----- 0/0 12 1969-12-31 19:00 ./build/scripts/string/1 -rw-r----- 0/0 26 1969-12-31 19:00 ./build/scripts/string/2
The evaluation loop invites each resource to call out to Some Protocol to do the actual execution work as the resources are loaded. Extraction (in this case a dry-run extraction so I have something to show you here) differs from creation only in the selection of protocol.
$ python -m ytree.tar -xn scripts/entries/string.yaml ./build/scripts/string ./build/scripts/string/1 ./build/scripts/string/2
Now it's a straightforward thing that execvm.ysh is nearly
identical, differing in both the types of resources he knows how to
input and the protocol he supplies to them.
$ python -m execvm.ysh scripts/hello.yaml hello world $ python -m execvm.ysh -n scripts/hello.yaml echo hello world $ python -m execvm.ysh -nv scripts/hello.yaml --- !execvm/posix/command/v1 argv: [echo, hello, world] cwd: null env: {} stderr: !execvm/posix/stream/v1 stderr stdout: !execvm/posix/stream/v1 stdout
One of the abstractions metrologyf provides is an orchestration
layer over lupa for wrangling lua
runtimes. His evaluation is again nearly identical but for the added
challenge of having to be explicit about synchronization. Each
endpoint exposed to lua is backed by a python function that returns
Some Object, which lets me pass those objects around in lua. In this
example, ursula.env() returns a string, execvm.posix.execv()
returns a command object, and execvm.posix.stream() returns a
stream object.
foo = ursula.env "FOO" ursula.execv { argv = {"echo", "hello", foo}, stdout = execvm.posix.execv { argv = {"cat", "-"}, stdout = execvm.posix.stream "stderr" } }
And then there's the seeming asymmetry of ursula.execv. Under the
hood he defers to the same execvm.posix.execv endpoint to
construct a command, which he executes immediately. The protocol I use
here returns the same value because gifting witches a piece of
myself just to get the same thing back sounds like my kind of magic.
def command(self, table): return command(...) ... def execv(self, table): return self.lua.protocol.execute( execvm.command(table) )
Plus it makes things easier. Since the value of ursula.execv is a
command object, it's valid to use anywhere that expects a command even
after its execution. What value in destruction!
ursula.execv { argv = {"echo", "bar"}, stdout = ursula.execv { argv = {"cat", "-"}, stdout = execvm.posix.stream "stderr" } }
This probably isn't a typical usage of the language, but I can't not wonder what could be expressed. That script first serializes the child command and then the parent, which we can say includes a copy of the child even though we all know it refers to the same object. How could it not? All the constructor parameters are the same!
$ python -m dynastyf.ursula scripts.nested --- !execvm/posix/command/v1 argv: [cat, '-'] cwd: null env: {} stderr: !execvm/posix/stream/v1 stderr stdout: !execvm/posix/stream/v1 stderr --- !execvm/posix/command/v1 argv: [echo, bar] cwd: null env: {} stderr: !execvm/posix/stream/v1 stderr stdout: !execvm/posix/command/v1 argv: [cat, '-'] cwd: null env: {} stderr: !execvm/posix/stream/v1 stderr stdout: !execvm/posix/stream/v1 stderr
I need to pass data on stdin to run that script so I can't pipe the
script itself into execvm.ysh. This is actually illuminating a
portion of the API I'd rather tweak so that I can better supply both
scripts and data without needing a filesystem, but our vocabulary
barely has morphemes. So here I'm simply using bash magic, the chip
clip at the edge of the universe holding it all together.
So far all of the ursula examples I've shown generate either
ytree or execvm objects, but there's nothing to stop her from
conjuring what you ask for, which might be some combination of both.
local ursula = require("ursula") local ytree = require("ytree") return function (values) ursula.fs.file { name = "foo", contents = ytree.res.string( string.format("hello, %s", values.domain) ) } ursula.execv { argv = {"cat", "foo"} } end
Nothing except your needing to ask her to begin with. Spells, Ursula, please!
$ python -m dynastyf.ursula \ --values files/values.yaml \ files.version --- !ytree/tar/file/v1 name: files/version.yaml contents: !ytree/res/string/v1 value: expressions --- !execvm/posix/command/v1 argv: [echo, updated version to, expressions] cwd: null env: {} stderr: !execvm/posix/stream/v1 stderr stdout: !execvm/posix/stream/v1 stdout
One thing is comfortable but it's difficult to grow without a
dialectic. Two things is fruitful but dependency cycles become
unwieldy. A trinity is a longtime crowd pleaser, and with some care
around the boundaries we can manifest even more expressive
functions. Neither execvm nor ytree has awareness of the
other's types, but both sets of types are built on abstractions
defined in metrologyf, the language that empowers and constricts
the two. So long as both are reasonably mutually exclusive it's not
all that difficult to define a union of the two, the reaction from
which poseidon emerged. He takes the same set of arguments as both
tools combined and implements a protocol that defers to the protocols
defined by each individual library. With all of that in place I can
finally write a sentence like this.