execvm

Source

There was some overlap between ursula and ytree so I pulled the common bits into a separate project, metrologyf. He standardizes my YAML serialization and lua parsing and supplies libraries for assorted bits like merging dictionaries and loading environment variables.

I also revived execvm, which in some ways is where I started from. He emerged in winter a few years ago and evolved a bit before budding like ytree this month. The current CLI execvm.ysh is like ytree.tar but for a stream of command descriptions, executing them in sequence rather than producing tarchive output.

$ echo '!execvm/posix/command/v1 {argv: [echo, hello, world]}' \
      | python -m execvm.ysh
hello, world

Commands can be defined statically in YAML or through a python API a la ytree, which is also exposed through ursula.compile. I started this example in the ytree post. In that case I had a shell script that defined a version monitoring pipeline. This is a lua script that generates a sequence of commands (an execvm script) equivalent to the original shell script.

local execvm = require("execvm")
local ursula = require("ursula")

return function (values)

   domain = ursula.env("DOMAIN", values.domain)
   url = domain .. "/status.yaml"

   ursula.execv {
      argv = {"curl", "-s", url},
      stdout = execvm.posix.command {
         argv = {"python", "-m", "ytree.tar", "-c"},
         stdout = execvm.posix.command {
            argv = {"tar", "-Oxf", "-", "version.yaml"},
            stdout = execvm.posix.command {
               argv = {"python", "-m", "ytree.tar", "-c"},
               stdout = execvm.posix.command {
                  argv = {"tar", "-Oxf", "-", "version"}
               }
            }
         }
      }
   }

end

The determination of that URL happens at ursula.compile runtime and gets baked into the generated script.

$ DOMAIN=http://localhost:8000 \
      python -m ursula.compile --values values.yaml monitor
--- !execvm/posix/command/v1
argv: [curl, -s, 'http://localhost:8000/status.yaml']
cwd: null
env: {}
stderr: !execvm/posix/stream/v1 stderr
stdout: !execvm/posix/command/v1
  argv: [python, -m, ytree.tar, -c]
  cwd: null
  env: {}
  stderr: !execvm/posix/stream/v1 stderr
  stdout: !execvm/posix/command/v1
    argv: [tar, -Oxf, '-', version.yaml]
    cwd: null
    env: {}
    stderr: !execvm/posix/stream/v1 stderr
    stdout: !execvm/posix/command/v1
      argv: [python, -m, ytree.tar, -c]
      cwd: null
      env: {}
      stderr: !execvm/posix/stream/v1 stderr
      stdout: !execvm/posix/command/v1
        argv: [tar, -Oxf, '-', version]
        cwd: null
        env: {}
        stderr: !execvm/posix/stream/v1 stderr
        stdout: !execvm/posix/stream/v1 stdout

That's a little verbose but it's just a stream of commands so it can pipe right into execvm.ysh. The pipeline itself can also be captured as an execvm command.

--- !execvm/posix/command/v1
argv: [python, -m, dynastyf.ursula, --values, values.yaml, monitor]
stdout: !execvm/posix/command/v1
  argv: [python, -m, execvm.ysh]

All the referenced files are in the blog source and included directly into this page for these snippets. The files/ directory at the root of the repository contains these entries.

monitor/init.lua
monitor/init.sh
monitor/init.yaml
status.yaml
values.yaml
version.yaml

I can point DOMAIN at localhost to run against the nikola serve instance that's running while I write or run without the override, which defaults to polling from the website. That's functional but it's still assuming there's a shell for running everything so I still need one more layer of abstraction to glue it all together. And there's a weird hidden hardcoded URL in there still, but don't look too much at that. Or do, you made it this far!

#!/bin/sh

python -m execvm.ysh -C files files/monitor/init.yaml