spiro-deploy

spiro-deploy is the first-party bundling and deployment tool for use with SpiroFS. It's meant to handle standard use cases without too many specialized frills.

Installation

There are two ways to install spiro-deploy: * The PyPI package spiro-deploy (eg, pip install spiro-deploy) * The container registry.gitlab.com/spirostack/spiro-deploy:master (Also :v0-2, :v0-3, etc)

Configuration

spiro-deploy accepts core configuration options via environment variables:

  • SPIRO_URL: The URL that the SpiroFS server is running on (default: https://salt:4510/)
  • SPIRO_TOKEN: The authentication token given by eg the spiro.issue_token runner

Usage

$ spiro-deploy [options] <project name> <deployment name>

In addition to the options described below, --help will give a useful overview.

Options

By default, spiro-deploy will bundle everything found in the _salt directory as the environment. For example, if you have _salt/spam.sls, spiro-deploy will bundle this as salt://spam.sls. Additionally, if git is installed, a .gitcommit file is added; the first line is the commit hash and the second line is the output of git describe.

  • --artifact: Add the given file to an _artifacts directory inside the bundle. For example, --artifact=build/eggs will appear as salt://_artifacts/build/eggs

  • --include-source: Add the entire source code under _source. This optional takes a method to discover the files:

    • filesystem/fs: Adds any files in the current directory that are not in _salt and not VCS metadata (.git is not added)
    • git: Adds the contents of git archive

  • --highstate/--no-highstate: Enables/disable the server automatically highstated affected minions on deployment. Default is to do so.

  • --server, --token: Override SPIRO_URL/SPIRO_TOKEN (see above).
  • --ssl-cert: Instead of standard certificate authorities, use this certificate to authenticate the server.
  • --insecure: Just don't authenticate the server (also allows unencrypted HTTP to be used). This is dangerous and not recommended in production.

Danger

As stated previously, it is strongly recommended that HTTPS is used with a validated certificate. Using a CA-issued certificate is much simpler to configure than making your own CA or using self-signed certificates.

Deploy Protocol

The deploy protocol is based on Server Sent Events, which is used to stream updates about the deploy process. The full protocol is described in the source. If developing your own deploy client, it is suggested that you show these updates to the user as they are received.

If developing your own client from scratch is not to your liking, spiro-deploy is also usable as a library.

Library API

spirodeploy.client

This does the actual networking. The only thing important to users is upload():

  • url, token: The server URL and authentication token
  • tarball: A file-like object containing the tar bundle (see builder below)
  • project, deployment: The project/deployment to apply the bundle to
  • highstate: If the server should highstate minions.
  • sslverify: True to do standard verification, False for insecure mode (see warnings above), or a string of the certificate file to verify with.
  • connect_timeout: How long (in seconds) to wait for the server to accept the connection.

A generator is returned, yielding event/data pairs of the updates (see the protocol documentation).

spirodeploy.builder

Handles building the tarball to give to the client. Major contents are the class TarballBuilder.

Sample (from cli.py):

with TarballBuilder() as builder:
    try:
        builder.add_gitcommit()
    except FileNotFoundError:
        print("Warning: No git binary", file=sys.stderr, flush=True)
    if os.path.exists('_salt'):
        builder.add_saltdir('_salt')
    for a in args.artifact:
        builder.add_artifact(a, a)
    if args.include_source is not ...:
        builder.add_source(args.include_source)
  • .buffer: A BytesIO of the tarball.
  • .__enter__()/.__exit__() (context manager): Manges the buffer and the data structures building it. .buffer does not exist before entering, and it should not be used until after exit. Most of the rest of the methods can only be used inside the context manager.
  • .add_saltdir(path): Just add a directory to the bundle root.
  • .add_artifact(path, name): Add a file or directory (path) to the bundle under _artifacts as name.
  • .add_virtual(name, data): Add a blob of data (as bytes) to the bundle as name.
  • .add_gitcommit(): May add a file .gitcommit containing the commit hash and the output of git describe. Raises FileNotFound if git is unavailable. The file is not added if this information is not applicable.
  • .add_source(scanner): Add source under _source, using the given scanner. Two scanners are pre-defined:
    • spirodeploy.builder.git_scanner: Uses the output of git archive
    • spirodeploy.builder.fs_scanner: Walks the file system

Scanners

Scanners are callables that return iterables--they're generators. They accept a single argument: should_skip, a callable that indicates if a given file/directory should be skipped (may return False on descendents). They produce name/callable pairs:

  1. The name of the file inside of the bundle
  2. A callable to open the file: Takes no arguments and return a binary readable file-like.