Projects

A project is a collection of resources that define node configurations. Projects contain specs. When a node starts, it processes and runs a sequence of specs to configure the node.

Azure CycleCloud uses projects to manage clustered applications, such as batch schedulers. In the CycleCloud HPCPack cluster, the project uses hn and cn specs to define the configurations and recipes for the HPCPack headnode and compute node.

In the following partial node definition, the docker-registry node runs three specs: the bind spec from the Okta project version 1.3.0, and the core and registry specs from the docker project version 2.0.0:

[[node docker-registry]]
    Locker = base-storage
    [[[cluster-init okta:bind:1.3.0]]]
    [[[cluster-init docker:core:2.0.0]]]
    [[[cluster-init docker:registry:2.0.0]]]

The trailing tag is the project version number:

[[[cluster-init <project>:<spec>:<project version>]]]

A locker is a reference to a storage account container and credential. Nodes have a default locker, so you don't always need to specify this attribute.

Azure CycleCloud uses a shorthand for storage accounts. For example, you can write https://mystorage.blob.core.windows.net/mycontainer as az://mystorage/mycontainer.

If you define a project on a node but it doesn't exist in the expected storage location, the node reports a Software Installation Failure to CycleCloud.

CycleCloud has internal projects that run by default on all nodes to perform special volume and network handling and setup communication to CycleCloud. The system automatically mirrors these internal projects to the locker.

You're responsible for mirroring any extra projects to the locker. The CycleCloud CLI offers methods to compose projects:

cyclecloud project init myproject

And, mirror projects to lockers:

cyclecloud project init mylocker

Specs consist of Python, shell, or PowerShell scripts.

Create a New Project

To create a new project, use the CLI command cyclecloud project init myproject where myproject is the name of the project you want to create. myproject has one default spec that you can change. The command creates the directory tree with skeleton files that you update with your own information.

Directory structure

The project command creates the following directories:

      myproject
          ├── project.ini
          ├── blobs
          ├── templates
          ├── specs
          │   ├── default
          │     └── cluster-init
          │        ├── scripts
          │        ├── files
          │        └── tests

The templates directory holds your cluster templates, while specs contains the specifications that define your project. The specs directory has a subdirectory named cluster-init (but see also Chef Orchestration). The cluster-init directory contains directories with special meanings, including the scripts directory (which contains scripts that run in lexicographical order on the node), files (which contains raw data files that go on the node), and tests (which contains tests that run when you start a cluster in testing mode).

project.ini

project.ini is the file that contains all the metadata for your project. It can contain:

Lockers

Project contents are stored within a locker. You can upload the contents of your project to any locker defined in your CycleCloud installation by running the command cyclecloud project upload (locker), where (locker) is the name of a cloud storage locker in your CycleCloud installation. This locker is the default target. Alternatively, you can run the command cyclecloud locker list to see what lockers are available to you. You can view details about a specific locker with cyclecloud locker show (locker).

If you add more than one locker, you can set your default locker with cyclecloud project default_target (locker), and then run cyclecloud project upload. You can also set a global default locker for projects to share by running the command cyclecloud project default locker (locker) -global.

When you upload your project contents, CycleCloud syncs the cluster-init contents to your target locker, at projects/(project)/(version)/(spec_name)/cluster-init.

Blob Download

Use project download to download all blobs referenced in project.ini to your local blobs directory. The command uses the [locker] parameter and tries to download blobs listed in project.ini from the locker to local storage. If the command can't find the files, it returns an error.

Project Setup

Specs

When you create a new project, you define one default spec. Use the cyclecloud project add_spec command to add more specs to your project.

Versioning

By default, all projects use version 1.0.0. Set a custom version as you develop and deploy projects by setting version=x.y.z in the project.ini file.

For example, if the locker_url is az://my-account/my-container/projects, the project name is "Order66", the version is 1.6.9, and the spec is default, the URL is:

az://my-account/my-container/projects/Order66/1.6.9/default

Blobs

There are two types of blobs: project blobs and user blobs.

Project Blobs

Project authors provide project blobs, which are binary files that they can distribute (for example, binary files for an open-source project that someone can legally redistribute). Project blobs go into the blobs directory of a project and are located in /project/blobs when you upload them to a locker.

To add blobs to projects, add the files to your project.ini:

[[blobs optionalname]]
  Files = projectblob1.tgz, projectblob2.tgz, projectblob3.tgz

Separate multiple blobs with a comma. You can also specify the relative path to the project's blob directory.

User blobs

User blobs are binary files, such as Univa Grid Engine binaries, that the author of the project can't legally redistribute. These files are not packaged with the project. Users must manually stage them to their locker. The files are located in /blobs/my-project/ in the locker (for example, /blobs/my-project/my-blob.tgz). You don't need to define user blobs in project.ini.

To download any blob, use the jetpack download command. CycleCloud looks for the user blob first and uses the project-level blob if it can't locate the file.

Specify projects within a cluster template

Specs are defined in your cluster template, using the [[[cluster-init]]] section on a node:

[[node defaults]]
[[[cluster-init my-project:common:1.0.0]]]

[[node scheduler]]
[[[cluster-init my-project:scheduler:1.0.0]]]

[[nodearray execute]]
[[[cluster-init my-project:execute:1.0.0]]]

This example takes advantage of the defaults node definition that all nodes inherit from. The scheduler node gets both the common and scheduler specs, and nodes in the execute node array get both the common and execute specs.

File locations

The node downloads the cluster-init files to /mnt/cluster-init/(project)/(spec)/. For my-project and my-spec, your scripts, files, and tests are in /mnt/cluster-init/my-project/my-spec.

Sync projects

You can sync CycleCloud projects from mirrors into cluster local cloud storage. Set a SourceLocker attribute on a [cluster-init] section within your template. The name of the locker you specify is the source of the project, and the contents sync to your locker when the cluster starts. You can also use the name of the locker as the first part of the cluster-init name. For example, if the source locker is cyclecloud, the following two definitions are the same:

[cluster-init my-project:my-spect:1.2.3]
  SourceLocker=cyclecloud

[cluster-init cyclecloud/my-proect:my-spec:1.2.3]

Large file storage

Projects support large files. At the top level of a newly created project, you see a blobs directory for your large files (blobs). Blobs you place in this directory serve a specific purpose and act differently than the items within the files directory.

Items within the blobs directory act independently of specs and versions. You can share anything in blobs between specs or project versions. For example, you can store an installer for a program that changes infrequently within blobs and reference it within your project.ini. As you iterate on versions of your project, that single file remains the same and copies to your cloud storage once, which saves on transfer and storage costs.

To add a blob, place a file into the blobs directory, and edit your project.ini to reference that file:

[blobs]
  Files=big_file1.tgz

When you use the project upload command, it transfers all blobs referenced in project.ini to cloud storage.

Log files

Log files generated when running cluster-init are located in $JETPACK_HOME/logs/cluster-init/(project)/(spec).

Run files

When a cluster-init script runs successfully, it places a file in /mnt/cluster-init/.run/(project)/(spec) to ensure the script doesn't run again on a subsequent converge. To run the script again, delete the appropriate file in this directory.

Script directories

When CycleCloud executes scripts in the scripts directory, it adds environment variables to the path and name of the spec and project directories:

CYCLECLOUD_PROJECT_NAME
CYCLECLOUD_PROJECT_PATH
CYCLECLOUD_SPEC_NAME
CYCLECLOUD_SPEC_PATH

On Linux, a project named test-project with a spec of default has the following paths:

CYCLECLOUD_PROJECT_NAME = test-project
CYCLECLOUD_PROJECT_PATH = /mnt/cluster-init/test-project
CYCLECLOUD_SPEC_NAME = default
CYCLECLOUD_SPEC_PATH = /mnt/cluster-init/test-project/default

Run Scripts Only

To run only cluster-init scripts, use the following command:

jetpack converge --cluster-init

The command sends its output to STDOUT and to jetpack.log. Each script's output is also logged to:

      $JETPACK_HOME/logs/cluster-init/(project)/(spec)/scripts/(script.sh).out

jetpack download

To download a blob within a cluster-init script, use the command jetpack download (filename) to pull it from the blobs directory. Running this command from a cluster-init script lets it determine the project and base URL for you. To use it in a non-cluster-init context, you need to specify the project. For more information, use the --help option.