Private Cloud on a Compute Cluster

Introduction

pcocc (pronounced like "peacock") stands for Private Cloud On a Compute Cluster. It allows users of an HPC cluster to host their own clusters of VMs on compute nodes, alongside regular jobs. Users are thus able to fully customize their software environments for development, testing, or facilitating application deployment. Compute nodes remain managed by the batch scheduler as usual since the clusters of VMs are seen as regular jobs. For each virtual cluster, pcocc allocates the necessary resources to host the VMs, including private Ethernet and/or Infiniband networks, creates temporary disk images from the selected templates and instantiates the requested VMs.

Working principle

pcocc leverages SLURM to start, stop and supervise virtual clusters in the same way as regular parallel jobs. It allocates CPU and memory resources using sbatch/salloc and a SLURM plugin allows to setup virtual networks on the allocated nodes. Once the nodes are allocated and configured, VMs are launched by SLURM as any other task with the rights of the invoking user. VMs are configured to replicate, as much as possible, the resources and capabilities of the portion of the underlying host that is allocated for them (CPU model and core count, memory amount and NUMA topology, CPU and memory binding...) so as to maximize performance.

To launch a virtual cluster, the user selects a template from which to instantiate its VMs and the number of requested VMs (it is possible to combine several templates among a cluster). A template defines, among other things, the base image disk to use, the virtual networks to setup, and optional parameters such as host directories to export to the VMs via 9p. Administrators can define system-wide templates from which users can inherit to define their own templates. When a VM is instantiated from a template, an ephemeral disk image is built from the reference image using copy-on-write. By default, any changes made to the VMs' disks are therefore lost once the virtual cluster stops but it is possible to save these changes to create new revisions of the templates.

List of help topics

This documentation is organized into help topics which are listed in the following sections. These topics include tutorials to help you get started, individual pcocc sub-commands to manage and interact with virtual clusters and configuration files.

You may get further information on each of these topics listed below by doing:

pcocc help [TOPIC]

For example to open the newvm tutorial:

pcocc help newvm-tutorial

To example read help about the ssh sub-command:

pcocc help ssh

For installing pcocc on a cluster, have a look at the installation guide. [1]

Sub-Commands

pcocc supports the following sub-commands:

  • Define and Allocate VMs:

    alloc

    Instantiate or restore a virtual cluster (interactive mode)

    batch

    Instantiate or restore a virtual cluster (batch mode)

    template

    List and manage VM templates

    image

    List and manage VM images

  • Connect to VMs:

    console

    Connect to a VM console

    nc

    Connect to a VM via nc

    scp

    Transfer files to a VM via scp

    ssh

    Connect to a VM via ssh

    exec

    Execute commands through the pcocc guest agent

    display

    Display the graphical output of a VM

  • Manage running VMs:

    reset

    Reset a VM

    ckpt

    Checkpoint a virtual cluster

    dump

    Dump the memory of a VM to a file

    monitor-cmd

    Send a command to the monitor

    save

    Save the disk of a VM

    ps

    List current pcocc jobs

Tutorials

newvm-tutorial
How to import VM images and define VM templates
cloudconfig-tutorial
How to configure cloud-init enabled VMs
9pmount-tutorial
How to mount host directories in VMs

Configuration Files

batch.yaml
Batch environment configuration file
networks.yaml
Networks configuration file
resources.yaml
Resource sets configuration file
repos.yaml
Image repositories configuration file
templates.yaml
VM templates definition file