OCCAM works differently from most HPC facilities: application software is autonomously packaged by users in containers, which are then run as isolated Virtual Workstations or Virtual Clusters.
The OCCAM user interface is based on GitLab; we encourage use of the OCCAM GitLab repository for your code, but you need only to use the image registry if you don't want to do so.
An example project is provided on OCCAM's GitLab instance that can be used as a starting point for building your own project. Please find the project at https://gitlab.c3s.unito.it/bagnasco/docker-example
Please note that directly running a container is currently supported only for single-node, single-step computations. Multi-node MPI applications and pipelines need some more massaging.
Please note, again, that this is not a full Docker tutorial but only a step-by-step example on how to build, upload and run a container on OCCAM. Please refer to the Docker official documentation for more information.
Obtaining an account
First of all, you need an account on our facility: you can ask for an account through the contact form located in this site. Are admmitted only the Researchers from the departments/centers/organizations/firms who subscribed to C3S; students are not allowed. Every Researcher should use the institutional email account. After contacting C3S staff through the web form, you will be enabled to access the gitlab service, which is the main interface to the facility. We will refer to your username in the scope of these instructions as YOURUSERNAME.
What first ?
In order to be able to connect via ssh on the access node (ssh://occam.c3s.unito.it), and for all the OCCAM machinery to know you, you are required to upload an ssh key to you user profile. Please go to https://gitlab.c3s.unito.it/profile/keys. There you will find detailed instructions, or ask to google if you are in trouble with your personal setup: there is a plenty of howto on ssh credential creation.
In OCCAM, all user software is run in containers. Even if this is not technically completely correct, you can think of a container as a bag of software that holds the full software stack you need for your computation: OS, libraries, executables, files, except the Linux kernel that is shared with the physical host on which the container runs. Of course it is possible to pass configuration files and command line arguments to executables inside the container at runtime.
There are several implementations of software containers; in OCCAM we currently use Docker containers, an industry-standard tool to build, manage and run containers on Linux. You can find the full documentation at the Docker site , e.g. here: https://docs.docker.com/learn/, but you will need only very basic knowledge of it (pointers to relevant documentation will be provided wherever needed).
Jargon alert: what you will build and upload is not a container but an image; a container is a running instance of an image, more or less like a process is a running program.
In order to build and upload the image, you will have to install Docker in your machine. Please download and install from https://store.docker.com/search?offering=community&type=edition; yes, you can build and run Linux containers on MacOS, Windows or Linux (any), and they will run on OCCAM.
The basic workflow is the following (in parenthesis the
Docker command used):
- You create a gitlab project that will store all artifacts needed for running your computation on OCCAM;
- You create (build) and test the image on your machine by writing a
Dockerfileand using standard Docker commands;
- You upload (push) the image on the OCCAM image registry located in your gitlab project (only images from our internal registry can be run on OCCAM);
- You login (login) to the OCCAM access node and copy all input files somewhere in your home directory;
- You use OCCAM special commands to run your container in the cluster.
In the following, you will be guided through these steps to build and run the example image.
Create a project on gitlab
A project on gitlab is needed to store all the artifacts, files, Dockerfile, docker images that will be required to run your computation environment. It's simple and straightforward:
- using a web browser, Sign in to https://gitlab.c3s.unito.it with the web credentials obtained from our staff;
- click on the New Project green button;
- write down a name for your project in the Project name field; we'll refer to this name in the scope of these instructions as PNAME
- for the first project, you can now got to the end of the page and click on the Create Project green button;
- all done.
Building the image
Building an image means picking a base image and adding all the pieces and configurations needed to run your applications, much like you would do on a new workstation. The base image typically contains just the OS, but there exist images with preinstalled software (for example, R). You can find official Docker images you can use as base images at the Docker Store https://store.docker.com/search?q=&source=verified&type=image or from nVidia project https://github.com/NVIDIA/nvidia-docker/wiki if you need to use our GPUs.
If the Linux distribution is not important, we suggest using CentOS 7. This is the OS of choice in this example.
First, you will need to copy all the relevant files to a directory in your machine, then write a Dockerfile to tell Docker how to build the container. In this example, the Dockerfile is provided and there is only one file, the Python script that computes the Mandelbrot set, with the rather unimaginative name of
If you're familiar with git, you can get both the files (script and Dockerfile) by cloning this project's repository:
git clone:bagnasco/docker-example.git cd docker-example
Otherwise, you can simply download them with wget or curl, or even with a browser, for example
mkdir docker-example cd docker-example wget https://gitlab.c3s.unito.it/bagnasco/docker-example/blob/master/Dockerfile wget https://gitlab.c3s.unito.it/bagnasco/docker-example/blob/master/mandelbrot.py
Take your time to peruse the Dockerfile before building; the full docker commands reference is at https://docs.docker.com/engine/reference/builder/.
Now you can build the image with
docker build -t gitlab.c3s.unito.it:5000/<YOURUSERNAME>/<PNAME> .
The last dot in the build command is mandatory !!
You will see that Docker downloads the base image, then step-by-step does all the operations described in the Dockerfile.
<username>/mandelbrot is the name you give to the image. Please replace \<YOURUSERNAME> with your OCCAM username and PNAME with your gitlab project name. Docker caches everything that can be cached, so if you change something in the Dockerfile and build again it will not, for example, download again the base image.
At this point, you can use the command
docker images to see the newly-built image (as well as all other previously built images, if any), and
docker run to test the container:
The output should be something like the following lines, where YOURUSERNAME and PNAME are replaced with the values used in the build command:
REPOSITORY TAG IMAGE ID CREATED SIZE gitlab.c3s.unito.it:5000/<YOURUSERNAME>/<PNAME> latest 43d6299dbca8 2 weeks ago 240 MB
Now let's try to run it on your desktop/laptop to see docker in action:
docker run -ti --rm --volume $PWD:$PWD --workdir $PWD gitlab.c3s.unito.it:5000/<YOURUSERNAME>/<PNAME>
In our convention, the software must read and write to the current directory from which the container is launched. This is exposed inside the container by the
--volume $PWD:$PWD --workdir $PWD options to the command. You can find the full
docker run reference at https://docs.docker.com/engine/reference/run/.
The output should be something like
Computing... Drawing... Done.
and you should find a new file
mandelbrot.png in the current directory. You can pass arguments to the script, that will override the defaults defined in the Dockerfile, by adding them to the command line:
docker run -ti --rm --volume $PWD:$PWD --workdir $PWD <YOURUSERNAME>/<PNAME> 10 10. 101 301
Uploading the image to occam registry
An image registry is a store where the images can be pushed/pulled when needed; every gitlab project own a private registry useful for storing our project images and occam is able to pull from there you "bag of tools" for doing computation. So, in order to run on OCCAM, you need to push the image to the image registry.
First you will need to login to the GitLab repository, using your OCCAM username and password:
docker login gitlab.c3s.unito.it:5000
You should see something like:
Username: <Enter you USERNAME on gitlab.c3s.unito.it> Password: <Enter you PASSWORD on gitlab.c3s.unito.it> Login Succeded
Then you can push it with a name including the registry name and push it:
docker push gitlab.c3s.unito.it:5000/<YOURUSERNAME>/<PNAME>
If everything went well, you can go to the registry page at https://gitlab.c3s.unito.it/\<YOURUSERNAME>/\<PNAME>/container_registry and you will see your image entry. It will be named "latest": you can add version tags to your containers like
<YOURUSERNAME>/<PNAME>:v1, but the "latest" tag will always be an alias for the, well, latest version built.
Running on OCCAM
Now you can login on the access node occam.c3s.unito.it with you ssh client and run your image on the cluster. If everything is set up correctly, your ssh key has already been propagated to your home directory on the system, and you should be able to log in without a password:
The only way to login into occam access node is through the use of the ssh-keys uploaded to gitlab web interface; login/password access to ssh is disabled
If the logon went ok, then you will see something like this:
Last login: Tue May 16 11:18:37 CEST 2017 on pts/1 Setup (once) your occam private registry Please login to the Occam Registry with the occam-login command before doing computations.
Following the suggestion, start using our first occam- command
The command output will be as follow:
When prompted for a username/login, please insert your gitlab credentials
Info: creating repo: /archive/home/YOURUSERNAME/.udocker Info: installing from tarball 1.0.1 username: *enter YOUR USERNAME ON gitlab.c3s.unito.it* password: *enter YOUR PASSWORD ON gitlab.c3s.unito.it*
this needs to be done only once, since it will store a token in your home dir.
If everythis went ok, create a directory to work in:
mkdir docker-example cd docker-example
Please note that for security reasons you are not allowed to directly run Docker on the system; however, some extra machinery under the hood will do so for you.
Now you can use the
occam-run command to run your container:
You will see some extra output generated by the machinery that re-builds the container in a special format and run it on one of the nodes. If you don't specify anything, like in the example above, it will use one of the "light" nodes that was set apart for testing.
You are logged-in to: Occam Registry Current Directory is: /archive/home/YOURUSERNAME/docker-example Container Image is: YOURUSERNAME/PNAME Finished building personalized image for: YOURUSERNAME Other args: LogFile will be: /archive/home/YOURUSERNAME/docker-example/node22-152708.log ErrFile will be: /archive/home/YOURUSERNAME/docker-example/node22-152708.err When job will finish, the file /archive/home/srabelli/node22-152708.done will be created Remote ID for occam-kill command parameter: node22-152708 Running detached from terminal, bye.
The Remote ID (in this case
node22-152708) is the unique handle you can use to interact with the running container. Any output to stderr and stdout will go, respectively, to
<id>.err files. When the job will finish, an additional file
<id>.done will be created along the other two files to signal that all works are done.
As soon as the container starts on the node, the occam-run command exits: "Running detached from terminal, bye." here means "done dispatching the container", not "done running". Of course in this example the container runs only for a very short time, but you can check whether it is still running or killing it with
occam-inspect occam-kill node22-152708
At the end of the execution (which in this example means almost immediately) you will again find a
mandelbrot.png file in the current directory. Of course, you can pass arguments and options to your container through the
occam-run command line:
occam-run <YOURUSERNAME>/PNAME 10 10. 1001 1001
That's all folks!
Now if you are confident on your understanding, go to OCCAM command reference and discover all the OCCAM commands parameters/functions.