Commit 573191db authored by Paul Bethge's avatar Paul Bethge
Browse files

add more docu

parent bcbd9003
......@@ -19,7 +19,6 @@ You can use the CI/CD pipeline to submit a job that gets picked up by a GitLab R
In order to specify a job and its environment you need to have the `.gitlab-ci.yml` in the top-level of your repository.
The pipeline runs every time changes are pushed to any branch in the project.
## Minimal CPU Job
```yaml
my-first-job:
image: ubuntu:18.04
......@@ -29,42 +28,58 @@ my-first-job:
- docker
- cpu
```
### Explanation
## Minimal CPU Job
#### Jobs
In this file, you define the structure and order of jobs that the runner should execute.
Jobs are defined as entries at the top-level, here: `my-first-job`.
Jobs are defined as entries at the top-level and should not interfere with keywords, here: `my-first-job`.
Those entries contain constraints stating under what conditions the job should be executed.
Scripts may contain multiple jobs, and jobs run as part of a larger pipeline.
Imagine the scripts you add to jobs are the same as CLI commands you run on your computer.
Scripts may contain multiple jobs, and jobs run as part of a larger pipeline.
#### Docker Images
For our purposes, we make use of docker images. They are specified with the `image` constraint. Images are snapshots of environments such as the system your currently working on. You can build them yourself or use prebuild images, e.g. from [docker hub](https://hub.docker.com/). The runner will spawn an instance of this image and you can use or modify it by running scripts and executables, like `apt update && apt upgrade`. After your job is done the container will be deleted.
For our purposes, we make use of docker images. They are specified with the `image` constraint. Images are snapshots of environments such as the system you are currently working on. You can build them yourself or use prebuild images, e.g. from [docker hub](https://hub.docker.com/). The runner will spawn an instance of this image and you can use or modify it by running scripts and executables, like `apt update && apt upgrade`. After your job is done the container will be deleted.
__NOTE__: please use images uploaded to https://harbor.zkm.de/.
#### Scripts
With the `script` constraint you can run any executable that is available in either the container or the repository. A Job has to have at least on item in script.
With the `script` constraint you can run any executable that is available in either the container or the repository.
Imagine the scripts you add to jobs are the same as CLI commands you run on your computer. A Job has to have at least on item in script.
#### Tags
Using the `tags` constraint we define which Runner will be chosen to run your code.
##### CPU Runner
For running the job on the CPU use:
```yaml
tags:
- docker
- cpu
```
##### GPU Runner
## View the state of the pipeline
To see the history of all submitted pipeline go to `CI/CD` > `Pipelines` on the left side. Here you will find most useful information of your current and passed pipelines.
![](media/pipeline.png)
After the pipeline has completed, you can view the output by first clicking on the status or pipeline id and then select the job you want to inspect. The ouput should look something like this.
![](media/job_output.png)
## Minimal GPU Job
In order to use the GPU of the system, we need to make sure we are using an image that has the required libraries installed. In this example we are using the `devel-gpu` version of the latest `tensorflow` image.
Furthermore, we need to specify the necessary tags to trigger the GPU Runner.
By executing `nvidia-smi` we should see information about the available GPU.
```yaml
my-first-gpu-job:
image: tensorflow/tensorflow:devel-gpu
script:
- echo "here could be your executable"
- nvidia-smi
tags:
- docker
- gpu
```
#### View the state of the pipeline
__NOTE:__ Please be mindful of the resources you use. For complex or long running jobs we recommend triggering your training or other pipelines manually so that you only use the ressource wenn actually needed. Use the CPU runner(s) if no GPU access is required.
#### other useful constraints
## Other useful constraints
check the usage of additional constraints in the [reference guide](https://docs.gitlab.com/ee/ci/yaml/index.html)
- _stage_:
You can group multiple independent jobs into stages that run in a defined order.
......@@ -75,6 +90,3 @@ You can group multiple independent jobs into stages that run in a defined order.
## Further Reading
- https://docs.gitlab.com/ee/ci/
## Notes
Please be mindful of the resources you use. For complex or long running jobs we recommend triggering your training or other pipelines manually so that you only use the ressource wenn actually needed. Use the CPU runner(s) if no GPU access is required.
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment