Skip to content

Commit 4b291b6

Browse files
ppodgorsekppodgorsek
authored
Merge pull request #504 from ppodgorsek/issue-503
Issue #503 - Add CI pipeline example for Azure DevOps
2 parents 8ccb571 + 814968e commit 4b291b6

File tree

1 file changed

+167
-93
lines changed

1 file changed

+167
-93
lines changed

‎README.md‎

Lines changed: 167 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616
* [Installing additional dependencies](#installing-additional-dependencies)
1717
* [Security consideration](#security-consideration)
1818
* [Continuous integration](#continuous-integration)
19+
* [Azure DevOps pipeline](#ci-azure-devops)
20+
* [Jenkins pipeline](#ci-jenkins)
1921
* [Defining a test run ID](#defining-a-test-run-id)
2022
* [Upload test reports to an AWS S3 bucket](#upload-test-reports-to-an-aws-s3-bucket)
2123
* [Testing this project](#testing-this-project)
@@ -74,10 +76,12 @@ As stated by [the official GitHub project](https://github.com/robotframework/Sel
7476

7577
This container can be run using the following command:
7678

77-
docker run \
78-
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
79-
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
80-
ppodgorsek/robot-framework:<version>
79+
```sh
80+
docker run \
81+
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
82+
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
83+
ppodgorsek/robot-framework:<version>
84+
```
8185

8286
<a name="switching-browsers"></a>
8387

@@ -114,9 +118,11 @@ It is possible to use different directories to read tests from and to generate r
114118

115119
It is possible to parallelise the execution of your test suites. Simply define the `ROBOT_THREADS` environment variable, for example:
116120

117-
docker run \
118-
-e ROBOT_THREADS=4 \
119-
ppodgorsek/robot-framework:latest
121+
```sh
122+
docker run \
123+
-e ROBOT_THREADS=4 \
124+
ppodgorsek/robot-framework:latest
125+
```
120126

121127
By default, there is no parallelisation.
122128

@@ -126,20 +132,24 @@ By default, there is no parallelisation.
126132

127133
When using parallelisation, it is possible to pass additional [pabot options](https://github.com/mkorpela/pabot#command-line-options), such as `--testlevelsplit`, `--argumentfile`, `--ordering`, etc. These can be passed by using the `PABOT_OPTIONS` environment variable, for example:
128134

129-
docker run \
130-
-e ROBOT_THREADS=4 \
131-
-e PABOT_OPTIONS="--testlevelsplit" \
132-
ppodgorsek/robot-framework:latest
135+
```sh
136+
docker run \
137+
-e ROBOT_THREADS=4 \
138+
-e PABOT_OPTIONS="--testlevelsplit" \
139+
ppodgorsek/robot-framework:latest
140+
```
133141

134142
<a name="passing-additional-options"></a>
135143

136144
### Passing additional options
137145

138146
RobotFramework supports many options such as `--exclude`, `--variable`, `--loglevel`, etc. These can be passed by using the `ROBOT_OPTIONS` environment variable, for example:
139147

140-
docker run \
141-
-e ROBOT_OPTIONS="--loglevel DEBUG" \
142-
ppodgorsek/robot-framework:latest
148+
```sh
149+
docker run \
150+
-e ROBOT_OPTIONS="--loglevel DEBUG" \
151+
ppodgorsek/robot-framework:latest
152+
```
143153

144154
<a name="testing-emails"></a>
145155

@@ -157,9 +167,11 @@ This project is meant to allow your tests to run anywhere. Sometimes that can be
157167

158168
To set the timezone used inside the Docker image, you can set the `TZ` environment variable:
159169

160-
docker run \
161-
-e TZ=America/New_York \
162-
ppodgorsek/robot-framework:latest
170+
```sh
171+
docker run \
172+
-e TZ=America/New_York \
173+
ppodgorsek/robot-framework:latest
174+
```
163175

164176
<a name="installing-additional-dependencies"></a>
165177

@@ -169,10 +181,12 @@ It is possible to install additional dependencies dynamically at runtime rather
169181

170182
To do so, simply mount a text file containing the list of dependencies you would like to install using `pip`: (by default, this file is empty if not mounted)
171183

172-
docker run \
173-
-v <local path to the dependency file>:/opt/robotframework/pip-requirements.txt:Z \
174-
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
175-
ppodgorsek/robot-framework:latest
184+
```sh
185+
docker run \
186+
-v <local path to the dependency file>:/opt/robotframework/pip-requirements.txt:Z \
187+
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
188+
ppodgorsek/robot-framework:latest
189+
```
176190
177191
The file must follow [Pip's official requirements file format](https://pip.pypa.io/en/stable/reference/requirements-file-format/).
178192

@@ -191,9 +205,11 @@ rpa==1.50.0
191205
192206
By default, containers are implicitly run using `--user=1000:1000`, please remember to adjust that command-line setting accordingly, for example:
193207
194-
docker run \
195-
--user=1001:1001 \
196-
ppodgorsek/robot-framework:latest
208+
```sh
209+
docker run \
210+
--user=1001:1001 \
211+
ppodgorsek/robot-framework:latest
212+
```
197213

198214
Remember that that UID/GID should be allowed to access the mounted volumes in order to read the test suites and to write the output.
199215

@@ -208,42 +224,90 @@ This is a good security practice to make sure containers cannot perform unwanted
208224

209225
## Continuous integration
210226

227+
<a name="ci-azure-devops"></a>
228+
229+
### Azure DevOps pipeline
230+
231+
Azure DevOps provides a `Docker@2` task but it only allows a limited number of commands, it is therefore required to use a `script` instead:
232+
233+
```yml
234+
- job: FunctionalRegressionTests
235+
displayName: Robot Framework tests
236+
steps:
237+
- script: |
238+
set -x
239+
240+
mkdir -p $(Build.Repository.LocalPath)/robot-framework-reports
241+
chmod -R ugo+rwx $(Build.Repository.LocalPath)/robot-framework-reports
242+
243+
# The ROBOT_OPTIONS allow to generate a specific output file that can be
244+
# used to publish test results in the Azure DevOps pipeline run
245+
# Remember to replace ${{ parameters.robotFrameworkFolder }} by the correct
246+
# folder name in your repository
247+
docker run --rm \
248+
--shm-size=1g \
249+
-v $(Build.Repository.LocalPath)/robot-framework-reports:/opt/robotframework/reports:Z \
250+
-v $(Build.Repository.LocalPath)/${{ parameters.robotFrameworkFolder }}:/opt/robotframework/tests:Z \
251+
-e ROBOT_OPTIONS="-x outputxunit.xml" \
252+
-t docker.io/ppodgorsek/robot-framework:latest
253+
displayName: Run Robot Framework tests
254+
255+
- task: PublishTestResults@2
256+
condition: always()
257+
displayName: Publish Robot Framework test results
258+
inputs:
259+
testResultsFormat: 'JUnit'
260+
testResultsFiles: '**/outputxunit.xml'
261+
searchFolder: '$(Build.Repository.LocalPath)/robot-framework-reports'
262+
failTaskOnFailedTests: true
263+
failTaskOnFailureToPublishResults: true
264+
failTaskOnMissingResultsFile: true
265+
```
266+
267+
<a name="ci-jenkins"></a>
268+
269+
### Jenkins pipeline
270+
211271
It is possible to run the project from within a Jenkins pipeline by relying on the shell command line directly:
212272
213-
pipeline {
214-
agent any
215-
stages {
216-
stage('Functional regression tests') {
217-
steps {
218-
sh "docker run --shm-size=1g -e BROWSER=firefox -v $WORKSPACE/robot-tests:/opt/robotframework/tests:Z -v $WORKSPACE/robot-reports:/opt/robotframework/reports:Z ppodgorsek/robot-framework:latest"
219-
}
220-
}
221-
}
273+
```yml
274+
pipeline {
275+
agent any
276+
stages {
277+
stage('Functional regression tests') {
278+
steps {
279+
sh "docker run --shm-size=1g -e BROWSER=firefox -v $WORKSPACE/robot-tests:/opt/robotframework/tests:Z -v $WORKSPACE/robot-reports:/opt/robotframework/reports:Z ppodgorsek/robot-framework:latest"
280+
}
222281
}
282+
}
283+
}
284+
```
223285

224286
The pipeline stage can also rely on a Docker agent, as shown in the example below:
225287

226-
pipeline {
227-
agent none
228-
stages {
229-
stage('Functional regression tests') {
230-
agent { docker {
231-
image 'ppodgorsek/robot-framework:latest'
232-
args '--shm-size=1g -u root' }
233-
}
234-
environment {
235-
BROWSER = 'firefox'
236-
ROBOT_TESTS_DIR = "$WORKSPACE/robot-tests"
237-
ROBOT_REPORTS_DIR = "$WORKSPACE/robot-reports"
238-
}
239-
steps {
240-
sh '''
241-
/opt/robotframework/bin/run-tests-in-virtual-screen.sh
242-
'''
243-
}
244-
}
245-
}
288+
```yml
289+
pipeline {
290+
agent none
291+
stages {
292+
stage('Functional regression tests') {
293+
agent { docker {
294+
image 'ppodgorsek/robot-framework:latest'
295+
args '--shm-size=1g -u root' }
296+
}
297+
environment {
298+
BROWSER = 'firefox'
299+
ROBOT_TESTS_DIR = "$WORKSPACE/robot-tests"
300+
ROBOT_REPORTS_DIR = "$WORKSPACE/robot-reports"
301+
}
302+
steps {
303+
sh '''
304+
/opt/robotframework/bin/run-tests-in-virtual-screen.sh
305+
'''
306+
}
246307
}
308+
}
309+
}
310+
```
247311
248312
<a name="defining-a-test-run-id"></a>
249313
@@ -257,9 +321,11 @@ For that purpose, the `ROBOT_TEST_RUN_ID` variable was introduced:
257321
258322
It can simply be passed during the execution, such as:
259323
260-
docker run \
261-
-e ROBOT_TEST_RUN_ID="feature/branch-name" \
262-
ppodgorsek/robot-framework:latest
324+
```sh
325+
docker run \
326+
-e ROBOT_TEST_RUN_ID="feature/branch-name" \
327+
ppodgorsek/robot-framework:latest
328+
```
263329
264330
By default, the test run ID is empty.
265331
@@ -269,48 +335,54 @@ By default, the test run ID is empty.
269335
270336
To upload the report of a test run to an S3 bucket, you need to define the following environment variables:
271337
272-
docker run \
273-
-e AWS_ACCESS_KEY_ID=<your AWS key> \
274-
-e AWS_SECRET_ACCESS_KEY=<your AWS secret> \
275-
-e AWS_DEFAULT_REGION=<your AWS region e.g. eu-central-1> \
276-
-e AWS_BUCKET_NAME=<name of your S3 bucket> \
277-
ppodgorsek/robot-framework:latest
338+
```sh
339+
docker run \
340+
-e AWS_ACCESS_KEY_ID=<your AWS key> \
341+
-e AWS_SECRET_ACCESS_KEY=<your AWS secret> \
342+
-e AWS_DEFAULT_REGION=<your AWS region e.g. eu-central-1> \
343+
-e AWS_BUCKET_NAME=<name of your S3 bucket> \
344+
ppodgorsek/robot-framework:latest
345+
```
278346
279347
<a name="testing-this-project"></a>
280348
281349
## Testing this project
282350
283351
Not convinced yet? Simple tests have been prepared in the `test/` folder, you can run them using the following commands:
284352
285-
# Using Chromium
286-
docker run \
287-
-v `pwd`/reports:/opt/robotframework/reports:Z \
288-
-v `pwd`/test:/opt/robotframework/tests:Z \
289-
-e BROWSER=chrome \
290-
ppodgorsek/robot-framework:latest
291-
292-
# Using Firefox
293-
docker run \
294-
-v `pwd`/reports:/opt/robotframework/reports:Z \
295-
-v `pwd`/test:/opt/robotframework/tests:Z \
296-
-e BROWSER=firefox \
297-
ppodgorsek/robot-framework:latest
353+
```sh
354+
# Using Chromium
355+
docker run \
356+
-v `pwd`/reports:/opt/robotframework/reports:Z \
357+
-v `pwd`/test:/opt/robotframework/tests:Z \
358+
-e BROWSER=chrome \
359+
ppodgorsek/robot-framework:latest
360+
361+
# Using Firefox
362+
docker run \
363+
-v `pwd`/reports:/opt/robotframework/reports:Z \
364+
-v `pwd`/test:/opt/robotframework/tests:Z \
365+
-e BROWSER=firefox \
366+
ppodgorsek/robot-framework:latest
367+
```
298368
299369
For Windows users who use **PowerShell**, the commands are slightly different:
300370
301-
# Using Chromium
302-
docker run \
303-
-v ${PWD}/reports:/opt/robotframework/reports:Z \
304-
-v ${PWD}/test:/opt/robotframework/tests:Z \
305-
-e BROWSER=chrome \
306-
ppodgorsek/robot-framework:latest
307-
308-
# Using Firefox
309-
docker run \
310-
-v ${PWD}/reports:/opt/robotframework/reports:Z \
311-
-v ${PWD}/test:/opt/robotframework/tests:Z \
312-
-e BROWSER=firefox \
313-
ppodgorsek/robot-framework:latest
371+
```sh
372+
# Using Chromium
373+
docker run \
374+
-v ${PWD}/reports:/opt/robotframework/reports:Z \
375+
-v ${PWD}/test:/opt/robotframework/tests:Z \
376+
-e BROWSER=chrome \
377+
ppodgorsek/robot-framework:latest
378+
379+
# Using Firefox
380+
docker run \
381+
-v ${PWD}/reports:/opt/robotframework/reports:Z \
382+
-v ${PWD}/test:/opt/robotframework/tests:Z \
383+
-e BROWSER=firefox \
384+
ppodgorsek/robot-framework:latest
385+
```
314386
315387
Screenshots of the results will be available in the `reports/` folder.
316388
@@ -384,11 +456,13 @@ As mentioned on the [Docker Hub](https://hub.docker.com/r/ppodgorsek/robot-frame
384456

385457
As mentioned in the official documentation, [Podman](https://docs.podman.io/en/latest/markdown/podman-run.1.html#platform-os-arch) and [Docker](https://docs.docker.com/build/building/multi-platform/) provide a `--platform` option which selects a given application architecture, such as:
386458

387-
docker run \
388-
--platform linux/amd64 \
389-
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
390-
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
391-
ppodgorsek/robot-framework:<version>
459+
```sh
460+
docker run \
461+
--platform linux/amd64 \
462+
-v <local path to the reports' folder>:/opt/robotframework/reports:Z \
463+
-v <local path to the test suites' folder>:/opt/robotframework/tests:Z \
464+
ppodgorsek/robot-framework:<version>
465+
```
392466

393467
Please note: builds and automated tests of this project will remain performed on a `linux/amd64` architecture so such emulation might not work, depending on your device and operating system.
394468

0 commit comments

Comments
 (0)