By default, Azure DevOps pipeline runs on Microsoft-hosted agent.
Every time the pipeline runs, it takes long time to install middleware (usually 5minutes+)
Microsoft provides the other way to avoid these wasted time, “Self-hosted” agent.
“Self-hosted” agent is provided with configurable Dockerfile and .sh.
So, configured “self-hosted” image run on VM or ACI provides “Job runner” environment.
I tried to set up “Self-hosted” Linux agent and configured ACI/ACR to run the agent.
ref. Use Running Azure Container Instance in Azure Pipeline
Prerequisite
x86 mac and Docker desktop.
|
1 2 3 4 5 6 7 8 9 10 |
$ sw_vers ProductName: macOS ProductVersion: 11.6.6 BuildVersion: 20G624 $ uname -m x86_64 $ docker --version ocker version 20.10.17, build 100c701 |
Build Self-hosted Linux agent
see Create and build the Dockerfile
Make workdir on mac and Dockerfile, entry script.
|
1 2 3 |
$ mkdir -p ~/apps/dockeragent && cd ~/apps/dockeragent $ touch Dockerfile $ touch start.sh |
Dockerfile is as follows.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
FROM ubuntu:20.04 RUN DEBIAN_FRONTEND=noninteractive apt-get update RUN DEBIAN_FRONTEND=noninteractive apt-get upgrade -y RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \ apt-transport-https \ apt-utils \ ca-certificates \ curl \ git \ iputils-ping \ jq \ lsb-release \ software-properties-common RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash # Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'. ENV TARGETARCH=linux-x64 WORKDIR /azp COPY ./start.sh . RUN chmod +x start.sh ENTRYPOINT [ "./start.sh" ] |
Entry script (start.sh) is as follows.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 |
#!/bin/bash set -e if [ -z "$AZP_URL" ]; then echo 1>&2 "error: missing AZP_URL environment variable" exit 1 fi if [ -z "$AZP_TOKEN_FILE" ]; then if [ -z "$AZP_TOKEN" ]; then echo 1>&2 "error: missing AZP_TOKEN environment variable" exit 1 fi AZP_TOKEN_FILE=/azp/.token echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE" fi unset AZP_TOKEN if [ -n "$AZP_WORK" ]; then mkdir -p "$AZP_WORK" fi export AGENT_ALLOW_RUNASROOT="1" cleanup() { if [ -e config.sh ]; then print_header "Cleanup. Removing Azure Pipelines agent..." # If the agent has some running jobs, the configuration removal process will fail. # So, give it some time to finish the job. while true; do ./config.sh remove --unattended --auth PAT --token $(cat "$AZP_TOKEN_FILE") && break echo "Retrying in 30 seconds..." sleep 30 done fi } print_header() { lightcyan='\033[1;36m' nocolor='\033[0m' echo -e "${lightcyan}$1${nocolor}" } # Let the agent ignore the token env variables export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE print_header "1. Determining matching Azure Pipelines agent..." AZP_AGENT_PACKAGES=$(curl -LsS \ -u user:$(cat "$AZP_TOKEN_FILE") \ -H 'Accept:application/json;' \ "$AZP_URL/_apis/distributedtask/packages/agent?platform=$TARGETARCH&top=1") AZP_AGENT_PACKAGE_LATEST_URL=$(echo "$AZP_AGENT_PACKAGES" | jq -r '.value[0].downloadUrl') if [ -z "$AZP_AGENT_PACKAGE_LATEST_URL" -o "$AZP_AGENT_PACKAGE_LATEST_URL" == "null" ]; then echo 1>&2 "error: could not determine a matching Azure Pipelines agent" echo 1>&2 "check that account '$AZP_URL' is correct and the token is valid for that account" exit 1 fi print_header "2. Downloading and extracting Azure Pipelines agent..." curl -LsS $AZP_AGENT_PACKAGE_LATEST_URL | tar -xz & wait $! source ./env.sh print_header "3. Configuring Azure Pipelines agent..." ./config.sh --unattended \ --agent "${AZP_AGENT_NAME:-$(hostname)}" \ --url "$AZP_URL" \ --auth PAT \ --token $(cat "$AZP_TOKEN_FILE") \ --pool "${AZP_POOL:-Default}" \ --work "${AZP_WORK:-_work}" \ --replace \ --acceptTeeEula & wait $! print_header "4. Running Azure Pipelines agent..." trap 'cleanup; exit 0' EXIT trap 'cleanup; exit 130' INT trap 'cleanup; exit 143' TERM chmod +x ./run-docker.sh # To be aware of TERM and INT signals call run.sh # Running it with the --once flag at the end will shut down the agent after the build is executed ./run-docker.sh "$@" & wait $! |
Build image.
AZP_URL is Azure DevOps organization url. ( https://dev.azure.com/{organization} ) .
AZP_TOKEN is Personal Access Token for DevOps. ( full access for simplicity ) .
AZP_AGENT_NAME is ID on DevOps ( ex. mydockeragent ).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
$ docker run -e AZP_URL=<AZP_URL> \ -e AZP_TOKEN=<AZP_TOKEN> \ -e AZP_AGENT_NAME=<AZP_AGENT_NAME> \ dockeragent:latest --once . Determining matching Azure Pipelines agent... 2. Downloading and extracting Azure Pipelines agent... 3. Configuring Azure Pipelines agent... ___ ______ _ _ _ / _ \ | ___ (_) | (_) / /_\ \_____ _ _ __ ___ | |_/ /_ _ __ ___| |_ _ __ ___ ___ | _ |_ / | | | '__/ _ \ | __/| | '_ \ / _ \ | | '_ \ / _ \/ __| | | | |/ /| |_| | | | __/ | | | | |_) | __/ | | | | | __/\__ \ \_| |_/___|\__,_|_| \___| \_| |_| .__/ \___|_|_|_| |_|\___||___/ | | agent v2.210.1 |_| (commit fae6a0b) >> End User License Agreements: .. >> Connect: .. >> Register Agent: Scanning for tool capabilities. Connecting to the server. Successfully added the agent Testing agent connection. 2022-09-29 18:39:40Z: Settings Saved. 4. Running Azure Pipelines agent... Scanning for tool capabilities. Connecting to the server. 2022-09-29 18:39:44Z: Listening for Jobs |
“Running Azure Pipelines agent” on Mac connects to Azure DevOps Pipeline.
then waits for running job.
—
to be continued…