Running CI/CD runners on the same VPS as GitLab and when it breaks

You set up GitLab CE on your VPS. It works. You add the first CI pipeline, a simple build step. Also fine. Then your team grows, pipelines queue up, and suddenly GitLab itself becomes sluggish. Merge requests take minutes to load. The web UI times out. The root cause: your GitLab runner and your GitLab server are fighting over the same CPU, RAM, and disk I/O. Running the gitlab runner same server works, up to a point. This post shows you where that point is, how to detect when you've crossed it, and what to do next.

构建量上来后，Runner必须与GitLab主机分开部署。

Once build volume grows, runners have to move off the GitLab host.

Key Takeaways

• A single GitLab instance + runner on the same VPS typically breaks at around 3-5 concurrent pipelines with moderate build steps.
• Resource contention manifests as slow GitLab web UI, 502 errors, failed jobs, and extended pipeline queue times.
• Monitoring with htop, docker stats, and GitLab's own Prometheus metrics gives you the data to decide when to separate.
• Separating runners onto dedicated VPS instances is the cleanest fix once you hit the contention threshold.

Prerequisites

• A self-hosted GitLab CE instance on a Linux VPS (Debian 12 or Ubuntu 24.04 recommended).
• At least one GitLab Runner registered to that instance, running in docker or shell executor mode.
• Root or sudo access to the VPS.
• Basic familiarity with SSH, systemd, and Docker (if using Docker executor).
• Your VPS should have at least 4 GB RAM, 2 GB VPS plans for GitLab alone are undersized even before adding runners.

Why 'GitLab Runner Same Server' Works at Small Scale

For a solo developer or a small team running one or two pipelines a day, the gitlab runner same server setup is perfectly functional. GitLab CE on a 4 GB RAM VPS with 2 vCPUs can handle the web application, background jobs, and a runner executing builds in Docker containers, as long as the builds are simple. A Node.js npm build or a Python pip install finishes in seconds. The Docker container spins up, runs, and exits before GitLab's background processes even notice.

The problem starts when concurrency increases. GitLab itself is a Ruby on Rails application that consumes memory steadily, about 1-1.5 GB of RAM for the core services (Unicorn or Puma, Sidekiq, Workhorse). Add PostgreSQL and Redis (both always running), and you're at roughly 2.5 GB before any pipeline runs. Now a runner spawns a Docker container for a build. That container might need 512 MB to compile, plus the runner's own process. Run three such builds concurrently, and you hit 4 GB total. The system starts swapping. The web UI becomes unresponsive. Jobs time out.

The sweet spot ends somewhere around 3-5 concurrent pipeline runs with moderate resource demands. Beyond that, you need a plan.

How Resource Contention Happens

When GitLab runner and GitLab server share the same VPS, they compete for four resources:

CPU

GitLab's web application uses CPU for request handling (rendering merge requests, serving repository data). Sidekiq processes background jobs like email notifications, repository mirroring, and pipeline scheduling. The runner uses CPU to clone repositories, execute build scripts, and run tests. If you have 2 vCPUs, two concurrent builds can saturate both cores, leaving GitLab's web requests starved. The typical sign: the web UI loads but feels laggy, and API calls take 5-10 seconds instead of <1 second.

RAM

RAM is the biggest bottleneck in a shared setup. GitLab CE on Debian 12 with default settings uses approximately:

• GitLab Puma: 300-500 MB
• Sidekiq: 200-400 MB
• PostgreSQL: 200-400 MB
• Redis: 100-200 MB
• Workhorse: 50-100 MB
• Total: ~1.5-2 GB

Each running build adds the runner process (50-100 MB) plus the Docker container's memory limit. A Node.js build might use 300-500 MB. A Java/Gradle build can use 1-2 GB. With four builds running and GitLab already using 2 GB, a 4 GB VPS is completely exhausted.

Disk I/O

GitLab writes constantly: repository data, artifacts, logs, database transactions, Sidekiq job queues. A runner building a project clones the repository, writes build artifacts, and potentially caches dependencies, all to the same disk. On an NVMe VPS, this is less of an issue since NVMe handles concurrent writes well. But even NVMe has limits. When the runner git clones a 500 MB repository while GitLab's PostgreSQL writes a checkpoint and the system swaps to disk, I/O latency spikes. Build times double. The web UI reports 502 errors because Workhorse can't serve files fast enough.

Network

GitLab serves HTTP/HTTPS traffic to users and receives push/pull operations via Git. The runner downloads Docker images from a registry and clones repositories from the local GitLab instance. In a shared setup, these compete for the same network throughput. With a VPS capped at 100 Mbps (mid-range plan), one pipeline pulling a 1 GB Docker image can saturate the connection, making the web UI feel unresponsive.

At What Build Volume Must You Separate?

There is no universal number, it depends on build complexity, concurrency, and your VPS specs. But based on real-world experience and common self-hosted setups, here are practical thresholds:

The simplest indicator: run htop during peak pipeline activity. If you see total RAM usage consistently above 85% and CPU idle time below 10%, you are past the comfortable threshold. Another indicator: the GitLab web UI becomes slow when you click "Merge requests" during active pipelines. Measure response time with curl -o /dev/null -s -w '%{time_total}' https://your-gitlab-domain/. If it exceeds 2 seconds during builds, you need separate runners.

If you use Docker executor, check per-container memory with docker stats --no-stream. When any container exceeds 70% of your VPS's available RAM, separation is overdue.

Monitoring to Detect the Breaking Point

Before separating, collect data. Blindly adding hardware wastes money. Use these tools:

1. System Resource Monitoring

htop gives you real-time per-core CPU and per-process memory. Watch the gitlab and docker processes. iostat -x 2 shows disk I/O, look at %util (should stay below 60% on NVMe) and await (should stay below 10ms). free -h tells you total RAM usage.

Set up basic alerting with a cron script that checks memory every minute:

#!/bin/bash
MEM=$(free | grep Mem | awk '{print $3/$2 * 100.0}')
if (( $(echo "$MEM > 90" | bc -l) )); then
    echo "$(date) - WARNING: RAM usage at $MEM%" >> /var/log/gitlab-monitor.log
fi

2. GitLab's Built-in Monitoring

GitLab CE includes Prometheus metrics (if enabled in /etc/gitlab/gitlab.rb with prometheus['enable'] = true). Check the /-/metrics endpoint for gitlab_runner_jobs_running and job_duration_seconds. These tell you exactly how many jobs are running concurrently and how long they take. When running jobs consistently exceeds 80% of your available runner concurrency, separate.

3. The Web UI Response Test

The user-experience test is definitive. During a busy pipeline session, load your GitLab merge requests page. If it takes more than 3 seconds, your shared setup is failing your team. The gitlab-rake gitlab:check command also reports background job queue lengths, a growing Sidekiq backlog means resources are too tight.

How to Separate Runners onto a Dedicated VPS

Once you've crossed the threshold, the fix is straightforward: move the GitLab runner to a separate VPS. This is a one-time configuration change that recovers your GitLab performance and lets pipelines run without interference.

Step 1, Provision a Runner VPS

Provision a separate Linux VPS. For runners, specs matter less than isolation, 2 vCPU / 4 GB RAM is a good starting point for 3-5 concurrent builds. If your builds compile large binaries (Go, Rust, Java) or run integration tests that consume memory, go to 8 GB. GitLab VPS plans from thueVPS with NVMe storage handle concurrent cloning and artifact uploads well, the key metric is low I/O latency, which NVMe delivers.

Step 2, Install Docker and the Runner

On the new runner VPS (Debian 12 example):

apt update && apt install -y docker.io curl
systemctl enable --now docker
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | bash
apt install -y gitlab-runner

Register the runner against your GitLab instance. Replace YOUR_GITLAB_URL and REGISTRATION_TOKEN with values from Settings > CI/CD > Runners in your GitLab web UI:

gitlab-runner register \
  --non-interactive \
  --url "https://your-gitlab-domain" \
  --registration-token "YOUR_TOKEN" \
  --executor "docker" \
  --docker-image "alpine:latest" \
  --description "dedicated-runner-01" \
  --tag-list "dedicated,linux" \
  --run-untagged="true" \
  --locked="false"

Step 3, Stop the Local Runner on Your GitLab VPS

On the GitLab VPS, disable the shared runner to prevent jobs from running locally:

gitlab-runner stop
systemctl disable gitlab-runner

Then, in GitLab's web UI, go to Settings > CI/CD > Runners and disable the shared runner. Only the newly registered runner on the separate VPS should be active. Now all pipelines will run on the dedicated runner VPS, leaving GitLab's server free to serve the web UI and process requests.

Step 4, Verify the Separation

Trigger a pipeline. On the GitLab VPS, run htop, you should see no Docker containers from the runner. On the runner VPS, docker ps should show the build container. GitLab's web UI should feel snappy even with 5-6 concurrent jobs running.

Troubleshooting Common Issues During Separation

• Runner can't connect to GitLab: Ensure the runner VPS can reach the GitLab domain. If your GitLab VPS uses a self-signed certificate, add it to the runner with --tls-ca-file during registration, or use --tls-skip-verify true (not recommended for production).
• Jobs failing with "dial tcp: connection refused": The runner VPS needs outbound access to your GitLab server on port 443 (or 80 if HTTP). Check the firewall on both sides: ufw status on Ubuntu, firewall-cmd --list-all on AlmaLinux, or nft list ruleset for nftables.
• Docker container exits immediately: The runner VPS might be out of disk space for Docker images. Check with df -h and docker system prune -a to clean unused images and containers.
• GitLab web UI still slow after separation: Your GitLab VPS might have lingering load from accumulated background jobs. Restart GitLab with gitlab-ctl restart and monitor with htop for 10 minutes. If still slow, your GitLab VPS itself might need a spec upgrade for the user load.

FAQ

Related articles

• Self-hosted GitLab CE minimum viable server spec
• Migrate a website to a new VPS with zero downtime
• Self-hosting n8n for workflow automation on a VPS
• VPS performance benchmarking with fio, sysbench, iperf3