Protect web apps with ModSecurity WAF and OWASP CRS on Nginx

You have Nginx serving a real application, and you just watched a bot crawl your login page with a payload that looks like UNION SELECT in the query string. You want a Web Application Firewall (WAF) in front of your app that blocks SQL injection, cross-site scripting (XSS), and remote code execution attempts before they reach your backend. ModSecurity with the OWASP Core Rule Set (CRS) is the standard open-source answer, and it runs natively inside Nginx. On Ubuntu 24.04, you can have it filtering traffic in about twenty minutes.
Prerequisites
- An Ubuntu 24.04 VPS with a non-root sudo user. This guide assumes the Linux VPS runs a fresh install with Nginx already serving a site.
- Nginx installed, preferably the
nginx-extraspackage so ModSecurity compiles against the exact same version the distro ships. - Build tools:
build-essential,libpcre3-dev,libxml2-dev,libcurl4-openssl-dev,libyajl-dev,libgeoip-dev,pkg-config,libssl-dev. - Root or sudo access to reload Nginx and edit its configuration.
Why a WAF belongs on your Nginx server
A WAF filters HTTP traffic at the application layer, which is exactly where the attacks you actually worry about happen. A network firewall blocks ports; it does not understand that ?id=1 OR 1=1 in a URL is an attack. ModSecurity inspects every request, runs it against a large set of rules, and decides whether to pass, block, or challenge it.
The OWASP CRS is the rule set that gives ModSecurity its teeth. It ships with hundreds of rules covering the OWASP Top 10: SQL injection, XSS, local file inclusion, command injection, and bot detection. The combination is battle-tested and free, which is why it sits in front of everything from small WordPress VPS setups to large production clusters. You still need to patch your application and keep TLS current, but a WAF catches the automated garbage that scans every public IP constantly.
ModSecurity 只拦截已配置的规则,误报需要按业务调整。
ModSecurity only blocks what the configured rules match, and false positives need tuning to your application.
Install ModSecurity 3 and the Nginx connector
ModSecurity 3 is the current generation, rewritten as a library called libmodsecurity3 with a separate connector for Nginx. The apt repositories hold part of what you need, but the Nginx connector must be compiled. Start by installing the dependencies and the library package.
sudo apt update
sudo apt install -y build-essential libpcre3-dev libxml2-dev libcurl4-openssl-dev \
libyajl-dev libgeoip-dev pkg-config libssl-dev libmodsecurity3 nginx-extras
The nginx-extras package matters. It ships with the same Nginx version as the core package but includes the dynamic module loading hooks you need. Next, download and compile the connector. The version number matters, so check the ModSecurity-nginx releases page and use the latest stable tag.
cd /usr/local/src
sudo wget https://github.com/owasp-modsecurity/ModSecurity-nginx/archive/refs/tags/v1.0.3.tar.gz
sudo tar xzf v1.0.3.tar.gz
cd ModSecurity-nginx-1.0.3
sudo nginx -v
You need the exact Nginx version string for the compile step. The configure script takes it with --add-dynamic-module and builds against your installed headers. The nginx -v output tells you the version, for example nginx/1.24.0. Substitute that in the compile command.
sudo ./configure --with-compat --add-dynamic-module=.
sudo make modules
The build produces ngx_http_modsecurity_module.so in the objs directory. Copy it into Nginx's module directory, then load it in the main configuration file.
sudo cp objs/ngx_http_modsecurity_module.so /usr/share/nginx/modules/
echo "load_module modules/ngx_http_modsecurity_module.so;" | sudo tee /etc/nginx/modules-enabled/50-modsecurity.conf
Verify: test the configuration to confirm the module loaded without errors.
sudo nginx -t
Expected output contains syntax is ok and test is successful. If this fails, check that /usr/share/nginx/modules/ngx_http_modsecurity_module.so exists and the version matches your Nginx build.
Download and configure the OWASP Core Rule Set
The CRS is a separate download, a set of rule files that ModSecurity loads at startup. Clone the current release into /etc/nginx/modsecurity alongside the recommended base configuration file.
sudo mkdir -p /etc/nginx/modsecurity
cd /etc/nginx/modsecurity
sudo wget https://github.com/coreruleset/coreruleset/archive/refs/tags/v4.11.0.tar.gz
sudo tar xzf v4.11.0.tar.gz
sudo mv coreruleset-4.11.0 /etc/nginx/modsecurity/crs
sudo cp /etc/nginx/modsecurity/crs/crs-setup.conf.example /etc/nginx/modsecurity/crs/crs-setup.conf
Now create the main ModSecurity configuration file that ties everything together. This file sets the detection engine, loads the CRS, and defines what happens when a rule matches. For a first deployment, set the engine to DetectionOnly so you can watch what it flags before blocking anything.
sudo tee /etc/nginx/modsecurity/modsecurity.conf > /dev/null <<'EOF'
SecRuleEngine DetectionOnly
SecRequestBodyAccess On
SecResponseBodyAccess Off
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecPcreMatchLimit 100000
SecPcreMatchLimitRecursion 100000
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog /var/log/nginx/modsec_audit.log
SecAuditLogType Serial
SecAuditLogStorageDir /var/log/nginx/modsec_audit
Include /etc/nginx/modsecurity/crs/crs-setup.conf
Include /etc/nginx/modsecurity/crs/rules/*.conf
EOF
Set the rule engine to On only after you have watched the audit log in detection mode for a few days. The SecRequestBodyAccess On line is what lets ModSecurity inspect POST bodies, which is where SQL injection and XSS payloads usually hide. The audit log settings are critical for tuning; without them you are flying blind.
Connect ModSecurity to your Nginx sites
The module loads globally, but each server block decides whether to run it. Add the ModSecurity directives to the server block or location that serves your application. The modsecurity on directive activates the engine for that context, and modsecurity_rules_file points to the configuration you just created.
sudo nano /etc/nginx/sites-available/your-site
Inside the server { } block, add these two lines:
modsecurity on;
modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
If you run multiple sites, place these directives inside each server block that needs protection. Putting them inside a location block limits the WAF to that path, useful when only part of your app handles untrusted input. Save the file and reload Nginx.
sudo nginx -t
sudo systemctl reload nginx
Verify: confirm the module is active in the running configuration.
nginx -V 2>&1 | grep modsecurity
Expected output shows --add-dynamic-module=.../ModSecurity-nginx-1.0.3. Next, confirm the WAF is actually inspecting traffic by sending a request with a suspicious payload.
curl -I "http://your-server-ip/?id=1%20OR%201=1"
sudo tail -n 20 /var/log/nginx/modsec_audit.log
The audit log should contain an entry for this request. Because the engine is in DetectionOnly mode, the request is logged but not blocked. This is your first real confirmation that the pipeline works.
Tune the rule set for false positives
The CRS ships with three levels of aggressiveness, set with the SecAction directive in crs-setup.conf. The default is PL1, which blocks obvious attacks with few false positives. Higher levels inspect more rules and catch subtler attacks but flag more legitimate traffic. Edit crs-setup.conf and uncomment the paranoia level you want.
sudo nano /etc/nginx/modsecurity/crs/crs-setup.conf
Look for this block and set the level:
SecAction \
"id:900000,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:tx.executing_paranoia_level=1"
Change the executing_paranoia_level value to 2 if you handle only your own traffic and can tolerate stricter checks. Most production deployments stop at PL2. Levels 3 and 4 are for environments where attackers are specifically targeting you and you can whitelist aggressively.
Verify: reload Nginx and re-run the previous SQLi test, then inspect the audit log. The rule ID that fired will tell you which rules are too strict for your app. The most common false positive culprits are rule 920420 (malformed request) and rule 932160 (remote command execution via the User-Agent header), which can collide with less common but legitimate requests.
Enable blocking mode with confidence
Once the audit log shows your legitimate traffic passing and only real attacks flagged, switch the engine to On. This is the moment the WAF starts protecting your application instead of just watching it.
sudo sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /etc/nginx/modsecurity/modsecurity.conf
sudo systemctl reload nginx
Verify: this time the same SQLi payload must be rejected with an HTTP 403 response.
curl -I "http://your-server-ip/?id=1%20OR%201=1"
Expected output shows HTTP/1.1 403 Forbidden. Now send a normal request to confirm your real traffic still works.
curl -I "http://your-server-ip/"
Expected output shows HTTP/1.1 200 OK. If a legitimate request gets blocked, search the audit log for the rule ID and add an exception for that specific path or parameter, not a blanket rule disable.
Handle legitimate traffic that looks like an attack
No rule set is perfect, and the CRS will occasionally flag a request your application genuinely needs. The correct fix is a precise exclusion rule, not disabling the whole CRS. If your app legitimately accepts base64-encoded data in a JSON body field, add an exclusion like this to a separate rules file.
sudo tee /etc/nginx/modsecurity/crs/exclusions.conf > /dev/null <<'EOF'
SecRule REQUEST_BASENAME "@streq /api/upload" \
"id:1001,\
phase:1,\
pass,\
nolog,\
ctl:ruleRemoveById=920420"
EOF
Include this file from modsecurity.conf after the CRS includes:
echo 'Include /etc/nginx/modsecurity/crs/exclusions.conf' | sudo tee -a /etc/nginx/modsecurity/modsecurity.conf
sudo systemctl reload nginx
Keep exclusions specific. Removing a rule globally for your whole site is how WAFs stop being useful. A targeted path-and-rule exclusion protects the feature that needs it while keeping everything else under the full rule set.
Troubleshooting
Nginx fails to compile the connector: the error version mismatch means your Nginx headers do not match the running binary. Reinstall nginx-core and nginx-common to the same version before building. Run sudo apt install --reinstall nginx-core nginx-common and try again.
Module loads but no rule matches: check the order of directives. The modsecurity_rules_file must be included in the same server block as modsecurity on. If the audit log is empty, run sudo nginx -T | grep modsecurity to confirm the directives are present in the active configuration.
All requests return 403 after switching to blocking mode: your app sends a header or cookie the CRS dislikes. Inspect /var/log/nginx/modsec_audit.log, find the rule ID, and add a targeted exclusion like the example above. Do not set SecRuleEngine DetectionOnly again as a permanent solution, that just turns the WAF off.
High memory usage: the PCRE match limits in modsecurity.conf control how much memory a single request can consume. If you see OOM kills under heavy load, reduce SecPcreMatchLimit to 50000 and restart rather than accepting unlimited resource use from crafted requests.
FAQ
Does ModSecurity 3 work with Nginx without recompiling Nginx itself?
Yes. ModSecurity 3 ships as a shared library (libmodsecurity3) and the Nginx connector builds as a dynamic module. As long as you build the connector with --with-compat against the same Nginx version you run, you load it with a single load_module line and never touch the Nginx binary.
What is the difference between ModSecurity and the OWASP CRS?
ModSecurity is the engine that inspects traffic and executes rules. The OWASP CRS is the rule set, a collection of detection and blocking rules covering the OWASP Top 10. You need both, and the CRS is what actually knows what an SQL injection looks like.
Will the WAF break my legitimate API requests?
Possibly, especially at higher paranoia levels. This is why you deploy in DetectionOnly mode first, review the audit log, and add targeted exclusions for legitimate traffic that collides with a rule. The CRS documentation and the audit log are your tuning tools.
What is the performance cost of running ModSecurity on Nginx?
Every request now runs through hundreds of regex rules. The cost is significant, typically between 10 and 40 percent additional CPU on the same traffic, depending on request size and paranoia level. A single-core VPS serving a low-traffic site handles it fine. For higher traffic you want extra vCPU capacity. A VPS with dedicated vCPU handles this predictably under load.
Can I run ModSecurity and a reverse proxy together?
Yes, and it is a common pattern. The module runs inside Nginx itself, so the WAF inspects traffic before your proxy passes it to the backend. This works whether the backend is another Nginx, Apache, or an application server, and it protects the reverse proxy layer with the same TLS termination you already have there.
Related articles
- Security audit and hardening with Lynis on a VPS
- Hardening SSH on a new VPS, keys, ports, fail2ban and CrowdSec
- Set up nftables firewall on a VPS to replace iptables
- Set up a LEMP stack, Nginx, MariaDB, PHP 8.3 on Ubuntu 24.04
ModSecurity 与 OWASP CRS 配置要点
在 Nginx 上部署 ModSecurity 3 与 OWASP 核心规则集,可以拦截 SQL 注入、XSS 和命令执行攻击。先在检测模式下运行,观察审计日志中的误报,再切换为阻断模式。规则排除必须针对具体路径和规则编号,不要全局禁用。建议将执行偏执级别设为 1 或 2,在安全与可用性之间取得平衡。


