Skip to content

ip collection does not seem to work properly with latest ModSecurity #3394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ne20002 opened this issue May 29, 2025 · 13 comments
Open

ip collection does not seem to work properly with latest ModSecurity #3394

ne20002 opened this issue May 29, 2025 · 13 comments
Assignees
Labels
3.x Related to ModSecurity version 3.x

Comments

@ne20002
Copy link

ne20002 commented May 29, 2025

I am using @rbl xbl.spamhaus.org. rule for protecting my Fediverse server and I face the issue that with latest version of either ModSecurity or the docker.io/owasp/modsecurity-crs:nginx container the collection handling for the IP collection seems to not longer work properly.

In my pi-hole I see a huge number of queries to .xbl.spamhaus.org for all the ip addresses checked and with queries to the same ip based query many times where I would expect that this is not the case due to how I set up the ModSecurity rules.

  • The issue is happening with the latest version of the docker.io/owasp/modsecurity-crs:nginx container.
  • It is far less a occuring with docker.io/owasp/modsecurity-crs:4.10-nginx-202501050801 even when mounting ruleset 4.14 to this container.
  • I don't believe it is a problem with the CRS.

Based on what I see I would see it as if

  • the expirevar for the IP collection is not working properly
  • maybe the persistent store is causing the problem (as I don't see any problems with the TX collection which is also a collection but not persistet). This might be an docker image problem.

According to modsecurity.conf the SecTmpDir is set to /tmp/modsecurity/tmp (as data and upload directory are also located in /tmp/modsecurity).
This directory is empty. Wouldn't this be the directory for persistent storage and shouldn't there be a file for the IP collection database?

This is the rules I use:

# xbl.spamhaus.org to block malicious/infected ips
SecAction \
    "phase:1,id:1100,\
    t:none,pass,nolog,\
    tag:'COLLECTIONS',\
    initcol:ip=%{remote_addr},\
    setvar:'tx.real_ip=%{remote_addr}'"

SecRule IP:SPAMMER "@eq 1" \
    "phase:1,id:1101,\
    t:none,deny,log,auditlog,\
    msg:'Request from Known SPAM Source (Previous RBL Match)',\
    tag:'AUTOMATION/MALICIOUS',\
    severity:'CRITICAL',\
    setvar:'tx.msg=%{rule.msg}'"

SecRule IP:PREVIOUS_RBL_CHECK "@eq 1" "phase:1,id:1102,t:none,pass,nolog,skipAfter:END_RBL_LOOKUP"

SecAction "phase:1,id:1103,\
    t:none,pass,nolog,\
    setvar:ip.previous_rbl_check=1,\
    expirevar:ip.previous_rbl_check=3600"

SecRule REMOTE_ADDR "@rbl xbl.spamhaus.org." \
   "phase:1,id:1104,\
    t:none,deny,log,auditlog,\
    msg:'RBL Match for SPAM Source',\
    tag:'AUTOMATION/MALICIOUS',\
    severity:'CRITICAL',\
    setvar:'tx.msg=%{rule.msg}',\
    setvar:ip.spammer=1,\
    expirevar:ip.spammer=3600"

SecMarker END_RBL_LOOKUP

This is part of my setup.conf before including the CRS rules. As said, it worked much better (but also not perfect) with the older docker image mentioned above.

@ne20002 ne20002 added the 3.x Related to ModSecurity version 3.x label May 29, 2025
@ne20002 ne20002 changed the title expirevar does not seem to work properly with latest ModSecurity ip collection does not seem to work properly with latest ModSecurity May 29, 2025
@ne20002
Copy link
Author

ne20002 commented May 29, 2025

I have tried to capture some log output and this is what I found:

Occurence of 205.166.94.38:


[174852413370.405117] [/inbox] [4] (Rule: 1100) Executing unconditional rule...
[174852413370.405117] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852413370.405117] [/inbox] [4] Running (non-disruptive) action: tag
[174852413370.405117] [/inbox] [5] Collection `ip' initialized with value: 205.166.94.38
[174852413370.405117] [/inbox] [4] Running (disruptive)     action: pass.
[174852413370.405117] [/inbox] [4] (Rule: 1101) Executing operator "Eq" with param "1" against IP:SPAMMER.
[174852413370.405117] [/inbox] [4] Rule returned 0.
[174852413370.405117] [/inbox] [4] (Rule: 1102) Executing operator "Eq" with param "1" against IP:PREVIOUS_RBL_CHECK.
[174852413370.405117] [/inbox] [4] Rule returned 0.
[174852413370.405117] [/inbox] [4] (Rule: 1103) Executing unconditional rule...
[174852413370.405117] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852413370.405117] [/inbox] [4] Running (disruptive)     action: pass.
[174852413370.405117] [/inbox] [4] (Rule: 1104) Executing operator "Rbl" with param "xbl.spamhaus.org." against REMOTE_ADDR.
[174852413370.405117] [/inbox] [5] RBL lookup of 205.166.94.38 failed.
[174852413370.405117] [/inbox] [4] Rule returned 0.

[174852421441.145581] [/inbox] [4] (Rule: 1100) Executing unconditional rule...
[174852421441.145581] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852421441.145581] [/inbox] [4] Running (non-disruptive) action: tag
[174852421441.145581] [/inbox] [5] Collection `ip' initialized with value: 205.166.94.38
[174852421441.145581] [/inbox] [4] Running (disruptive)     action: pass.
[174852421441.145581] [/inbox] [4] (Rule: 1101) Executing operator "Eq" with param "1" against IP:SPAMMER.
[174852421441.145581] [/inbox] [4] Rule returned 0.
[174852421441.145581] [/inbox] [4] (Rule: 1102) Executing operator "Eq" with param "1" against IP:PREVIOUS_RBL_CHECK.
[174852421441.145581] [/inbox] [4] Rule returned 0.
[174852421441.145581] [/inbox] [4] (Rule: 1103) Executing unconditional rule...
[174852421441.145581] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852421441.145581] [/inbox] [4] Running (disruptive)     action: pass.
[174852421441.145581] [/inbox] [4] (Rule: 1104) Executing operator "Rbl" with param "xbl.spamhaus.org." against REMOTE_ADDR.
[174852421441.145581] [/inbox] [5] RBL lookup of 205.166.94.38 failed.

[174852413370.405117] [/inbox] [4] (Rule: 1100) Executing unconditional rule...
[174852413370.405117] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852413370.405117] [/inbox] [4] Running (non-disruptive) action: tag
[174852413370.405117] [/inbox] [5] Collection `ip' initialized with value: 205.166.94.38
[174852413370.405117] [/inbox] [4] Running (disruptive)     action: pass.
[174852413370.405117] [/inbox] [4] (Rule: 1101) Executing operator "Eq" with param "1" against IP:SPAMMER.
[174852413370.405117] [/inbox] [4] Rule returned 0.
[174852413370.405117] [/inbox] [4] (Rule: 1102) Executing operator "Eq" with param "1" against IP:PREVIOUS_RBL_CHECK.
[174852413370.405117] [/inbox] [4] Rule returned 0.
[174852413370.405117] [/inbox] [4] (Rule: 1103) Executing unconditional rule...
[174852413370.405117] [/inbox] [4] Running [independent] (non-disruptive) action: setvar
[174852413370.405117] [/inbox] [4] Running (disruptive)     action: pass.
[174852413370.405117] [/inbox] [4] (Rule: 1104) Executing operator "Rbl" with param "xbl.spamhaus.org." against REMOTE_ADDR.
[174852413370.405117] [/inbox] [5] RBL lookup of 205.166.94.38 failed.

So setting the ip address to the collection seems to have no effect?

@airween
Copy link
Member

airween commented May 30, 2025

HI @ne20002,

thanks for reporting this issue, and for the detailed explanation.

Let me check this soon and give a feedback.

@airween airween self-assigned this May 30, 2025
@ne20002
Copy link
Author

ne20002 commented May 31, 2025

I have done a logging session with debug level 9. It creates a rather big log file. From what I see there is no other log statement related to collection. I would have expected at least something according to storing the collection. But .. nothing like that.

@airween, if should do some special tests or can help in any other way ...

@airween
Copy link
Member

airween commented Jun 12, 2025

@ne20002

sorry for the late reply, finally I was able to take a look at this issue.

I tried your setup and got the same result - especially this line in debug.log:

RBL lookup of 205.166.94.38 failed.

I think this line is generated by this part of @rbl operator's code.

    rc = getaddrinfo(host.c_str(), NULL, NULL, &info);

    if (rc != 0) {
        if (info != NULL) {
            freeaddrinfo(info);
        }
        ms_dbg_a(t, 5, "RBL lookup of " + ipStr + " failed.");
        return false;
    }

The used host variable is set up by mapIpToAddress function, this produces the expected format by Spamhaus - in your example above, from the IP 205.166.94.38 this function creates the host name 38.94.166.205.xbl.spamhaus.org and does a DNS query. But because that IP is not on the list, therefore the getaddrinfo() returns with a non-zero value.

I suggest you to first try to use the IP collection with some other way, eg. use a static list with @ipMatch operator.

@ne20002
Copy link
Author

ne20002 commented Jun 13, 2025

Thank you @airween for the answer. Actually the lookup of 38.94.166.205.xbl.spamhaus.org is not the problem.

The inverse number order is how this is handled by the rbl systems (they all use it) and a query to 38.94.166.205.xbl.spamhaus.org results in NXDOMAIN as expected as 205.166.94.38 is not listed as a spam source. So returning false here is correct and 205.94.166.205 should not be added to the ip:spammers collection.

But the 205.166.94.38 seems also not to be added to ip:previous_rbl_check and this is a problem. The ip:previous_rbl_check in this setup is used to bypass further checks for requests from 205.166.94.38.

This is what I wanted to achieve:

  • Rule 1101: Check if ip is a known spammer by looking up ip:spammer. If the ip is known as spammer deny the request.

If the ip is not known as spammer...

  • Rule 1102: Check is the ip has been checked already within the last hour (timeout 3600s on the ip:previous_rbl_check). If the ip is known as already checked then skip the following test.

If the ip has not been tested before...

  • Rule 1103: mark the ip as tested in the ip collection with ip.previous_rbl_check set to 1

  • Rule 1104: now test the ip with the rbl check. If the ip is a spammer, add it to ip.collection setting ip:spammer=1

So I expect for the second and following requests from an ip address (within the 3600s timeframe) that it is

  • either known as spammer and the request is denied or

  • the rbl check is skipped as it has already been done.

The last part is the problem. From what I see is, that the checks on the ip collection (for ip:spammer or ip:previous_rbl_check) always return 0. And this then causes a rbl check for each request as I see it on my pi-hole.

The interessting part is, that it has worked before. So I believe the problem is not the rbl check but the collection.

@airween
Copy link
Member

airween commented Jun 13, 2025

hi @ne20002,

ah, thanks for the clarification, now it's clear. I see the problem.

The interessting part is, that it has worked before. So I believe the problem is not the rbl check but the collection.

Do you remember in which version was that this worked as well?

@ne20002
Copy link
Author

ne20002 commented Jun 13, 2025

I'm using the docker.io/owasp/modsecurity-crs:nginx container, but with my own configuration which includes the setup as described above. I don't use the container's startup scripts, just the nginx with the modsecurity module.

With a previous version (docker.io/owasp/modsecurity-crs:4.10-nginx-202501050801) this worked not fully, but mostly well.

I also read the documentation, and based on that:

  • docs say I can't have more than one expirevar statement per rule, that's the reason for having rule 1103 (and it also worked with the previous image).
  • the trx collection doesn't have problems.

So I don't think it is the collection code except for (based on what I see happening here):

  • it might be a problem with the persistance (one difference between trx and ip collection) or/and
  • synchronization (ip collection should be shared between requests, trx not) or/and
  • maybe a size problem with the collection or problem with the expirevar code (not used in trx)

This is also why I looked for the persistant storage file. And I haven't found one (so this e.g. might be a container problem).
I don't have the ability to setup a plain nginx with modsecurity so that is why I'm limited to the container.

So I'm limited in options to dig deeper but still I'm willing to try or test whatever you need.

@airween
Copy link
Member

airween commented Jun 13, 2025

CRS's Docker uses LMDB (based on this configuration part), which means you have to have a file inside the container with name modsec-shared-collections (may be with some extension).

Here you can find a Gist, with that you can read the content of the LMDB file (which is a "database": keys and values).

Could you try to read that file (and share the relevant content here)?

@ne20002
Copy link
Author

ne20002 commented Jun 14, 2025

I think I don't have that file.

pi@dmz1:~$ podman exec -it friendica-web /bin/bash
nginx@friendica:/$ id
uid=101(nginx) gid=101(nginx) groups=101(nginx)
nginx@friendica:/$ cd /tmp
nginx@friendica:/tmp$ ls -la 
total 8
drwxrwxrwt 8 root  root   180 Jun  6 00:12 .
dr-xr-xr-x 1 root  root  4096 Jun  6 00:12 ..
drwx------ 2 nginx nginx   40 Jun  6 00:12 client_temp
drwx------ 2 nginx nginx   40 Jun  6 00:12 fastcgi_temp
drwxr-xr-x 5 nginx nginx  100 Jun  6 00:12 modsecurity
-rw-r--r-- 1 nginx nginx    2 Jun  6 00:12 nginx.pid
drwx------ 2 nginx nginx   40 Jun  6 00:12 proxy_temp_path
drwx------ 2 nginx nginx   40 Jun  6 00:12 scgi_temp
drwx------ 2 nginx nginx   40 Jun  6 00:12 uwsgi_temp
nginx@friendica:/tmp$ cd modsecurity/
nginx@friendica:/tmp/modsecurity$ ls -la
total 0
drwxr-xr-x 5 nginx nginx 100 Jun  6 00:12 .
drwxrwxrwt 8 root  root  180 Jun  6 00:12 ..
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 data
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 tmp
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 upload
nginx@friendica:/tmp/modsecurity$ cd data
nginx@friendica:/tmp/modsecurity/data$ ls -la
total 0
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 .
drwxr-xr-x 5 nginx nginx 100 Jun  6 00:12 ..
nginx@friendica:/tmp/modsecurity/data$ cd ../tmp
nginx@friendica:/tmp/modsecurity/tmp$ ls -la
total 0
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 .
drwxr-xr-x 5 nginx nginx 100 Jun  6 00:12 ..
nginx@friendica:/tmp/modsecurity/tmp$ cd ../upload/
nginx@friendica:/tmp/modsecurity/upload$ ls -la
total 0
drwxr-xr-x 2 nginx nginx  40 Jun  6 00:12 .
drwxr-xr-x 5 nginx nginx 100 Jun  6 00:12 ..
nginx@friendica:/tmp/modsecurity/upload$ 
nginx@friendica:/$ find / -type f -name "modsec-shared-collection*"
find: '/root': Permission denied
find: '/etc/ssl/private': Permission denied
find: '/var/cache/ldconfig': Permission denied
find: '/var/cache/apt/archives/partial': Permission denied
find: '/var/lib/apt/lists/partial': Permission denied
find: '/proc/tty/driver': Permission denied

@ne20002
Copy link
Author

ne20002 commented Jun 14, 2025

@airween

Can you help me with defining (a few) rules that should result in writing a lmdb file? Maybe by setting dummy values?
I will then check if it works and also do a debug log on debug level 9.

This way we may get info if writing a lmdb is working at all or not with the container.

@airween
Copy link
Member

airween commented Jun 14, 2025

@ne20002,

Today I investigated this issue and found a few problems. I will write a summary later.

@airween
Copy link
Member

airween commented Jun 14, 2025

Here is what I found:

  • in case of LMDB, collections' data (except TX) stored in a file
  • the path of this file is hardcoded
  • this path is the / (root) directory; engine needs to create, open and write files /modsec-shared-collections and /modsec-shared-collections-lock in name of running user (www-data? or I don't know what's that in Docker...)
  • unfortunately if the engine cannot open the file, this fact remains hidden - see this part of code; there is no else branch (rc == 0 means the file was opened successfully)
  • the other problem is that the collections are case sensitive in case of using LMDB; if you want to store and retrieve a variable in a collection (like you do in your example), then you should put and refer the key with the same format. In your rules you use mixed capital and lower case.

I changed your rules and set the keys to upper case format, then I get this:

[174993034342.235412] [/] [4] (Rule: 1102) Executing operator "Eq" with param "1" against IP:PREVIOUS_RBL_CHECK.
[174993034342.235412] [/] [4] Rule returned 0.
[174993034342.235412] [/] [9] Matched vars cleaned.
[174993034342.235412] [/] [4] (Rule: 1103) Executing unconditional rule...
[174993034342.235412] [/] [4] Running [independent] (non-disruptive) action: setvar
--
[174993035551.202951] [/?q=1] [4] (Rule: 1102) Executing operator "Eq" with param "1" against IP:PREVIOUS_RBL_CHECK.
[174993035551.202951] [/?q=1] [9] Target value: "1" (Variable: IP:172.35.40.50::::PREVIOUS_RBL_CHECK)
[174993035551.202951] [/?q=1] [9] Matched vars updated.
[174993035551.202951] [/?q=1] [4] Rule returned 1.
[174993035551.202951] [/?q=1] [9] Saving msg: Found IP in collection.

As you can see, in the second request the rule 1102 matched.

Other sad fact, that this does not work in case of in-memory backend. In that case the collection handling does not work to me at all - this needs more investigations.

Workaround

Even though I changed the code and put the database to under /tmp, I tried these steps:

sudo systemctl nginx stop
sudo rm /tmp/modsec-shared-collections*
sudo touch /tmp/modsec-shared-collections /tmp/modsec-shared-collections-lock
sudo chown www-data:www-data /tmp/modsec-shared-collections /tmp/modsec-shared-collections-lock
sudo systemctl nginx start

After the engine started and received the first request, the files were filled with data, so this could work in your case too.

I don't know how to solve it permanently in Docker. @theseion do you have some idea?

Note that if the engine runs and uses those files, you can open and look its content with this version of lmdbread, this can open the file for read in a different process.

@ne20002
Copy link
Author

ne20002 commented Jun 15, 2025

Thank you very much @airween

I updated my config to fix the lower-/uppercase issue (all lowercase now).

According to the documentation the storage path for the persistant collections should be SecDataDir (but is not supported in v3). This is by default set to /tmp/modsecurity/data/ in the docker image. This path is created and it is owned by user nginx but is empty. So I assume you're right with the / being the path used.

I tried to create the files as u did with touch by mounting a shell script into the docker-entrypoint.d directory (mounting the script file worked).
I could touch the files in /tmp/modsecurity/data/ but it is not used (still always 0 bytes size). So again, SecDataDir seems not be used as path.
I wasn't able to create the files in the root directory or in /var/www. In both cases I got a 'Permission denied'. Seems as if the scripts in docker-entrypoint.d are runned with the nginx user.

I have now created those two files on my host and mounted them rw to ogw to the root directory inside the container. The files seems to be used as the size increased. According to the debug log it now works. :)

I believe the best option to solve the issue would be to fix the creation of the persistent storage to use SecDataDir as according to the documentation. If so, no change to the docker image would be required.

And also an error log message should be written if the attempt to create or write to the persistent store fails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.x Related to ModSecurity version 3.x
Projects
None yet
Development

No branches or pull requests

2 participants