Skip to content

phone_number warninglist contains invalid regex, and causes slowdowns when enabled #298

@UFOSmuggler

Description

@UFOSmuggler

This regex based warninglist contains the regex global g modifier/flag to match all occurrences, which causes preg_match in the warninglist model to fail since preg_match stops after the first occurrence so does not support g:

2025-05-20 04:25:38 Warning: Warning (2): preg_match() [<a href='http://php.net/function.preg-match'>function.preg-match</a>]: Unknown modifier 'g' in [/var/www/MISP/app/Model/Warninglist.php, line 738]
Trace:
ErrorHandler::handleError() - APP/Lib/cakephp/lib/Cake/Error/ErrorHandler.php, line 232
preg_match - [internal], line ??
Warninglist::__evalRegex() - APP/Model/Warninglist.php, line 738
Warninglist::checkValue() - APP/Model/Warninglist.php, line 669
WarninglistsController::checkValue() - APP/Controller/WarninglistsController.php, line 497
ReflectionMethod::invokeArgs() - [internal], line ??
Controller::invokeAction() - APP/Lib/cakephp/lib/Cake/Controller/Controller.php, line 500
Dispatcher::_invoke() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 193
Dispatcher::dispatch() - APP/Lib/cakephp/lib/Cake/Routing/Dispatcher.php, line 167
[main] - APP/webroot/index.php, line 107

preg_match_all would be more appropriate for the author's intended use, though this does not appear to be a part of the warninglist model's execution pipeline.

It looks like this would need to be fixed here: https://github.com/MISP/misp-warninglists/blob/main/tools/generate_phone_numbers.py

When querying the warninglists endpoint with a lot of indicators, this can cause as much as a 5-6x increase in execution time. as an example, 70k indicators took 1 minute with this warninglist enabled, 11 seconds with it disabled. i suspect this is due to php having to write 70k errors to the log, and perhaps reinitialise a portion of each worker once preg_match fails 70k times.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions