Skip to content

perf(RHINENG-24466): skip host_count query during MQ ingestion#3843

Open
rodrigonull wants to merge 1 commit intomasterfrom
perf/skip-host-count-in-mq
Open

perf(RHINENG-24466): skip host_count query during MQ ingestion#3843
rodrigonull wants to merge 1 commit intomasterfrom
perf/skip-host-count-in-mq

Conversation

@rodrigonull
Copy link
Copy Markdown
Member

@rodrigonull rodrigonull commented Mar 26, 2026

Overview

This PR is being created to address RHINENG-24466.

Add with_host_count parameter to serialize_group() and pass False from add_host() during MQ ingestion. The ingress path only needs group metadata for the JSONB groups field, not an accurate host count. This eliminates 1 COUNT/JOIN query per message.

PR Checklist

  • Keep PR title short, ideally under 72 characters
  • Descriptive comments provided in complex code blocks
  • Include raw query examples in the PR description, if adding/modifying SQL query
  • Tests: validate optimal/expected output
  • Tests: validate exceptions and failure scenarios
  • Tests: edge cases
  • Recovers or fails gracefully during potential resource outages (e.g. DB, Kafka)
  • Uses type hinting, if convenient
  • Documentation, if this PR changes the way other services interact with host inventory
  • Links to related PRs

Secure Coding Practices Documentation Reference

You can find documentation on this checklist here.

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Summary by Sourcery

Make group serialization optionally skip host count calculation and use this faster path during MQ host ingestion.

Enhancements:

  • Add a with_host_count flag to group serialization to allow bypassing host count queries when an accurate count is not required.

Tests:

  • Add tests verifying that serialize_group skips the host count query when with_host_count is False and performs it by default when True.

@github-actions
Copy link
Copy Markdown
Contributor

SC Environment Impact Assessment

Overall Impact:NONE

No SC Environment-specific impacts detected in this PR.

What was checked

This PR was automatically scanned for:

  • Database migrations
  • ClowdApp configuration changes
  • Kessel integration changes
  • AWS service integrations (S3, RDS, ElastiCache)
  • Kafka topic changes
  • Secrets management changes
  • External dependencies

@rodrigonull rodrigonull force-pushed the perf/skip-host-count-in-mq branch 4 times, most recently from 3be1468 to ec010c2 Compare March 31, 2026 17:39
@rodrigonull rodrigonull marked this pull request as ready for review March 31, 2026 18:06
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_models.py" line_range="2843-2852" />
<code_context>
     assert existing_host.display_name_reporter == "yupana"
+
+
+def test_serialize_group_with_host_count_false(flask_app, mocker):  # noqa: ARG001
+    """serialize_group(with_host_count=False) should skip the host count query."""
+    from lib.group_repository import serialize_group
+
+    mock_group = mocker.Mock()
+    mock_group.id = "test-group-id"
+    mock_group.name = "Test Group"
+    mock_group.org_id = "test_org"
+
+    count_spy = mocker.patch("lib.group_repository.get_host_counts_batch")
+    mocker.patch(
+        "lib.group_repository.serialize_db_group_with_host_count",
+        return_value={"id": "test-group-id", "name": "Test Group", "host_count": 0},
+    )
+
+    result = serialize_group(mock_group, "test_org", with_host_count=False)
+
+    count_spy.assert_not_called()
+    assert result["host_count"] == 0
+
+
</code_context>
<issue_to_address>
**suggestion (testing):** Add coverage for the RBAC workspace (`dict`) branch of `serialize_group`.

These tests only cover the DB/ORM `Group` path of `serialize_group`. Please also add tests for the RBAC v2 workspace (`dict`) path that:

1. Call `serialize_group` with a `dict` and `with_host_count=False`, asserting `get_host_counts_batch` is not called and `serialize_rbac_workspace_with_host_count` receives `host_count=0`.
2. Call `serialize_group` with a `dict` and `with_host_count=True` (or default), asserting `get_host_counts_batch` is called once with the expected arguments and that the returned `host_count` is included in the serialized result.

This will ensure both code paths behave consistently and the host-count optimization applies to RBAC workspaces as well.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +2843 to +2852
def test_serialize_group_with_host_count_false(flask_app, mocker): # noqa: ARG001
"""serialize_group(with_host_count=False) should skip the host count query."""
from lib.group_repository import serialize_group

mock_group = mocker.Mock()
mock_group.id = "test-group-id"
mock_group.name = "Test Group"
mock_group.org_id = "test_org"

count_spy = mocker.patch("lib.group_repository.get_host_counts_batch")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (testing): Add coverage for the RBAC workspace (dict) branch of serialize_group.

These tests only cover the DB/ORM Group path of serialize_group. Please also add tests for the RBAC v2 workspace (dict) path that:

  1. Call serialize_group with a dict and with_host_count=False, asserting get_host_counts_batch is not called and serialize_rbac_workspace_with_host_count receives host_count=0.
  2. Call serialize_group with a dict and with_host_count=True (or default), asserting get_host_counts_batch is called once with the expected arguments and that the returned host_count is included in the serialized result.

This will ensure both code paths behave consistently and the host-count optimization applies to RBAC workspaces as well.

Copy link
Copy Markdown
Collaborator

@kruai kruai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - host.groups doesn't have the group's host count, so we don't need to calculate it. Nice thinking!

Add with_host_count parameter to serialize_group() and pass False
from add_host() during MQ ingestion. The ingress path only needs
group metadata for the JSONB groups field, not an accurate host
count. This eliminates 1 COUNT/JOIN query per message.
@rodrigonull rodrigonull force-pushed the perf/skip-host-count-in-mq branch from ec010c2 to 2dcb464 Compare April 1, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants