Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[dashbord] Investigate and fix N+1 query issue #1806

Open
bmispelon opened this issue Dec 8, 2024 · 2 comments · May be fixed by #1813
Open

[dashbord] Investigate and fix N+1 query issue #1806

bmispelon opened this issue Dec 8, 2024 · 2 comments · May be fixed by #1813
Labels
help-needed Help needed ops Operations python Pull requests that update Python code

Comments

@bmispelon
Copy link
Member

I noticed today that Sentry was reporting a possible N+1 query issue on the dashboard homepage.
Here's the query that Sentry is claiming is being repeated:

SELECT dashboard_datum.id, dashboard_datum.content_type_id, dashboard_datum.object_id,
  dashboard_datum.timestamp, dashboard_datum.measurement
FROM dashboard_datum
WHERE (
  dashboard_datum.content_type_id = %s AND dashboard_datum.object_id = %s
)
ORDER BY dashboard_datum.timestamp DESC
LIMIT 1

I'd recomment trying to see ifthe Django Debug Toolbar (installed by default I think) is reporting the duplicated query also. And if so, we should try to fix it. Usually it's a matter of adding a select_related() or prefetch_related() somewhere in the view, but it might also be more complicated than that.
@bmispelon bmispelon added python Pull requests that update Python code ops Operations help-needed Help needed labels Dec 8, 2024
@bmispelon bmispelon changed the title [dashbaord] Investigate and fix N+1 query issue [dashbord] Investigate and fix N+1 query issue Dec 8, 2024
@ontowhee
Copy link
Contributor

ontowhee commented Dec 9, 2024

DjangoDebugToolbar is reporting this query as repeating 19 times, which corresponds to 1 time for each metric that is displayed on the dashboard.

Screenshot 2024-12-08 at 9 05 07 PM

There is a for loop iterating over the metrics to get the latest datum for each metric.
https://github.com/django/djangoproject.com/blob/main/dashboard/views.py#L19-L30

I'll open a draft PR that seems to reduce the number of sql queries down from 35 to 9 for the dashboard page.

@ontowhee ontowhee linked a pull request Dec 9, 2024 that will close this issue
Open
@Mamatha1718
Copy link

Hi @bmispelon
As I observed, data is a reverse relationship (defined using GenericRelation in the Metric model), so I have updated the code to use prefetch_related("data") to optimize the query. Please review the changes in the PR.
Will you please let me know any changes need.
Thank you.

Mamatha1718 added a commit to Mamatha1718/djangoproject.com that referenced this issue Dec 30, 2024
@Mamatha1718
Investigate and fix N+1 query issue django#1806
6b41ad6
Mamatha1718 added a commit to Mamatha1718/djangoproject.com that referenced this issue Dec 30, 2024
@Mamatha1718
Investigate and fix N+1 query issue django#1806
5029aa3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help-needed Help needed ops Operations python Pull requests that update Python code
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix #1806: N+1 query issue on dashboard index page ontowhee/djangoproject.com
3 participants
@bmispelon @ontowhee @Mamatha1718