Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove old device_connections in worker, remove connection state stored on device #1807

Merged
merged 6 commits into from
Jan 23, 2025
Merged

Remove old device_connections in worker, remove connection state stored on device #1807

merged 6 commits into from
Jan 23, 2025

Conversation

nshoes
Copy link
Contributor

@nshoes nshoes commented Jan 22, 2025 •
edited
Loading

  • Introduce device.latest_connection, which references the latest device_connection record for the device
  • Start cleaning up device_connections every hour with NervesHub.Workers.DeleteOldDeviceConnections
    • This targets rows that have a last_seen_at older than two weeks by default (configurable), a status of :disconnected and does not delete a device's last device_connection, even if the previous criteria are met
  • Remove connection_status, connection_established_at, and connection_last_seen_at from the device schema.
  • Slot in device.latest_connection, simplifying a lot of queries and logic related to device_connections
  • Remove Devices.clean_connection_states and it's call from a worker since device.connection_status was removed and connection logic is moving to device.latest_connection
  • When a device connects to the socket, we set the created device_connection as device.lastest_connection
  • Create device.latest_connection when seeding devices

Todo:

  • check filtering UI locally

@nshoes nshoes force-pushed the remove-old-device-connections-worker branch from 87be70f to ab30c3d Compare January 23, 2025 01:18
joshk
joshk reviewed Jan 23, 2025
View reviewed changes
lib/nerves_hub/devices.ex Outdated Show resolved Hide resolved
joshk
joshk reviewed Jan 23, 2025
View reviewed changes
lib/nerves_hub/devices/connections.ex Outdated
|> join(:inner, [dc], d in Device, on: dc.device_id == d.id)
|> where([dc, _d], dc.last_seen_at < ^days_ago)
|> where([dc, _d], dc.status != :connected)
|> where([dc, d], dc.id != d.latest_connection_id)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is sooooo smart!!!!!

joshk
joshk reviewed Jan 23, 2025
View reviewed changes
lib/nerves_hub/tracker.ex Outdated Show resolved Hide resolved
elinol
elinol reviewed Jan 23, 2025
View reviewed changes
lib/nerves_hub/devices/device.ex Outdated
@@ -44,6 +40,7 @@ defmodule NervesHub.Devices.Device do
embeds_one(:firmware_metadata, FirmwareMetadata, on_replace: :update)
has_many(:device_certificates, DeviceCertificate, on_delete: :delete_all)
has_many(:device_connections, DeviceConnection, on_delete: :delete_all)
has_one(:latest_connection, DeviceConnection)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

@nshoes
Remove old and disconnected device_connections in
e666f7f 
worker, introduce device.latest_connection, remove
unused connection references on devices
@nshoes nshoes force-pushed the remove-old-device-connections-worker branch from ab30c3d to e666f7f Compare January 23, 2025 18:37
@nshoes nshoes mentioned this pull request Jan 23, 2025
Open
nshoes
nshoes commented Jan 23, 2025
View reviewed changes
lib/nerves_hub/workers/schedule_org_audit_log_truncation.ex
@@ -1,6 +1,7 @@
defmodule NervesHub.Workers.ScheduleOrgAuditLogTruncation do
use Oban.Worker,
queue: :truncation
queue: :truncation,
max_attempts: 1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this from config.exs.

@nshoes nshoes requested review from joshk, jjcarstens and elinol January 23, 2025 19:07
@nshoes nshoes changed the title Remove old device_connections in worker Remove old device_connections in worker, remove connection state stored on device Jan 23, 2025
joshk
joshk requested changes Jan 23, 2025
View reviewed changes
lib/nerves_hub/devices.ex Outdated Show resolved Hide resolved
lib/nerves_hub/devices.ex Show resolved Hide resolved
lib/nerves_hub/devices.ex Show resolved Hide resolved
lib/nerves_hub/devices/connections.ex Show resolved Hide resolved
priv/repo/seeds.exs Outdated Show resolved Hide resolved
joshk
joshk approved these changes Jan 23, 2025
View reviewed changes
Copy link
Collaborator

@joshk joshk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me!

joshk
joshk approved these changes Jan 23, 2025
View reviewed changes
Copy link
Collaborator

@joshk joshk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment, but lets test this out!

lib/nerves_hub/devices/connections.ex Show resolved Hide resolved
@nshoes nshoes force-pushed the remove-old-device-connections-worker branch from 6c8f522 to bd8e1c6 Compare January 23, 2025 22:54
@nshoes nshoes merged commit 53e6dc5 into main Jan 23, 2025
2 checks passed
@nshoes nshoes deleted the remove-old-device-connections-worker branch January 23, 2025 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joshk joshk joshk approved these changes

@jjcarstens jjcarstens Awaiting requested review from jjcarstens

@elinol elinol Awaiting requested review from elinol

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nshoes @joshk @jjcarstens @elinol