Centralize python monitoring artifact and simplify daemon monitoring with structured concurrency

[tiagonix]

Description

The tracer client currently writes and tails a python monitoring artifact using a hardcoded filesystem path, which creates a hidden dependency and breaks the invariant that file locations are resolved by a single authoritative configuration surface. In addition, the daemon process monitor orchestrates multiple workers using repetitive spawn and error-handling patterns that materially increase complexity without increasing correctness.

Rationale

This work restores compliance with the principles and invariants governing centralized configuration and simplicity. Centralizing the python monitoring file under the tracer workdir ensures that file layout is consistent, testable, and portable across environments. Refactoring the daemon monitor to a structured concurrency pattern reduces boilerplate, clarifies responsibility boundaries, and makes error propagation and shutdown mechanics easier to reason about. Replacing implicit task detach-on-drop with explicit cancellation and drain makes the shutdown sequence deterministic and eliminates a potential mutex deadlock between exiting workers and the final data submission.

Scope and boundaries

This is a behavior-preserving refactor with one intentional shutdown improvement. It must not alter monitoring semantics, error propagation expectations, or the functional meaning of the python monitoring feature. The change is limited to routing file locations through the workdir abstraction and replacing repetitive task orchestration with a JoinSet-based worker lifecycle. Shutdown semantics are strengthened: when any worker exits (panic, timeout, or cancellation), the server token broadcasts cancellation to all remaining workers, which are drained via the JoinSet before the final data submission acquires the client mutex.

Tasks

• Add a dedicated python_monitoring_file artifact path to the tracer workdir abstraction and initialize it consistently with other workdir artifacts.
• Replace all direct uses of the hardcoded python monitoring path with the workdir-provided artifact path.
• Update the tailing helper signature to accept a &Path to remove stringly-typed path passing.
• Refactor the daemon process monitor to use tokio::task::JoinSet for spawning and awaiting worker tasks, consolidating error handling and shutdown logic.
• Cancel the server token on any worker exit and drain the JoinSet before the final data submission to prevent mutex deadlock between live workers and the shutdown path.
• Return worker name from each spawned task to preserve per-worker identification in shutdown logs.

File manifest

• src/tracer/src/utils/workdir.rs
• src/tracer/src/client/tracer_client.rs
• src/tracer/src/daemon/server/process_monitor.rs

Validation

• Existing tests pass unchanged.
• Manual run confirms python monitoring file is created under the configured workdir.
• Daemon monitor behavior remains consistent under normal operation and under worker failure.
• Shutdown sequence verified: cancellation token propagates to all workers and JoinSet is drained before final data submission.

Acceptance criteria

• No remaining hardcoded python monitoring filesystem path exists in the client.
• Python monitoring I/O is routed through the workdir abstraction as a first-class artifact.
• The daemon monitor no longer contains repetitive spawn/error blocks and uses structured concurrency for worker management.
• Shutdown is deterministic: server token cancels on any worker exit, all workers are drained, and the client mutex is acquired only after all workers have terminated.