Sohail and Anthony - BLUE SQUARE EMAIL AUTOMATION

[AnthonyWeathers]

Description

Continues off of PR#1104 and PR#1334

Task: (PRIORITY HIGH) Jae: New Email function: Auto-reply for people who just forgot their summary on HGN Phase I Bugs and Needed Functionalities

Related PRs:

No frontend PR needed.

Main changes explained:

1. Robust Email Threading

• New Feature: Implemented intelligent email threading logic in emailSender.js.
• Mechanism: Emails are now grouped by recipientUserId and weekStart. The system checks for an existing thread in the EmailThread collection.
• Result: Weekly reminder emails will appear as a single conversation in Gmail/Outlook (using In-Reply-To and References headers), significantly reducing inbox clutter for users and admins.

2. Safe & Parameterized Testing

• New Script: Created src/scripts/testEmailJobs.js for safe, isolated testing.
• Safety: Instead of manually hardcoding test emails in userHelper.js, the functions now accept an optional emailConfig object.
• Logic: When testing, you pass { targetUserId, emailOverride, ccOverride, bccOverride }. The system guarantees 0 emails go to production users or lists when this mode is active.

3. Automated Email Functions (userHelper.js)

We significantly refactored/created these 4 functions to support the new features:

1. assignBlueSquareForTimeNotMet: Checks if a user failed to meet their weekly committed hours. If so, assigns a blue square and sends a notification.
2. completeHoursAndMissedSummary: Identifies users who met their hours but forgot to submit their weekly summary, sending a specific reminder to "Reply All".
3. inCompleteHoursEmailFunction: detailed checks for users who are partially complete (e.g., <85% or <65% of hours), sending tailored warnings based on how much they missed.
4. weeklyBlueSquareReminderFunction: audits the user's total blue squares (e.g., "2 in 2 months" or "4th Blue Square"), alerting them if they are close to the 5-strike limit.

4. Database Collection Updates

To enable the threading logic, we utilized EmailThread and optimized EmailHistory:

• EmailThread Collection: Stores distinct conversation threads keyed by recipientUserId + weekStart. This allows us to look up the threadRootMessageId for any given week.
• EmailHistory Collection: Updated to store RFC-compliant threading headers (Message-ID, In-Reply-To, References). This safeguards our ability to debug email chains if providers (Gmail/Outlook) behave unexpectedly.

5. Cron Job Updates (userProfileJobs.js)

• New Schedule: Added a new cron job at 4:00 AM PST every Sunday.
• Reasoning: separating these checks from the midnight jobs ensures all time-logging and badge processing is finished before we assess blue squares.

How to test:

We have introduced a dedicated test script so you do NOT need to modify the code to test it.

1. Follow this doc (Link) and update your .env file as instructed in step 2.
2. Check out the branch: git checkout <branch-name>
3. Setup: Ensure your local DB is running or connected to the dev DB. (You have the updated DB credentials in your .env)
4. Configure Test: Open src/scripts/testEmailJobs.js and update the constants at the top:

• Provide a valid USER_ID from your Dev account(s).

  const TARGET_USER_ID = '...'; // ID of a user who fits the email criteria (e.g. missed hours)
  const TESTER_EMAIL = '[email protected]';  // Recipient
  const TESTER_CC = '[email protected]';
  const TESTER_BCC = '[email protected]'; 

• Sender email ID will be setup in .env by following instructions in linked doc above.

5. Run the Test:

   node src/scripts/testEmailJobs.js

• Test for different email service providers. (e.g. Gmail, Outlook etc.)

6. Verify:

• Check your test inbox. You should receive the emails directed only to the addresses you defined.
• Threading Check: If you run the script multiple times for the same target user, the emails should thread together in your inbox.
• Check 'emailHsitory' and 'emailThreads' collections in MongoDB Compass to verify that valid entries with the correct details are being made. Also use the threadRootMessageId and weekStart fields in 'emailThreads' to uniquely filter entries in the 'emailHistory' collection.

Safety Features Implemented:

• Automatic Override: If targetUserId is passed, the system forces all CCs and BCCs to use the override list.
• Mandatory Safety Check: The test script will exit immediately if you do not provide test CC/BCC addresses, ensuring no accidental spam to the production DEFAULT_BCC_EMAILS list.

Note:

The expected flow of executions in prod is something like this:

• assignBlueSquareForTimeNotMet will run at the start of every week, i.e. Sunday 12 AM PST.
• completeHoursAndMissedSummary, inCompleteHoursEmailFunction, and weeklyBlueSquareReminderFunction will run 4 hours later. i.e. Sunday 4 am PST.

Important

• The test script currently runs the first two functions assignBlueSquareForTimeNotMet and completeHoursAndMissedSummary. You can uncomment other function calls in src/scripts/testEmailJobs.js to test them individually.
• Make sure to test all 4 functions.
• Make sure to test for for all possible vulnerabilities and edge cases.

[Copilot]

Pull request overview

This PR implements automated blue square email notifications with intelligent email threading to reduce inbox clutter. The system groups weekly reminder emails into single conversation threads using RFC-compliant Message-ID headers and introduces safe testing infrastructure to prevent accidental production emails.

Key Changes:

• Email threading system using EmailThread and EmailHistory models with RFC-compliant headers
• Four new automated email functions for different blue square scenarios (time not met, missed summary, incomplete hours, weekly reminders)
• Parameterized testing infrastructure via testEmailJobs.js to safely test email functions without spamming production users
• Updated cron schedule to run blue square checks at 4 AM PST on Sundays, after time-logging completion

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 26 comments.

Show a summary per file

File	Description
src/utilities/timeUtils.js	Enhanced timezone consistency by explicitly using Pacific time for all date formatting functions
src/utilities/emailSender.js	Implemented email threading with Message-ID generation, thread tracking via EmailThread model, and promise-based queue processing
src/utilities/__tests__/timeUtils.test.js	Added comprehensive unit tests for time utility functions
src/utilities/__tests__/objectUtils.test.js	Added unit tests for object manipulation utilities
src/scripts/testEmailJobs.js	Created testing script with safety checks to test email functions without affecting production users
src/models/emailThread.js	New model to track email thread roots by user and week for proper conversation threading
src/models/emailHistory.js	Extended with threading fields (messageId, threadRootMessageId, references) and metadata for RFC compliance
src/models/emailModels.spec.js	Comprehensive validation tests for EmailThread and EmailHistory schemas
src/helpers/userHelper.js	Refactored blue square functions to support testing parameters and added three new email automation functions
src/helpers/__tests__/getInfringementEmailBody.test.js	Unit tests for infringement email body generation logic
src/cronjobs/userProfileJobs.js	Added new cron job at 4 AM PST for summary-related emails and updated existing job schedule

Comments suppressed due to low confidence (2)

src/helpers/userHelper.js:1359

• Unused variable numMonthsOriginal.

        const numMonthsOriginal = currentMonthStart.diff(startOfMonth, 'months');

src/helpers/userHelper.js:1498

• Unused variable numMonthsOriginal.

          const numMonthsOriginal = currentMonthStart.diff(startOfMonth, 'months');

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sohailuddinsyed]

Tested all functions and they work as expected.

[Anusha-Gali]

Hi Anthony,

I have reviewed your PR locally and have been able to test all the scenarios. Thank you being patient and guiding me through the issues i faced.

[one-community]

Thank you all, merging!