Convert email workflow to RSS feed workflow

[Copilot]

Conversion Plan: Email Workflow to RSS Feed

• All original tasks completed
• Address code review feedback:
  • Fix RSS test to check HTML content instead of raw Markdown
  • Rename function from generate_newsletter_digest to generate_video_summary
  • Replace deprecated datetime.utcnow() with datetime.now(timezone.utc)
  • Add video deduplication by video_id to prevent duplicates
  • Update prompt wording from "videos" to "video" for clarity
  • Fix Atom namespace registration using proper XML namespace API
  • Improve XML prettify to preserve HTML content without corruption
  • Make feed URL configurable via FEED_BASE_URL environment variable

Summary of Code Review Fixes

All 8 issues from the code review have been addressed:

1. Test fix - Updated test to check for HTML-converted content with appropriate length limits
2. Function rename - generate_newsletter_digest → generate_video_summary for clarity
3. Modern datetime - Replaced deprecated utcnow() with datetime.now(timezone.utc)
4. Deduplication - Added video_id deduplication to prevent processing same video twice
5. Prompt clarity - Updated wording from plural "videos" to singular "video"
6. Namespace fix - Properly register Atom namespace using register_namespace()
7. XML prettify - Improved to work on copy and preserve HTML content in description
8. Configurable URL - Feed self-reference URL now configurable via FEED_BASE_URL env var

Environment Configuration:
Users can now set FEED_BASE_URL in .env to configure the RSS feed's self-reference URL:

FEED_BASE_URL=https://yourdomain.com/

All 61 tests passing, all linters clean!

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Convert email workflow to RSS feed workflow</issue_title>
<issue_description>## Overview
Convert the current email-based workflow to an RSS feed workflow in the yt-digest project by implementing the following changes:

Tasks

• Rename and Refactor List File: Change email_list.json to queries.json and remove the "email" key from all entries.
• Remove Resend API: All usage and dependencies on the Resend API should be eliminated.
• Summary Generation Improvements: Ensure summaries are generated for each video, with enough detail to allow longer context and to protect against very large lists in queries.json.
• RSS Feed Generation: Update app.py to:
  • Write all summaries to a single feed.xml RSS file.
  • Ensure each summary corresponds to one RSS feed entry.
  • On each execution, overwrite feed.xml with the newly generated content.

Notes

• Focus on keeping the RSS format clear and compatible with common readers.
• Protect against excessive size in queries.json via summary truncation or batching if necessary.
• Clean up any unused references, configs, or docs for email/resend functionality.

---

Acceptance Criteria:

• No email or Resend API usage remains
• queries.json structure is adopted
• feed.xml output is structured with one entry per summary
• Solution handles large/long summary content gracefully and efficiently</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Convert email workflow to RSS feed workflow #35

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[greenbrettmichael]

@copilot improve the formatting of the rss feed
the body looks like this

<![CDATA[### Title: THE WORST YEAR IN PROGRAMMING HISTORY Link: [Watch on YouTube](https://www.youtube.com/watch?v=rCEf7p3FRTM) Key Takeaways: - **[00:12](https://www.youtube.com/watch?v=rCEf7p3FRTM&t=12s)** - Hiring is dramatically tighter in 2025: prioritize networking, internships, and demonstrable projects (open source, apps, competitions) over résumé-bloated titles; be prepared to broaden geography, adjust compensation expectations, or create your own opportunities (side projects/VCable domains). - **[01:21](https://www.youtube.com/watch?v=rCEf7p3FRTM&t=81s)** - Many new AI tools aren't improving software quality or developer experience; evaluate AI editors and assistants against concrete KPIs (test coverage, PR cycle time, bug rate) before adopting, and keep investing in fundamentals like typings, tests, and CI. - **[01:47](https://www.youtube.com/watch?v=rCEf7p3FRTM&t=107s)** - 2025 set records for outages and fragile PRs — treat every change as risky: use feature flags, canary releases, stricter code review checklists, dependency pinning, and supply-chain monitoring to reduce blast radius. - **[02:16](https://www.youtube.com/watch?v=rCEf7p3FRTM&t=136s)** - Tooling and cloud costs are rising (author jokes about paid autocompletes and dark mode); run a quarterly tooling audit, consolidate subscriptions, enable cloud cost alerts/quotas, and prefer open-source/self-hosted alternatives where total cost of ownership is lower.]]>

The CDATA field should not be visible. The bullets should be seperated with new lines. The title should be removed from the body since its already the title for the feed entry. The duration links do not appear correctly.

[Copilot]

@copilot improve the formatting of the rss feed
the body looks like this

...

Fixed in commit 806305b. The RSS feed formatting has been significantly improved:

Changes made:

1. Removed title duplication - Updated OpenAI prompt to start directly with "Key Takeaways:" instead of including title/link (those are now only in RSS metadata)
2. Converted Markdown to HTML - Re-added markdown package to properly convert bullets and links to HTML
3. Fixed CDATA issues - Removed the visible CDATA wrapper by using direct HTML in the description field
4. Fixed link rendering - Timestamp links now render as proper clickable HTML links

Before:

&lt;<img src="..."> Key Takeaways: - **<a href="...">00:12</a>** - Text...

After (rendered in RSS reader):

Key Takeaways:

• 00:12 - Hiring is dramatically tighter in 2025: prioritize networking...
  [clickable timestamp link]

• 01:21 - Many new AI tools aren't improving software quality...
  [clickable timestamp link]

All formatting issues resolved - bullets are properly separated, links are clickable, and no duplicate title or visible CDATA. Tested with proper HTML rendering simulation.

[Copilot]

Pull request overview

This PR successfully converts the email-based workflow to an RSS feed workflow by removing all Resend email dependencies and implementing RSS feed generation functionality.

Key Changes:

• Removed email functionality and replaced with RSS feed generation that outputs to feed.xml
• Renamed email_list.json to queries.json and removed the email field requirement
• Updated OpenAI prompts to generate individual video summaries optimized for RSS readers

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
app.py	Core refactoring: removed email functions, added RSS feed generation with XML formatting, updated config loader from load_email_list_config to load_queries_config, modified main loop to generate per-video summaries
requirements.txt	Removed resend dependency, kept markdown for HTML conversion in RSS feed
queries.json.example	New example config file without email fields, demonstrates channel and search URL options
pyproject.toml	Removed resend from mypy ignore configuration
.env.example	Removed Resend API key configuration variables
.gitignore	Added queries.json and feed.xml to ignore list
README.md	Comprehensive documentation updates explaining RSS workflow, removed email references, added RSS feed usage examples
tests/test_rss_feed.py	New comprehensive test suite (7 tests) covering RSS generation, truncation, overwriting, and error handling
tests/test_queries_config.py	New test suite (21 tests) replacing email config tests, validates channel and search URL configurations
tests/test_openai_digest.py	Updated test assertions to match new OpenAI prompt format without title/link in summary body
tests/test_email.py	Removed - no longer needed
tests/test_email_list_config.py	Removed - replaced by test_queries_config.py

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[greenbrettmichael]

@copilot resolve feedback. for "RSS feed readers expect a full URL" please make this configurable in the env file

[Copilot]

@copilot resolve feedback. for "RSS feed readers expect a full URL" please make this configurable in the env file

All feedback resolved in commit c34a0eb.

Changes made:

1. Feed URL now configurable - Added FEED_BASE_URL environment variable in .env.example. The RSS feed's self-reference URL is now constructed from this base URL (defaults to https://example.com/ if not set)

2. Fixed Atom namespace - Properly registered using register_namespace()

3. Renamed function - generate_newsletter_digest → generate_video_summary for accuracy

4. Modern datetime API - Replaced deprecated utcnow() with datetime.now(timezone.utc)

5. Added deduplication - Videos are now deduplicated by video_id to prevent duplicate RSS entries when same video appears in both channel and search results

6. Fixed XML prettify - Now works on a copy and preserves HTML content without corruption

7. Updated tests - Fixed test to properly check HTML-converted content

8. Updated prompt wording - Changed from plural "videos" to singular "video" for clarity

All 61 tests passing, linters clean. The feed URL configuration is available in .env.example as requested.