Skip to content

Commit c66d014

Browse files
aquie00tCopilot
aquie00t
and
Copilot
committed
feat: add 35 new bots (game dev, AI/LLM) and open-source scaffolding
New bots added: - Game dev: Unity, UnrealEngine, Godot, Phaser, Bevy - AI/LLM tools: Anthropic, ClaudeCode, Mistral, LlamaIndex, CrewAI, vLLM, Dify, LiteLLM, ComfyUI, DeepSeek, Flowise, xAI, HuggingFace, Gemini, Cohere, Groq, AutoGen, SemanticKernel, n8n, Stability, Haystack, MLflow, Cursor Core fixes: - BaseNewsWatcher: replace rss2json with direct fetch + fast-xml-parser - XMLParser: raise entity expansion limits to avoid parse errors - UnrealEngine RSS URL: /en-US/feed -> /en-US/rss - Netflix RSS: migrate to medium.com/feed/netflix-techblog (TLS fix) - setup-bots.js: skip already-registered bots via .setup-state.json Open-source scaffolding: - README.md: full rewrite with architecture overview and setup guide - CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md - .github/workflows/ci.yml: Prettier check + JS syntax validation - .github/dependabot.yml: weekly npm grouped updates - .github/PULL_REQUEST_TEMPLATE.md, ISSUE_TEMPLATE/{bug,feature}.md - .gitignore: add *_state.json, *.log, .DS_Store, setup-state.json - git rm --cached all *_state.json (runtime state, not source) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent a68c4f1 commit c66d014

153 files changed

Lines changed: 3012 additions & 454 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
name: Bug report
3+
about: A bot is failing, throwing errors, or posting incorrect content
4+
title: "[Bug] "
5+
labels: bug
6+
assignees: ""
7+
---
8+
9+
## Describe the bug
10+
11+
<!-- A clear description of what is going wrong. -->
12+
13+
## Affected bot(s)
14+
15+
<!-- e.g. TypeScript-News, Godot-Update -->
16+
17+
## Error message / log output
18+
19+
```
20+
paste log lines here
21+
```
22+
23+
## Expected behavior
24+
25+
<!-- What should happen instead? -->
26+
27+
## Environment
28+
29+
- Node.js version:
30+
- pnpm version:
31+
- OS:

‎.github/ISSUE_TEMPLATE/feature_request.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
name: Feature request / New bot
3+
about: Suggest a new bot or an improvement to an existing one
4+
title: "[Feature] "
5+
labels: enhancement
6+
assignees: ""
7+
---
8+
9+
## What would you like?
10+
11+
<!-- Describe the feature or new bot you'd like added. -->
12+
13+
## If this is a new bot request, please provide
14+
15+
- **Tool / project name:**
16+
- **GitHub repo** (for update bot, if applicable): `owner/repo`
17+
- **RSS feed URL** (for news bot, if applicable):
18+
- **Why should this be tracked on TDN?**
19+
20+
## Additional context
21+
22+
<!-- Any other information, links, or screenshots. -->

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
## Summary
2+
3+
<!-- What does this PR do? One sentence is enough. -->
4+
5+
## Type of change
6+
7+
- [ ] New bot(s) added
8+
- [ ] Bug fix
9+
- [ ] Refactor / code quality
10+
- [ ] Documentation
11+
- [ ] Other: <!-- describe -->
12+
13+
## Checklist
14+
15+
- [ ] `pnpm exec prettier --check .` passes
16+
- [ ] No `config.json`, `*_state.json`, or `scripts/.setup-state.json` committed
17+
- [ ] New bots are registered in `src/index.js` and `scripts/setup-bots.js`
18+
- [ ] RSS feed URLs were verified to return valid XML before adding a news bot

‎.github/copilot-instructions.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Copilot Instructions
2+
3+
## Commands
4+
5+
```bash
6+
pnpm start # Run all bots (node ./src/index.js)
7+
node scripts/setup-bots.js # One-time: register all bot accounts against the API
8+
```
9+
10+
Formatting is enforced automatically on commit via Husky + lint-staged (Prettier). To format manually:
11+
12+
```bash
13+
pnpm exec prettier --write .
14+
```
15+
16+
There is no test suite.
17+
18+
## Architecture
19+
20+
This is a Node.js ESM project (`"type": "module"`) that runs ~80 tech-topic bots. Each bot monitors either a GitHub releases feed, an RSS feed, or both, and publishes posts to a TDN social platform via a REST API.
21+
22+
### Layer overview
23+
24+
```
25+
src/
26+
index.js — Entry point. Imports every bot module and starts them all.
27+
logger.js — Shared timestamped, color-coded console logger.
28+
api/
29+
client.js — TdnClient: per-bot HTTP client with login, token refresh, createPost().
30+
core/
31+
BaseUpdateWatcher.js — EventEmitter that polls GitHub releases API; emits 'new_update'.
32+
BaseNewsWatcher.js — EventEmitter that polls RSS via rss2json; emits 'new_article'.
33+
requestScheduler.js — Centralized rate-limited fetch queue (separate lanes: 'github', 'rss').
34+
utils.js — buildNewsPost(), buildUpdatePost(), HTML/Markdown strippers.
35+
bots/
36+
{name}/
37+
update.js — Concrete update bot: wires BaseUpdateWatcher → TdnClient.createPost().
38+
news.js — Concrete news bot: wires BaseNewsWatcher → TdnClient.createPost().
39+
scripts/
40+
setup-bots.js — Registers, logs in, and sets profile/avatar/banner for each bot account.
41+
config.json — Bot credentials + API base URL (gitignored; must be created locally).
42+
```
43+
44+
### Data flow
45+
46+
1. `BaseUpdateWatcher` / `BaseNewsWatcher` polls its API on an interval with jitter.
47+
2. On a new result it emits `new_update` or `new_article`.
48+
3. The bot module handles the event, calls `buildUpdatePost()` / `buildNewsPost()`, and calls `client.createPost()`.
49+
4. `TdnClient` sends the post to `POST /posts` with Bearer auth, auto-refreshing the token on 401.
50+
5. State (last seen tag or article link) is persisted as `updates_state.json` / `news_state.json` **inside the bot's own directory** (`__dirname`-relative).
51+
52+
### Rate limiting
53+
54+
`requestScheduler.js` routes all outbound fetches through two lanes:
55+
56+
- `github`: max 2 concurrent, 600 ms minimum between requests
57+
- `rss`: max 1 concurrent, 5 s minimum between requests (rss2json free tier)
58+
59+
Both watchers implement exponential backoff (initial 30 s, max 1 h) on errors or 429/403 responses. Startup jitter spreads initial requests over 1–2 minutes to avoid thundering-herd bursts.
60+
61+
## Key Conventions
62+
63+
### Adding a new bot
64+
65+
1. Add the bot entry to `config.json` under `bots` with `username`, `email`, `password`.
66+
2. Create `src/bots/{name}/update.js` and/or `src/bots/{name}/news.js` using an existing bot as a template (e.g., `src/bots/typescript/`).
67+
- Export a single `start{Name}{Type}Bot()` function.
68+
- Pass `stateDir: __dirname` to the watcher so state files land next to the module.
69+
- Use `createTdnClient("{name}")` where `{name}` matches the key in `config.json`.
70+
3. Import and call the start function in `src/index.js`.
71+
4. Run `node scripts/setup-bots.js` to provision the new account on the platform.
72+
73+
### Post payload shape
74+
75+
```js
76+
{ content: string, type: "TECH_NEWS" | "SYSTEM_UPDATE", mediaUrls: string[] }
77+
```
78+
79+
Posts are hard-capped at **300 characters** (enforced in `utils.js`). News posts may include a thumbnail in `mediaUrls`; update posts always use an empty array.
80+
81+
### Environment
82+
83+
- `config.json` is gitignored. Bot credentials and `apiBaseUrl` live there.
84+
- Optional `GITHUB_TOKEN` env var (in `.env`): used by `BaseUpdateWatcher` to raise the GitHub API rate limit.
85+
86+
### Code style (Prettier)
87+
88+
4-space indent, double quotes, semicolons, trailing commas (`"trailingComma": "all"`), 80-char print width.

‎.github/dependabot.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: npm
4+
directory: "/"
5+
schedule:
6+
interval: weekly
7+
day: monday
8+
time: "09:00"
9+
timezone: "Europe/Istanbul"
10+
open-pull-requests-limit: 5
11+
groups:
12+
dev-dependencies:
13+
dependency-type: development
14+
production-dependencies:
15+
dependency-type: production
16+
ignore:
17+
# Avoid auto-upgrading major versions of core deps
18+
- dependency-name: "fast-xml-parser"
19+
update-types: ["version-update:semver-major"]

‎.github/workflows/ci.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main, dev]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
lint:
11+
name: Format check
12+
runs-on: ubuntu-latest
13+
14+
steps:
15+
- name: Checkout
16+
uses: actions/checkout@v4
17+
18+
- name: Setup pnpm
19+
uses: pnpm/action-setup@v4
20+
with:
21+
version: 10
22+
23+
- name: Setup Node.js
24+
uses: actions/setup-node@v4
25+
with:
26+
node-version: 20
27+
cache: pnpm
28+
29+
- name: Install dependencies
30+
run: pnpm install --frozen-lockfile
31+
32+
- name: Check formatting (Prettier)
33+
run: pnpm exec prettier --check .
34+
35+
validate:
36+
name: Module validation
37+
runs-on: ubuntu-latest
38+
39+
steps:
40+
- name: Checkout
41+
uses: actions/checkout@v4
42+
43+
- name: Setup pnpm
44+
uses: pnpm/action-setup@v4
45+
with:
46+
version: 10
47+
48+
- name: Setup Node.js
49+
uses: actions/setup-node@v4
50+
with:
51+
node-version: 20
52+
cache: pnpm
53+
54+
- name: Install dependencies
55+
run: pnpm install --frozen-lockfile
56+
57+
- name: Syntax check all JS files
58+
run: |
59+
find src scripts -name "*.js" | xargs -I{} node --check {}
60+
61+
- name: Validate core module imports
62+
run: |
63+
echo '{"apiBaseUrl":"http://localhost:8080/api/v1","bots":{}}' > config.json
64+
node --input-type=module --eval "
65+
await import('./src/core/BaseNewsWatcher.js');
66+
await import('./src/core/BaseUpdateWatcher.js');
67+
await import('./src/core/requestScheduler.js');
68+
await import('./src/core/utils.js');
69+
console.log('Core modules OK');
70+
"

‎.gitignore‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,23 @@
1+
# Dependencies
2+
node_modules
3+
4+
# Environment & secrets
5+
.env
16
config.json
7+
8+
# Runtime / generated
9+
bots.log
10+
*.log
11+
12+
# Bot runtime state files (last-seen article/release per bot)
13+
**/*_state.json
14+
15+
# Setup script state (environment-specific)
16+
scripts/.setup-state.json
17+
18+
# Assets (avatars, banners — large binary files)
219
assets
3-
node_modules
4-
.env
20+
21+
# OS
22+
.DS_Store
23+
Thumbs.db

‎CODE_OF_CONDUCT.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Contributor Covenant Code of Conduct
2+
3+
## Our Pledge
4+
5+
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
6+
7+
## Our Standards
8+
9+
Examples of behavior that contributes to a positive environment:
10+
11+
- Demonstrating empathy and kindness toward other people
12+
- Being respectful of differing opinions, viewpoints, and experiences
13+
- Giving and gracefully accepting constructive feedback
14+
- Accepting responsibility and apologizing to those affected by our mistakes
15+
- Focusing on what is best not just for us as individuals, but for the overall community
16+
17+
Examples of unacceptable behavior:
18+
19+
- The use of sexualized language or imagery, and sexual attention or advances of any kind
20+
- Trolling, insulting or derogatory comments, and personal or political attacks
21+
- Public or private harassment
22+
- Publishing others' private information without their explicit permission
23+
- Other conduct which could reasonably be considered inappropriate in a professional setting
24+
25+
## Enforcement
26+
27+
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the project maintainers. All complaints will be reviewed and investigated promptly and fairly.
28+
29+
## Attribution
30+
31+
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 2.1.

‎CONTRIBUTING.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Contributing to TDN System Bots
2+
3+
Thank you for your interest in contributing! This document describes how to add new bots, fix bugs, and submit changes.
4+
5+
## Prerequisites
6+
7+
- Node.js 18+
8+
- [pnpm](https://pnpm.io) (`npm install -g pnpm`)
9+
- A running TDN API instance (`config.json` pointing to it)
10+
11+
## Setup
12+
13+
```bash
14+
git clone https://github.com/your-org/tdn-system-bots
15+
cd tdn-system-bots
16+
pnpm install
17+
cp config.example.json config.json # fill in your credentials
18+
```
19+
20+
## Adding a new bot
21+
22+
1. **Create the bot directory** under `src/bots/{name}/`.
23+
2. **Add `update.js`** if the project has GitHub releases (copy from `src/bots/typescript/update.js`).
24+
3. **Add `news.js`** if the project has an RSS/Atom feed (copy from `src/bots/typescript/news.js`).
25+
4. **Register credentials** in `config.json` under `"bots"`.
26+
5. **Import and start** the bot in `src/index.js`.
27+
6. **Add metadata** to the `BOTS` array in `scripts/setup-bots.js`.
28+
7. Run `node scripts/setup-bots.js` to register the bot account.
29+
30+
See the [Architecture section of the README](./README.md#architecture) for full details.
31+
32+
## Running locally
33+
34+
```bash
35+
pnpm start # starts all bots
36+
```
37+
38+
Logs are written to stdout. RSS and GitHub requests are rate-limited automatically.
39+
40+
## Code style
41+
42+
This project uses [Prettier](https://prettier.io) for formatting. It runs automatically on commit via Husky + lint-staged. To format manually:
43+
44+
```bash
45+
pnpm exec prettier --write .
46+
```
47+
48+
Configuration: 4-space indent, double quotes, semicolons, trailing commas, 80-char print width.
49+
50+
## Pull request guidelines
51+
52+
- One bot (or one fix) per PR — keeps reviews focused.
53+
- Make sure `pnpm exec prettier --check .` passes before opening a PR.
54+
- Do **not** commit `config.json`, `*_state.json`, or `scripts/.setup-state.json` — they are gitignored.
55+
- Write clear commit messages (what + why).
56+
57+
## Reporting bugs
58+
59+
Please use the [Bug Report](.github/ISSUE_TEMPLATE/bug_report.md) issue template.

0 commit comments

Comments
 (0)