Update ai agent instructions

.cursor/rules/ydb-project.mdc

@@ -0,0 +1,17 @@
+---
+description: YDB project structure, build, tests, and PR conventions
+alwaysApply: true
+---
+
+# YDB Project Rules
+
+- **Docs:** <https://ydb.tech/docs/en/>. See also repo root [AGENTS.md](/AGENTS.md) and [.github/copilot-instructions.md](/.github/copilot-instructions.md).
+- **Structure:** `/ydb/core` — core modules, `/ydb/library` — YDB libs, `/ydb/apps`, `/ydb/services`, `/ydb/tools`, `/ydb/tests` (integration/functional), `/ydb/docs`. `/library` and `/util` — do not modify unless change is explicitly required. `/yql` — YQL; `/contrib` — third-party deps. Build: `ya.make` (and `ya.make.inc`); tool: `/ya` in repo root.
+- **Build:** From repo root: `./ya make path/to/project` (e.g. `./ya make ydb/core/base`, `./ya make ydb/apps/ydb`).
+- **Tests:** From repo root. Default: `./ya make -tt --build "relwithdebinfo"`. Use `-tt` (small+medium); `-ttt` only when large tests needed. **SIZE(LARGE) tests do not run in PR-check.** When verification needs a large/specific test, suggest the exact command (e.g. `./ya make -ttt --build "relwithdebinfo" -F '...' path/to/target`) or workflow [add_run_tests_table.yml](.github/workflows/add_run_tests_table.yml) (workflow_dispatch + PR number). Single test: `-F 'TestNameOrPattern' path/to/test/target`. Broader: `-F ydb/`. Sanitizers: `--sanitize="address"` (or memory/thread). Do not abort test runs. **Finding targets:** C++ in `ut_*` dirs; integration under `ydb/tests/`.
+- **C++:** Modern C++ only, no later than C++20.
+- **Docs (in repo):** Follow <https://ydb.tech/docs/en/contributor/documentation/style-guide> and structure.
+- **PRs:** Description must include changelog entry (short summary) + detailed description for reviewers. Exactly one category: New Feature, Experimental Feature, Improvement, Performance Improvement, User Interface, Bugfix, Backward-Incompatible Change, Documentation (changelog not required), Not for Changelog (changelog not required). Discuss non-trivial changes in an issue first (see [CONTRIBUTING.md](/CONTRIBUTING.md)).
+- **YDB context:** YDB is complex and must be fault-tolerant. For functional changes or root-cause analysis: consider nemesis, OLAP, stability/stress, and `ydb workload` (stock, kv, clickbench, tpcc, tpch, tpcds, topic, transfer, query); suggest running the relevant test suite or workload when appropriate. See [AGENTS.md](/AGENTS.md) section "YDB context: fault tolerance, testing, and workloads".
+- **Muted tests:** Tests in `.github/config/muted_ya.txt` are disabled (broken product/test/infra). If a muted test might be hiding a real bug (change touches same area), say so and suggest checking or running that test.
+- **.github changes:** When editing workflows, scripts (`.github/scripts/**`), config (e.g. `muted_ya.txt`, `stable_tests_branches.json`), or actions (e.g. `test_ya`): (1) Map who uses the changed file (which workflows/jobs). (2) **Critical:** Warn if PR-check, gate, postcommit, or mute pipeline (update_muted_ya, create_issues_for_muted_tests, `muted_ya.txt` path/format) are affected. (3) **Potential:** Warn on schedule/trigger changes, reusable workflow inputs, analytics/Telegram scripts, secrets/vars usage. (4) **Backward compatibility:** If a script or shared step can run from another branch (e.g. stable-*, prestable-*), breaking changes (new required args, path/format changes) cause failures or divergent behavior on that branch; keep compatibility or plan backport and warn. (5) Suggest verification (e.g. workflow_dispatch, docs in `.github/config/mute_rules.md`, `.github/scripts/telegram/README.md`). See [.github/copilot-instructions.md](/.github/copilot-instructions.md) section "Changes to .github".

.github/copilot-instructions.md

@@ -0,0 +1,66 @@
+# Repository instructions for GitHub Copilot
+
+Instructions for working with the YDB codebase. **Apply these guidelines when writing code, reviewing PRs, or changing CI/config.** For project structure, building, running tests, coding conventions, and PR guidelines use [AGENTS.md](/AGENTS.md) in the repository root. This file adds only:
+
+1. **How to find test targets** (below) — supplement to the tests section in AGENTS.md.
+2. **Impact of changes under `.github/`** — workflows, scripts, config, actions.
+
+Docs: <https://ydb.tech/docs/en/>.
+
+---
+
+## How to find test targets
+
+- **C++ unit tests:** Look for directories named **`ut_*`** (e.g. `ut_index_build_reboots`) under the module. The target path is the directory path: e.g. `ydb/core/tx/schemeshard/ut_index_build_reboots`.
+- **`ya.make`** in such directories usually contains **`UNITTEST_FOR(...)`** and **`SIZE(SMALL|MEDIUM|LARGE)`**; `-tt` runs small+medium, `-ttt` adds large. **Tests with `SIZE(LARGE)` do not run in PR-check.** When verification needs a large or specific test, suggest the **exact command** (e.g. `./ya make -ttt --build "relwithdebinfo" -F '...' path/to/test/target`) or the workflow [add_run_tests_table.yml](.github/workflows/add_run_tests_table.yml) (workflow_dispatch with PR number) to add a run-tests table to the PR.
+- **Python / integration tests:** Under `ydb/tests/` (e.g. `ydb/tests/functional`, `ydb/tests/library`). Run via `./ya make -tt ...` with the appropriate test path.
+
+**YDB fault tolerance and workloads:** YDB is a complex, fault-tolerant system. When making functional changes or analyzing test failures, consider **nemesis** (fault injection), **OLAP** tests (`ydb/tests/olap`), **stability/stress** (`ydb/tests/stability`, `ydb/tests/stress`), and **load via `ydb workload`** (stock, kv, clickbench, tpcc, tpch, tpcds, topic, transfer, query). Suggest running the relevant suite or `ydb workload <type>` when appropriate. Full guidance: [AGENTS.md](/AGENTS.md) section "YDB context: fault tolerance, testing, and workloads".
+
+---
+
+## Changes to `.github` (workflows, scripts, config, actions)
+
+When you change anything under **`.github/`** (workflows, scripts, config files, or composite actions), **assess impact** and **warn** about consequences. Use two priority levels: **critical / real** (high impact, likely breakage) and **potential** (possible side effects, follow-up checks needed).
+
+### 1. Map impact before editing
+
+- **Workflows** (`.github/workflows/*.yml`, `*.yaml`): which jobs/steps call which scripts or actions; which use `secrets.*`, `vars.*`, or config paths.
+- **Scripts** (`.github/scripts/**`): which workflows or actions invoke them; expected CLI args, env vars, and input files (e.g. `muted_ya.txt`).
+- **Config** (`.github/config/*`): who reads it — e.g. `muted_ya.txt`, `stable_tests_branches.json`, `stable_nightly_branches.json` are used by multiple workflows and by the **test_ya** action (PR-check, postcommit, run_tests, update_muted_ya, create_issues_for_muted_tests).
+- **Actions** (`.github/actions/**`): which workflows call them; inputs/outputs and any hardcoded paths (e.g. ` .github/config/muted_ya.txt` in **test_ya**).
+
+### 2. Critical / real problems (warn with high priority)
+
+- **PR-check / gate / postcommit:** Changes to `pr_check.yml`, `gate_postcommits.yml`, `postcommit_*.yml`, or to **test_ya** / **build_and_test_ya** can break merge checks and postcommit runs. Any change to how `muted_ya.txt` is read or passed (path, format, YAML conversion in `mute_utils.py`) affects all PR and postcommit test runs.
+- **Mute pipeline:** `update_muted_ya.yml` and `create_issues_for_muted_tests.yml` depend on: `create_new_muted_ya.py`, `get_muted_tests.py`, `flaky_tests_history.py`, `tests_monitor.py`, and on **`.github/config/muted_ya.txt`** path and format. Changing mute/unmute rules (in code or in `mute_rules.md`) can change which tests get muted and affect analytics and Telegram notifications.
+- **Paths and interfaces:** Renaming or moving **`.github/config/muted_ya.txt`**, or changing the format expected by `mute_utils.py` / `transform_build_results.py` / `create_new_muted_ya.py`, will break workflows and the **test_ya** action unless all call sites are updated.
+- **Muted tests and real bugs:** Tests in `muted_ya.txt` are disabled (broken product/test/infra). When reviewing code or investigating failures, if a **muted test could be hiding a real bug** (e.g. the change or fix touches the same area as that test), **warn explicitly** and suggest verifying (e.g. run the test locally or consider unmuting to confirm no regression).
+- **Secrets and vars:** Workflows use `CI_YDB_SERVICE_ACCOUNT_KEY_FILE_CREDENTIALS`, `YDBOT_TOKEN`, `GH_PERSONAL_ACCESS_TOKEN`, `TELEGRAM_YDBOT_TOKEN`, `TELEGRAM_MUTE_CHAT_ID`, `TEAM_TO_RESPONSIBLE_TG`, etc. Changing step logic that uses these can cause auth/notification failures.
+
+### 3. Potential problems (warn, suggest verification)
+
+- **Schedules and triggers:** Changing `on.schedule` or `on.workflow_dispatch` / `on.pull_request` can change how often jobs run or when they start; document the new behavior.
+- **Reusable workflows:** `run_tests.yml` is called by regression_* and run_and_debug_tests; changing its inputs (e.g. `branches_config_path`, `test_targets`) affects all callers.
+- **Analytics chain:** `collect_analytics.yml`, `collect_analytics_fast.yml`, `monitoring_queries.yml`, and `create_issues_for_muted_tests` run scripts in `.github/scripts/analytics/` and write to YDB; changing query paths, table names, or script order can break dashboards and downstream jobs.
+- **Telegram / alerts:** `alert_queued_jobs.py`, `parse_and_send_team_issues.py`, and RAM/alert steps depend on Telegram secrets and vars; logic or env changes can break notifications.
+- **Branch/config lists:** `update_muted_ya.yml` reads branches from `BRANCHES_CONFIG_PATH` (e.g. `stable_tests_branches.json`). Changing that path or the JSON shape can change which branches get mute-update PRs.
+
+### 4. Backward compatibility when a component is used from another branch
+
+If a **script**, **composite action**, or **reusable workflow** can be run from **another branch** (e.g. `stable-*`, `prestable-*`, or any workflow that runs on a ref other than `main`), the job checks out **that branch** — so it executes **that branch’s version** of the script/action, not the one on `main`.
+
+**What this can lead to:**
+
+- **Failures on release/other branches:** A breaking change on `main` (new required CLI argument, changed config path, changed output format) is not present on the old branch. When the workflow runs there, it still uses the old script/action. If only the **workflow YAML** was backported (e.g. new step or new input), the **code** on that branch is still old — the new args or paths are passed to the old script → **runtime errors** or wrong behavior.
+- **Divergent behavior:** `main` has the new logic; stable/release branches keep the old one. The “same” workflow name behaves differently depending on branch — **hard to reason about** and **support**.
+- **Silent breakage after partial backport:** Someone backports a workflow change but not the script; the run may fail in CI or produce incorrect results (e.g. wrong mute list, wrong artifact).
+
+**What to do:** When changing a script or shared step that may be invoked from another branch (check which workflows run on which refs, e.g. `stable-*` in `on.branches` or matrix over branches), either **(1)** keep **backward compatibility** (optional args, same paths/formats, additive changes only), or **(2)** explicitly plan **backport** of the script/action to all branches that use it, and **warn** in the PR that release branches must be updated.
+
+### 5. What to do in the response
+
+- **Before applying changes:** Briefly list which workflows, actions, or scripts are affected and whether any **critical** path is touched (PR-check, gate, postcommit, mute pipeline, muted_ya path/format).
+- **Warn explicitly:** e.g. “Critical: this changes how muted_ya.txt is read; PR-check and postcommit use it — ensure path/format stay compatible” or “Potential: schedule change; verify run frequency is still intended.”
+- **Suggest checks:** If you changed a script or workflow, suggest running the workflow manually (workflow_dispatch) or checking dependent workflows’ docs (e.g. `.github/scripts/telegram/README.md`, `.github/config/mute_rules.md`).
+

.gitignore

@@ -101,8 +101,9 @@ path.bash.inc
 # Editor config
 /.editorconfig
 
-# Cursor ignore
-.cursor/
+# Cursor ignore (keep .cursor/rules/ in repo so team shares rules)
+.cursor/*
+!.cursor/rules
 .cursorignore
 
 # Harness local cluster working directory

AGENTS.md

@@ -13,29 +13,83 @@ Every directory is a project (a library or a program).
 - `/ydb/core`: Core modules.
 - `/ydb/library`: YDB libraries.
 - `/ydb/docs`: YDB documentation.
-- `/library`: Common libraries; never change them.
-- `/util`: Common libraries; never change them.
+- `/library`, `/util`: Common libraries; never change them unless the change is explicitly required.
+- `/yql`: YQL (query layer); shared with YDB.
+- `/contrib`: Third-party and restricted dependencies.
 
 ## Building and Testing Requirements for AI Agents
 
 Building and testing are performed using the Ya utility (`ya-tool`), which is located in the root of the codebase: `/ya`.
 
-To build a project, an AI agent should run:
+To build a project, from the **repository root** run:
 
 ```bash
-cd path/to/project
-/ya make
+./ya make path/to/project
 ```
 
-To test a project, an AI agent should run:
+Alternatively, `cd path/to/project` then run `/ya make`.
+
+### Running tests with ya make (recommended for C++ unit tests, gtest, py3tests)
+
+Use the following commands from the **repository root** (where `/ya` lives).
+
+**Note:** Test runs can take a long time — `ya make` may need to build the whole project or large parts of it if required by dependencies. Do not abort the run; let it complete.
+
+**Test size:** Use `-tt` by default — runs only **small** and **medium** tests (faster). Use `-ttt` when you need **large** tests too; prefer `-ttt` only for a narrow list of targets or a single test, as it can run much longer.
+
+**Tests marked `SIZE(LARGE)` in ya.make do not run in PR-check** (PR-check uses `-tt` = small+medium only). When verification **requires** a large or specific test that would cover the change, **suggest** either: (1) **the exact command** to run locally, e.g. `./ya make -ttt --build "relwithdebinfo" -F 'TestNameOrPattern' path/to/test/target`, or a short list of such commands; or (2) running the workflow **Add run tests table to PR** (`.github/workflows/add_run_tests_table.yml`) via `workflow_dispatch` with the PR number — it adds a run-tests table to the PR so important tests can be run. Prefer attaching the concrete command when suggesting a specific test.
+
+**Default (no sanitizers):** `./ya make -tt --build "relwithdebinfo"`. Use when there is no suspicion of memory/address/thread bugs.
+
+- **Single test or narrow scope** — when changes touch one test binary or a small area, filter by test name with `-F`:
+
+  ```bash
+  ./ya make -tt --build "relwithdebinfo" -F 'TestNameOrPattern' path/to/test/target
+  ```
+
+  Example:
+
+  ```bash
+  ./ya make -tt --build "relwithdebinfo" -F 'FulltextIndexBuildTestReboots::BaseCase[PipeResetsBucket1]' ydb/core/tx/schemeshard/ut_index_build_reboots
+  ```
+
+- **Broader scope** — when changes affect more code:
+
+  ```bash
+  ./ya make -tt --build "relwithdebinfo" -F ydb/
+  ```
+
+**With sanitizers** — when there are suspicions of address, memory, or thread-safety issues, add `--sanitize="address"`, `--sanitize="memory"`, or `--sanitize="thread"` (use one at a time as needed). Example:
 
 ```bash
-cd path/to/project
-/ya make -A
+./ya make -tt --build "relwithdebinfo" --sanitize="address" -F 'TestNameOrPattern' path/to/test/target
 ```
 
+Choose the single-test form for one test or one binary; use `-F ydb/` when the change touches multiple modules or you need broader coverage.
+
 More information is available at <https://ydb.tech/docs/en/contributor/build-ya>.
 
+## YDB context: fault tolerance, testing, and workloads
+
+YDB is a **complex distributed system** with strict correctness and **fault-tolerance** requirements. When implementing functional changes, searching for root cause of failures, or debugging test failures, be **extra careful** and consider the following.
+
+### Test and load contexts
+
+- **Integration/functional tests** — `ydb/tests/`: `functional`, `stability`, `stress`, `olap`, `sql`, `library`, `compatibility`, `datashard`, `fq`, etc. Run via `./ya make -tt ... ydb/tests/functional` (or the relevant suite).
+- **Nemesis** — fault-injection and load tests under failure conditions; results feed into analytics (e.g. `nemesis/aggregated_mart`). When changes touch replication, node failure, or cluster behavior, consider suggesting nemesis-related runs or stability/stress suites.
+- **OLAP** — analytical workloads and benchmarks; tests under `ydb/tests/olap` and performance results (e.g. ClickBench, TPC-H). When touching query execution, storage, or analytics, suggest OLAP tests or workloads.
+- **Workload (ydb CLI)** — load tests via `ydb workload` subcommands: **stock**, **key-value** (kv), **ClickBench**, **TPC-C**, **TPC-H**, **TPC-DS**, **topic**, **transfer**, **query**. Implementations live in `ydb/library/workload/`. Docs: `ydb/docs/.../workload` (e.g. reference in `ydb-cli/commands/workload`). When changes affect transactions, storage, or throughput, **suggest running the relevant workload** (e.g. `ydb workload stock ...` or the suite that matches the changed area).
+
+### What agents should do
+
+- **Functional changes:** Before finishing, consider impact on fault tolerance and data consistency; **suggest** running the most relevant test suite (e.g. `ydb/tests/stability`, `ydb/tests/stress`, `ydb/tests/olap`) or workload (`ydb workload <type>`).
+- **Root cause / test failure:** When analyzing a failing test or bug, keep in mind nemesis, OLAP, and workload contexts; if the failure might be environment or load-related, suggest running stability/stress or a specific workload to reproduce.
+- **Documentation:** Workload command reference is under `ydb/docs/.../ydb-cli/commands/workload`; link to it when suggesting load tests.
+
+### Muted tests (`.github/config/muted_ya.txt`)
+
+Some tests are **disabled** (in muted_ya.txt) because of a broken product, broken test, or broken infra. When reviewing code, investigating a failure, or making changes: **if you suspect that a muted test could be hiding a real bug** (e.g. the change touches the same component or code path that the muted test covers), **say so explicitly** and suggest checking or running that test (e.g. locally or after unmuting) to confirm there is no regression.
+
 ## Coding Conventions for AI agents
 
 ### C++ Standards for AI Agents
@@ -60,3 +114,5 @@ More information is available at <https://ydb.tech/docs/en/contributor/build-ya>
   - Backward-Incompatible Change
   - Documentation (changelog entry is not required)
   - Not for Changelog (changelog entry is not required)
+
+For changes under **`.github/`** (workflows, scripts, config) and for **how to find test targets** (e.g. `ut_*` directories, integration tests under `ydb/tests/`), see [.github/copilot-instructions.md](.github/copilot-instructions.md).