Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371

rohit-nayak-ps · 2024-12-11T16:53:44Z

Description

Follows up on the idea in #16517 by not considering tables already present in the globally routable list of tables.

See RFC for design and implementation details.

Related Issue(s)

#16516

Checklist

"Backport to:" labels have been added if this change should be back-ported to release branches
If this change is to be back-ported to previous releases, a justification is included in the PR description
Tests were added or are not required
Did the new or modified tests pass consistently locally and on CI?
Documentation was added or is not required

Deployment Notes

vitess-bot · 2024-12-11T16:53:48Z

codecov · 2024-12-11T17:48:45Z

Codecov Report

Attention: Patch coverage is 97.29730% with 1 line in your changes missing coverage. Please review.

Project coverage is 67.67%. Comparing base (9383943) to head (bc53f51).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
go/vt/vtgate/vtgate.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #17371   +/-   ##
=======================================
  Coverage   67.67%   67.67%           
=======================================
  Files        1583     1584    +1     
  Lines      254363   254391   +28     
=======================================
+ Hits       172140   172163   +23     
- Misses      82223    82228    +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

GuptaManan100

Just a nomenclature nit, otherwise looks good to me!

go/vt/vtgate/vindexes/vschema.go

harshit-gangal

The issue with this approach arises when there are multiple unsharded keyspaces. This makes adding the table to global routing non-deterministic, failing to fully achieve the intended purpose of this PR.

rohit-nayak-ps · 2024-12-13T12:20:05Z

The issue with this approach arises when there are multiple unsharded keyspaces. This makes adding the table to global routing non-deterministic, failing to fully achieve the intended purpose of this PR.

You mean if there are tables of the same name in multiple unsharded keyspaces (and which are not in the VSchemas).

We could collect all tables names which will be added like this (not in VSchema, but found by schema tracking) first and validate that there are no duplicates, marking ones with duplicates as ambiguous, while others get globally routed.

rohit-nayak-ps · 2024-12-13T14:05:39Z

Discussed with @harshit-gangal . His point is that:

We start with a sharded keyspace Sharded1 and one unsharded keyspace Unsharded1. Unsharded1 has no vschema defined yet.
After this PR, schema tracking finds a table ut1 in Unsharded1 that is not in the current VSchema. It then adds it to the global tables list. User starts depending on select * from ut1.
Later user adds another unsharded keyspace Unsharded2 which also has table named ut1.
Now we will mark ut1 as ambiguous (this logic is not yet part of this PR, but is the enhancement I referred to in Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371 (comment))
Suddenly the user's select * from ut1 query will start erroring out. User won't have visibility as to why this has happened since they have not done anything obvious in Vitess like modifying/adding a VSchema and will have downtime until they drop the other table, which could be difficult to diagnose/fix.

Options here are:

Close this PR and not do anything
Merge this PR (with enhanced logic to detect ambiguous tables outside of current VSchemas), which will solve the problem for X% of customers who currently have downtime because we don't do this

I guess the question is what is X. Is it high enough to merge this. Or even if X is high, do we want to do this from a principled viewpoint.

…they are not already present in a VSchema Signed-off-by: Rohit Nayak <[email protected]>

…e variable name had changed from positive to negative intent ... Signed-off-by: Rohit Nayak <[email protected]>

Signed-off-by: Rohit Nayak <[email protected]>

…ut vschemas to the global routing. Create more comprehensive unit test. Signed-off-by: Rohit Nayak <[email protected]>

…h no vschema as global. Enhance e2e test to handle both flag states with and without vschema Signed-off-by: Rohit Nayak <[email protected]>

Signed-off-by: Rohit Nayak <[email protected]>

rohit-nayak-ps · 2024-12-28T18:50:31Z

After discussions we decided to go ahead with the implementation as detailed in the RFC

… debugging Signed-off-by: Rohit Nayak <[email protected]>

rohit-nayak-ps · 2024-12-30T11:42:16Z

I have added a vtgate flag, --mark_unique_unsharded_tables_as_global (default true) to allow us the ability to disable the logic added in the PR in case some breaking changes have been introduced. I can remove it if reviewers feel it is not needed.

GuptaManan100

Rest LGTM!

GuptaManan100 · 2024-12-30T11:02:46Z

go/vt/vtgate/vtgate.go

@@ -200,6 +202,7 @@ func registerFlags(fs *pflag.FlagSet) {
 	fs.IntVar(&warmingReadsConcurrency, "warming-reads-concurrency", 500, "Number of concurrent warming reads allowed")
 	fs.DurationVar(&warmingReadsQueryTimeout, "warming-reads-query-timeout", 5*time.Second, "Timeout of warming read queries")

+	fs.BoolVar(&MarkUniqueUnshardedTablesAsGlobal, "mark_unique_unsharded_tables_as_global", MarkUniqueUnshardedTablesAsGlobal, "Mark unique unsharded tables as global tables")


As discussed, I don't think we need the flag

GuptaManan100 · 2024-12-30T11:08:24Z

go/vt/vtgate/vindexes/vschema.go

@@ -473,6 +473,46 @@ func buildGlobalTables(source *vschemapb.SrvVSchema, vschema *VSchema) {
 	}
 }

+// AddAdditionalGlobalTables adds unique tables from unsharded keyspaces to the global tables.
+// It is expected to be called from the schema tracking code.
+func AddAdditionalGlobalTables(source *vschemapb.SrvVSchema, vschema *VSchema) {


The code logic looks 💯 correct!

go/vt/vtgate/vindexes/vschema_routing_test.go

GuptaManan100 · 2024-12-30T13:24:47Z

go/vt/vtgate/vindexes/vschema_routing_test.go

+		Tables: map[string]*vschemapb.Table{
+			"table6":   {}, // unique
+			"scommon2": {}, // common with sharded2, but has RequireExplicitRouting
+			"scommon3": {}, // common with sharded2


Incorrect comment. scommon3 is not common with anything. It is unique.

go/test/endtoend/vreplication/r

go/test/endtoend/vreplication/r2

go/test/endtoend/vreplication/global_routing_test.go

Signed-off-by: Rohit Nayak <[email protected]>

github-actions bot added this to the v22.0.0 milestone Dec 11, 2024

rohit-nayak-ps requested a review from deepthi December 11, 2024 20:48

GuptaManan100 approved these changes Dec 12, 2024

View reviewed changes

go/vt/vtgate/vindexes/vschema.go Outdated Show resolved Hide resolved

rohit-nayak-ps marked this pull request as ready for review December 12, 2024 18:10

rohit-nayak-ps requested review from harshit-gangal, systay and frouioui as code owners December 12, 2024 18:10

harshit-gangal reviewed Dec 13, 2024

View reviewed changes

rohit-nayak-ps marked this pull request as draft December 16, 2024 16:23

rohit-nayak-ps added 7 commits December 28, 2024 19:48

Add missing tables to globally routed list in schema tracker only if …

0b9c0e1

…they are not already present in a VSchema Signed-off-by: Rohit Nayak <[email protected]>

Address review comment: needed to reverse the bools passed because th…

409046c

…e variable name had changed from positive to negative intent ... Signed-off-by: Rohit Nayak <[email protected]>

Fix incorrect logical check

39fa6b9

Signed-off-by: Rohit Nayak <[email protected]>

Add new logic for adding unique tables from unsharded keyspaces witho…

7dd53f4

…ut vschemas to the global routing. Create more comprehensive unit test. Signed-off-by: Rohit Nayak <[email protected]>

Add vtgate flag to turn on/off the logic to mark unsharded tables wit…

cb952b5

…h no vschema as global. Enhance e2e test to handle both flag states with and without vschema Signed-off-by: Rohit Nayak <[email protected]>

Fix flags test

c3aacf3

Signed-off-by: Rohit Nayak <[email protected]>

Refactor e2e test

917146c

Signed-off-by: Rohit Nayak <[email protected]>

rohit-nayak-ps added 5 commits December 28, 2024 19:49

Add global routing e2e to CI

8872dd9

Signed-off-by: Rohit Nayak <[email protected]>

Add Extended unit test case

af13c55

Signed-off-by: Rohit Nayak <[email protected]>

Add more cases

3537ea3

Signed-off-by: Rohit Nayak <[email protected]>

Add more cases

7d602a3

Signed-off-by: Rohit Nayak <[email protected]>

More tets cases

a3032db

Signed-off-by: Rohit Nayak <[email protected]>

rohit-nayak-ps force-pushed the rohit/global-routing-tables-not-in-vschema branch from f5d5a62 to a3032db Compare December 28, 2024 18:49

rohit-nayak-ps requested a review from GuptaManan100 December 28, 2024 18:50

rohit-nayak-ps changed the title ~~Experimental: Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema~~ Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema Dec 30, 2024

Remove incorrect comments from unit tests. Delete log lines added for…

e029da1

… debugging Signed-off-by: Rohit Nayak <[email protected]>

rohit-nayak-ps marked this pull request as ready for review December 30, 2024 11:42

rohit-nayak-ps requested a review from mattlord as a code owner December 30, 2024 11:42

rohit-nayak-ps requested a review from harshit-gangal December 30, 2024 11:42

GuptaManan100 approved these changes Dec 30, 2024

View reviewed changes

rohit-nayak-ps added 2 commits December 30, 2024 17:12

Address review comments

40f4302

Signed-off-by: Rohit Nayak <[email protected]>

Refactor e2e as suggested

bc53f51

Signed-off-by: Rohit Nayak <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371

Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371

rohit-nayak-ps commented Dec 11, 2024 •

edited

Loading

vitess-bot bot commented Dec 11, 2024

codecov bot commented Dec 11, 2024 •

edited

Loading

GuptaManan100 left a comment

harshit-gangal left a comment

rohit-nayak-ps commented Dec 13, 2024

rohit-nayak-ps commented Dec 13, 2024 •

edited

Loading

rohit-nayak-ps commented Dec 28, 2024

rohit-nayak-ps commented Dec 30, 2024

GuptaManan100 left a comment

GuptaManan100 Dec 30, 2024

GuptaManan100 Dec 30, 2024

GuptaManan100 Dec 30, 2024

Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371

Are you sure you want to change the base?

Add missing tables to globally routed list in schema tracker only if they are not already present in a VSchema #17371

Conversation

rohit-nayak-ps commented Dec 11, 2024 • edited Loading

Description

Related Issue(s)

Checklist

Deployment Notes

vitess-bot bot commented Dec 11, 2024

Review Checklist

General

Tests

Documentation

New flags

If a workflow is added or modified:

Backward compatibility

codecov bot commented Dec 11, 2024 • edited Loading

Codecov Report

GuptaManan100 left a comment

Choose a reason for hiding this comment

harshit-gangal left a comment

Choose a reason for hiding this comment

rohit-nayak-ps commented Dec 13, 2024

rohit-nayak-ps commented Dec 13, 2024 • edited Loading

rohit-nayak-ps commented Dec 28, 2024

rohit-nayak-ps commented Dec 30, 2024

GuptaManan100 left a comment

Choose a reason for hiding this comment

GuptaManan100 Dec 30, 2024

Choose a reason for hiding this comment

GuptaManan100 Dec 30, 2024

Choose a reason for hiding this comment

GuptaManan100 Dec 30, 2024

Choose a reason for hiding this comment

rohit-nayak-ps commented Dec 11, 2024 •

edited

Loading

codecov bot commented Dec 11, 2024 •

edited

Loading

rohit-nayak-ps commented Dec 13, 2024 •

edited

Loading