Refine Inspect queries and add CSVQ support for rule parsing #3595
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
encima
force-pushed
the
feat/inspect-rules
branch
from
80c020e to
e266458
Compare
| @@ -20,8 +20,7 @@ WITH table_opts AS ( | |||
| table_opts | |||
| ) | |||
| SELECT | |||
| vacuum_settings.nspname AS schema, | |||
| vacuum_settings.relname AS table, | |||
| vacuum_settings.nspname || '.' || vacuum_settings.relname AS name, | |||
There was a problem hiding this comment.
Suggested change
| vacuum_settings.nspname || '.' || vacuum_settings.relname AS name, | |
| format("%I.%I", vacuum_settings.nspname, vacuum_settings.relname) AS name, |
Since . is a legal character in schema, we should probably escape it to be unambiguous.
| @@ -1,5 +1,5 @@ | |||
| SELECT | |||
| schemaname || '.' || relname AS table, | |||
| schemaname || '.' || relname AS name, | |||
| @@ -1,6 +1,5 @@ | |||
| SELECT | |||
| n.nspname AS schema, | |||
| c.relname AS name, | |||
| n.nspname || '.' || c.relname AS name, | |||
Comment on lines
-110
to
-116
| inspectTotalIndexSizeCmd = &cobra.Command{ | ||
| Use: "total-index-size", | ||
| Short: "Show total size of all indexes", | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| return total_index_size.Run(cmd.Context(), flags.DbConfig, afero.NewOsFs()) | ||
| }, | ||
| } |
There was a problem hiding this comment.
better hide deprecated commands instead so it doesn't break existing users.
| @@ -38,12 +38,12 @@ WITH constants AS ( | |||
| JOIN pg_class c2 ON c2.oid = i.indexrelid | |||
| ) | |||
| SELECT | |||
| type, schemaname, object_name, bloat, pg_size_pretty(raw_waste) as waste | |||
| type, name, bloat, pg_size_pretty(raw_waste) as waste | |||
| FROM | |||
| (SELECT | |||
| 'table' as type, | |||
| schemaname, | |||
There was a problem hiding this comment.
this line seems redundant now
| out, _ = filepath.Abs(out) | ||
| // print the actual save location | ||
| if !filepath.IsAbs(dateDir) { | ||
| dateDir, _ = filepath.Abs(dateDir) |
There was a problem hiding this comment.
let's throw the error instead of failing silently
| array_to_string(rolconfig, ',', '*') as custom_config | ||
| FROM | ||
| pg_roles | ||
| ORDER BY 3 DESC |
There was a problem hiding this comment.
why order by connection_limit instead of role_name?
| COALESCE(rc.seq_scans, 0) AS seq_scans | ||
| FROM ( | ||
| SELECT | ||
| n.nspname || '.' || c.relname AS name, |
| @@ -30,7 +29,7 @@ func Run(ctx context.Context, config pgconn.Config, fsys afero.Fs, options ...fu | |||
| return err | |||
| } | |||
| defer conn.Close(context.Background()) | |||
| rows, err := conn.Query(ctx, TotalTableSizesQuery, reset.LikeEscapeSchema(utils.PgSchemas)) | |||
| rows, err := conn.Query(ctx, TotalTableSizesQuery, reset.LikeEscapeSchema(utils.InternalSchemas)) | |||
There was a problem hiding this comment.
are we sure that users are not interested in the size of managed schemas, like auth.users table for eg?
There was a problem hiding this comment.
maybe for auth, yep. For others, less so because they are managed and should not be touched (so they may cause issues but nothing the user can resolve)
Comment on lines
+17
to
+20
| query = "SELECT LISTAGG(index, ',') AS match FROM `unused_indexes.csv`" | ||
| pass = "✔" | ||
| fail = "There is at least one unused index" | ||
| name = "No unused indexes" |
There was a problem hiding this comment.
Suggested change
| query = "SELECT LISTAGG(index, ',') AS match FROM `unused_indexes.csv`" | |
| pass = "✔" | |
| fail = "There is at least one unused index" | |
| name = "No unused indexes" | |
| query = "SELECT LISTAGG(index, ',') AS match FROM `unused_indexes.csv`" | |
| pass = "✔" | |
| fail = "There is at least one unused index" | |
| name = "No unused indexes" |
could use more consistent formatting
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Feature
What is the current behavior?
Currently, inspect reports are quite modular and provide output where column titles can vary
What is the new behavior?
Refined queries to output in a similar format and reduced/combined work to minimise the number of output files produced (or commands needed)
Additional context
CSVQ has been added as a library to provide a table printout of some rules that are/are not met by the CSV files produced by
report