Implement `ramenctl test` command #33

nirs · 2025-03-13T16:17:41Z

Implement the actual ramenctl test command using the ramen/e2e module.

Example run:

% ramenctl test run -c rdr.yaml -o rdr
⭐ Using report "rdr"
⭐ Using config "rdr.yaml"
✅ Environment setup                                                          
✅ Application "appset-deploy-rbd" deployed                                   
✅ Application "appset-deploy-rbd" protected                                  
✅ Application "appset-deploy-rbd" failed over                                
✅ Application "appset-deploy-rbd" relocated                                  
✅ Application "appset-deploy-rbd" unprotected                                
✅ Application "appset-deploy-rbd" undeployed                                 
✅ Test run completed successfully

Example report:
rdr.tar.gz

Example error - I stopped the hub cluster during protect:

% ramenctl test run -c rdr.yaml -o error
⭐ Using report "error"
⭐ Using config "rdr.yaml"
✅ Environment setup                                                          
✅ Application "appset-deploy-rbd" deployed                                   
❌ failed to protect application "appset-deploy-rbd": Get "https://192.168.105.33:6443/apis/ramendr.openshift.io/v1alpha1/namespaces/argocd/drplacementcontrols/appset-deploy-rbd": read tcp 192.168.105.1:51530->192.168.105.33:6443: read: operation timed out

Example error report:
error.tar.gz

We can see the error in the log:

% cat error/test-run.log
...
2025-03-18T21:28:12.110+0200	ERROR	appset-deploy-rbd	test/test.go:68	failed to protect application "appset-deploy-rbd": Get "https://192.168.105.33:6443/apis/ramendr.openshift.io/v1alpha1/namespaces/argocd/drplacementcontrols/appset-deploy-rbd": read tcp 192.168.105.1:51530->192.168.105.33:6443: read: operation timed out
github.com/ramendr/ramenctl/pkg/test.(*Test).Protect
	/Users/nir/src/ramenctl/pkg/test/test.go:68
github.com/ramendr/ramenctl/pkg/test.Run
	/Users/nir/src/ramenctl/pkg/test/run.go:25
github.com/ramendr/ramenctl/cmd/commands.init.func3
	/Users/nir/src/ramenctl/cmd/commands/test.go:21
github.com/spf13/cobra.(*Command).execute
	/Users/nir/go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1019
github.com/spf13/cobra.(*Command).ExecuteC
	/Users/nir/go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1148
github.com/spf13/cobra.(*Command).Execute
	/Users/nir/go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1071
main.main
	/Users/nir/src/ramenctl/cmd/main.go:20
runtime.main
	/opt/homebrew/Cellar/go/1.24.0/libexec/src/runtime/proc.go:283

Example clean:

% ramenctl test clean -c rdr.yaml -o rdr
⭐ Using report "rdr"
⭐ Using config "rdr.yaml"
✅ Application "appset-deploy-rbd" unprotected                                
✅ Application "appset-deploy-rbd" undeployed                                 
✅ Environment cleaned                                                        
✅ Test clean completed successfully

Example report with clean log:
rdr.tar.gz

Future work:

Validat the config before using it Validate config before using it #38
Collect more info in the report direcotory Test run summary - host info #34
Run all tests in parallel Run all tests in parallel #39
Log the cluster names (may need more work in e2e)
Delete application namespaces created in the test (should be fixed in e2e)

Part of #8
Fixes #37
Fixes #23

go.mod

pkg/test/run.go

parikshithb

Current changes looks good.

go.mod

pkg/test/test.go

pkg/test/run.go

parikshithb

Lgtm, added minor comments.

pkg/command/command.go

pkg/test/test.go

nirs · 2025-03-18T16:13:32Z

Should be ready now, testing it...

nirs · 2025-03-18T17:25:08Z

Example channel created by the test:

% kubectl get channel -A --context hub
NAMESPACE    NAME                                             TYPE     PATHNAME                                           AGE
e2e-gitops   https-github-com-ramendr-ocm-ramen-samples-git   github   https://github.com/RamenDR/ocm-ramen-samples.git   33s

We need a different channel namespace for ramenctl, need to modify the config before we use it.

nirs · 2025-03-18T17:40:28Z

Example channel created by the test:
% kubectl get channel -A --context hub
NAMESPACE    NAME                                             TYPE     PATHNAME                                           AGE
e2e-gitops   https-github-com-ramendr-ocm-ramen-samples-git   github   https://github.com/RamenDR/ocm-ramen-samples.git   33s
We need a different channel namespace for ramenctl, need to modify the config before we use it.

Change to "test-gitops".

nirs · 2025-03-18T18:06:11Z

Doing another round to make this more robust.

This is a quick test for using ramen/e2e in ramenctl. We read the config using ramen/e2e/config package and use the configured test and clusters for the fake test. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

For every ramenctl command we need: - A report directory (e.g. "rdr") - A logger logging to command log (e.g. "rdr/test-run.log") - A config loaded from the configuration file - An environment loaded from the config Add a command package providing command.New(), creating a command.Command, keeping everything for a single command. Notes: - We don't validate the config since validation depends on the command. For example the test command need to validate that the clusters in the Env match the configured DRPolicy. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

The test.Commands embeds a command.Command, adapting it for testing. - Add NamespacePrefix that will be used by the test.Context to ensure that application namespaces starts with "test-". - Modify config.Channel.Namesapce to "test-gitops" instead of the default "e2e-gitops". - Keep mapping of pvcspec name to pvcspec, needed to create test from the config. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

The test.Context implements ramen/e2e/test.Context. This is the way to bind all the details needed for a single test. We implement only the test.Context interface so we can use later e2e implementation, and keep the code simple as possible. The ramenctl specific parts will be implemented on top of the context. Unlike ramen/e2e/test.Context, we keep a Command instead of the env and config. This matches the actual model - all tests share the same config and env, which are read-only at the point we use them. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

The test.Test type implement the basic operations for a DR test: - Deploy - Undeploy - Protect - Unprotect - Failover - Relocate We will use it to run a complete DR test flow. The operations log a progress message, call the e2e code implementing the operation, and log a completion message. When a test operation fails the test is failed and cannot continue. We log the error to the test log, and return an error to the caller to signal the failure. The caller must abort the test and report the error to the console. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

Create a test.Test instance from the first test in config.Tests, and run a complete flow similar to ramen/e2e/dr_test flow. Running all tests in parallel will be implemented later. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

For now we add the channel, needed for subscription based tests. In the future we may do more work here as needed. Add setupEnvironment() and cleanEnvironemnt() functions. We don't clean the environment yet, it will be done in the test clean command, which is still a demo. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

This is the same as the run command, but we run only the unprotect and undeploy steps, and clean the environment from test artifacts. This does not remove all test namespaces yet. This should be fixed in ramen/e2e. The clean command logs to `report/test-clean.log`. If we run both `test-run` and `test-clean` with the same output directory, we will get: % tree rdr rdr ├── test-clean.log └── test-run.log Signed-off-by: Nir Soffer <nsoffer@redhat.com>

nirs requested a review from parikshithb March 13, 2025 16:17

nirs commented Mar 13, 2025

View reviewed changes

go.mod Outdated Show resolved Hide resolved

nirs commented Mar 13, 2025

View reviewed changes

pkg/test/run.go Show resolved Hide resolved

nirs mentioned this pull request Mar 13, 2025

e2e: Access config and clusters via test context RamenDR/ramen#1916

Merged

3 tasks

nirs force-pushed the ramen-e2e branch from c662076 to 35b3a4e Compare March 13, 2025 21:08

nirs commented Mar 13, 2025

View reviewed changes

pkg/test/run.go Outdated Show resolved Hide resolved

parikshithb reviewed Mar 15, 2025

View reviewed changes

nirs force-pushed the ramen-e2e branch from 35b3a4e to 2b42fa2 Compare March 16, 2025 11:40

nirs commented Mar 16, 2025

View reviewed changes

go.mod Outdated Show resolved Hide resolved

nirs force-pushed the ramen-e2e branch from 2b42fa2 to 8c00489 Compare March 16, 2025 16:38

nirs changed the title ~~Try to use ramen/e2e~~ Implement ramenctl test using ramen/e2e Mar 16, 2025

nirs requested review from raghavendra-talur and rakeshgm March 16, 2025 18:35

nirs force-pushed the ramen-e2e branch from 9a8262e to 84bd099 Compare March 16, 2025 19:11

nirs requested a review from parikshithb March 16, 2025 19:15

nirs changed the title ~~Implement ramenctl test using ramen/e2e~~ Implement ramenctl test run using ramen/e2e Mar 16, 2025

nirs commented Mar 16, 2025

View reviewed changes

pkg/test/test.go Show resolved Hide resolved

nirs commented Mar 16, 2025

View reviewed changes

pkg/test/run.go Show resolved Hide resolved

rakeshgm approved these changes Mar 18, 2025

View reviewed changes

parikshithb reviewed Mar 18, 2025

View reviewed changes

pkg/command/command.go Outdated Show resolved Hide resolved

pkg/test/test.go Outdated Show resolved Hide resolved

nirs force-pushed the ramen-e2e branch from 84bd099 to cffaf42 Compare March 18, 2025 15:02

nirs marked this pull request as ready for review March 18, 2025 16:13

nirs requested review from rakeshgm and parikshithb March 18, 2025 16:13

nirs force-pushed the ramen-e2e branch from cffaf42 to b0e57ab Compare March 18, 2025 17:40

nirs marked this pull request as draft March 18, 2025 18:05

nirs force-pushed the ramen-e2e branch from b0e57ab to cb0e958 Compare March 18, 2025 19:10

nirs marked this pull request as ready for review March 18, 2025 19:16

nirs added 7 commits March 18, 2025 21:59

Try to use ramen/e2e

85d7e87

This is a quick test for using ramen/e2e in ramenctl. We read the config using ramen/e2e/config package and use the configured test and clusters for the fake test. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

test: Run a single real test with the env

e23aa35

Create a test.Test instance from the first test in config.Tests, and run a complete flow similar to ramen/e2e/dr_test flow. Running all tests in parallel will be implemented later. Signed-off-by: Nir Soffer <nsoffer@redhat.com>

nirs force-pushed the ramen-e2e branch from cb0e958 to c3296ea Compare March 18, 2025 20:00

nirs changed the title ~~Implement ramenctl test run using ramen/e2e~~ Implement ramenctl test command Mar 18, 2025

nirs force-pushed the ramen-e2e branch from 0333b12 to 1b868d5 Compare March 18, 2025 20:28

rakeshgm approved these changes Mar 19, 2025

View reviewed changes

nirs merged commit 2c8a1a0 into RamenDR:main Mar 19, 2025

nirs deleted the ramen-e2e branch March 19, 2025 10:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement `ramenctl test` command #33

Implement `ramenctl test` command #33

nirs commented Mar 13, 2025 •

edited

Loading

parikshithb left a comment

parikshithb left a comment

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

Implement ramenctl test command #33

Implement ramenctl test command #33

Conversation

nirs commented Mar 13, 2025 • edited Loading

parikshithb left a comment

Choose a reason for hiding this comment

parikshithb left a comment

Choose a reason for hiding this comment

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

nirs commented Mar 18, 2025

Implement `ramenctl test` command #33

Implement `ramenctl test` command #33

nirs commented Mar 13, 2025 •

edited

Loading