MGMT-19258: Split KubeAPI and non KubeAPI subsystem tests

[paul-maidment]

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the exist

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests 'github.com/openshift/assisted-service/subsystem/kubeapi' for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests 'github.com/openshift/assisted-service/subsystem/kubeapi' for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: paul-maidment

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [paul-maidment]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. References to package private members that have been moved to the new package have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. References to package private members that have been moved to the new package have been updated to point to the correct place.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal>

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the exist>

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. Ref>

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the exist

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

• New Feature
• Enhancement
• Bug fix
• Tests
• Documentation
• CI/CD

What environments does this code impact?

• Automation (CI, tools, etc)
• Cloud
• Operator Managed Deployments
• None

How was this code tested?

• assisted-test-infra environment
• dev-scripts environment
• Reviewer's test appreciated
• Waiting for CI to do a full test run
• Manual (Elaborate on how it was tested)
• No tests needed

Checklist

• Title and description added to both, commit and PR.
• Relevant issues have been associated (see CONTRIBUTING guide)
• This change does not require a documentation update (docstring, docs, README, etc)
• Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

• Are the title and description (in both PR and commit) meaningful and clear?
• Is there a bug required (and linked) for this change?
• Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.61%. Comparing base (6c35570) to head (49cde10).
Report is 1 commits behind head on master.

❗ Current head 49cde10 differs from pull request most recent head 0ce3c54

Please upload reports for the commit 0ce3c54 to get more accurate results.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7154   +/-   ##
=======================================
  Coverage   67.60%   67.61%           
=======================================
  Files         296      296           
  Lines       40266    40266           
=======================================
+ Hits        27223    27226    +3     
+ Misses      10590    10588    -2     
+ Partials     2453     2452    -1     

see 2 files with indirect coverage changes