TfsWorkItemTypeValidatorTool – use field mapping and field value mapping in validation

[satano]

This PR solves the last two items in improve work item type validation list.

Validation now respects field mapings (FieldToFieldMultiMap and FieldToFieldMap) and field value mappings (FieldValueMap) of FieldMappingTool. For FieldToFieldMap mappings only the ones with SourceToTarget mapping mode are used.

Configuration for validator TfsWorkItemTypeValidatorTool has changed a bit:

• SourceFieldMappings is removed, This existed only to define field mappings between source and target. Now real mappings from FieldMappingTool are respected.
• FixedTargetFields was removed. It is not needed and it was not really good because it worked with target fields and basically we validate source fields in target work item type.
• ExcludeSourceFields was added. Basically it replaces previous two properties and it is a list of source fields which will be excluded from validation.

Fields excluded from validation ExcludeSourceFields are not really excluded. They are still validated so user knows if there are some issues, but even when the issues are found, they are reported only as information instead of warning and the field itself is considered valid. This is especially useful, if the user still wants to migrate such a field, but it is not possible to define simple mapping for it. For example user will be notified, that the field X do not exist in target, but will configure it as excluded and define FieldMergeMap for this field.

There was a tip, that FieldSkipMap can also be used, but it is not possible. This mapping type is for target fields and we need to skip fields in source work item types.

Example log outputs

I tried to log as much thing as possible, so the user will know what to do.

Field exists and is valid

[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'System.Description' (Description).
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Target field 'System.Description' (Description) exists in 'Bug' and is valid.

Field is mapped and is valid

[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'Kros.VersionNumber' (Kros Version Number).
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]     Source field 'Kros.VersionNumber' is mapped to 'Custom.KrosVersionNumber'.
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Target field 'Custom.KrosVersionNumber' (KrosVersionNumber) exists in 'Bug' and is valid.

Field is missing in target

[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'Microsoft.VSTS.Scheduling.Effort' (Effort).
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]     Missing field 'Microsoft.VSTS.Scheduling.Effort' in 'Bug'.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source field reference name: Microsoft.VSTS.Scheduling.Effort
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source field name: Effort
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Field type: Double
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Allowed values:
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Allowed values type: Double

Allowed values for field are missing in target

[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]   Validating source field 'System.State' (State).
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]     Allowed values in target field do not match allowed values in source. Checking field value maps.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]     Missing values are: 'Approved', 'Committed', 'Done', 'Removed'
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]       Value 'Approved' is mapped to 'New', which exists in target.
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]       Value 'Done' is mapped to 'Closd', which does not exists in target. This is probably invalid 'FieldValueMap' configuration.
[16:09:37 DBG] [16.3.2-Local.4-5-gb4960cc1]       Value 'Removed' is mapped to 'Closed', which exists in target.
[16:09:37 WRN] [16.3.2-Local.4-5-gb4960cc1]     Source field 'System.State' and target field 'System.State' have different allowed values.
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Source allowed values: 'Approved', 'Committed', 'Done', 'New', 'Removed'
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Target allowed values: 'Active', 'Closed', 'New', 'Resolved'
[16:09:37 INF] [16.3.2-Local.4-5-gb4960cc1]       Missing values in target are: 'Committed', 'Done'

Field is excluded from validation

[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]   Validating source field 'Microsoft.VSTS.Scheduling.Effort' (Effort).
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]     Missing field 'Microsoft.VSTS.Scheduling.Effort' in 'Bug'.
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Source field reference name: Microsoft.VSTS.Scheduling.Effort
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Source field name: Effort
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Field type: Double
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Allowed values:
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]       Allowed values type: Double
[08:04:27 INF] [16.3.2-Local.4-7-g1ca7b77d]     Field 'Microsoft.VSTS.Scheduling.Effort' is excluded from validation, so it is considered valid.

Final log if there are any issues

[08:21:14 ERR] [16.3.2-Local.4-7-g1ca7b77d] Some work item types or their fields are not valid in the target system (see previous logs).
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If the work item type does not exist in target system, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Create it there.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure mapping to another work item type which exists in target using 'WorkItemTypeMappingTool' configuration.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Exclude it from validation. To configure which work item types are validated, use either 'IncludeWorkItemTypes' or 'ExcludeWorkItemTypes' of 'TfsWorkItemTypeValidatorTool' configuration (but not both at the same time).
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If field is missing in target, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Add missing field to the target work item type.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure field mapping using 'FieldToFieldMultiMap' in 'FieldMappingTool' configuration. This is simpler method for source to target field mapping, which allows to map multiple fields at once.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure field mapping using 'FieldToFieldMap' in 'FieldMappingTool' configuration. Mapping mode must be of type 'SourceToTarget'.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] If allowed values of the source and target fields do not match, you can:
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Add missing allowed values to the target field.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d]   - Configure value mapping. Add field maps of type 'FieldValueMap' to 'FieldMappingTool' configuration.
[08:21:14 INF] [16.3.2-Local.4-7-g1ca7b77d] To exclude field from validation, just configure it in 'ExcludeSourceFields' of 'TfsWorkItemTypeValidatorTool' configuration. If field is excluded from validation, all the issues are still logged, just the result of validation is 'valid'.

[satano]

I do not know why, but tests are failing because of authorization.

TF30063: You are not authorized to access https://dev.azure.com/nkdagility-preview/. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.

[MrHinsh]

I do not know why, but tests are failing because of authorization.

TF30063: You are not authorized to access https://dev.azure.com/nkdagility-preview/. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.

Lokley a token expiring. At the moment test data is live data

[MrHinsh]

Shoudl be good to go now