Extend impact analysis to automatically add static link dependencies …

…to build list (IBM#535)

* Expanded impact analysis for static link scenarios that want to automatically rebuild all required submodules

Signed-off-by: Anjali Abraham <[email protected]>

samples/MortgageApplication/application-conf/README.md

@@ -19,6 +19,7 @@ mainBuildBranch | The main build branch of the main application repository.  Use
 gitRepositoryURL | git repository URL of the application repository to establish links to the changed files in the build result properties | false
 excludeFileList | Files to exclude when scanning or running full build.
 skipImpactCalculationList | Files for which the impact analysis should be skipped in impact build
+addSubmodulesToBuildList  | Flag to include Static Sub module files in pipeline builds
 jobCard | JOBCARD for JCL execs
 resolveSubsystems | boolean flag to configure the SearchPathDependencyResolver to evaluate if resolved dependencies impact the file flags isCICS, isSQL, isDLI, isMQ when creating the LogicalFile
 impactSearch | Impact finder resolution search configuration leveraging the SearchPathImpactFinder API. Sample configurations are inlcuded below, next to the previous rule definitions.

samples/MortgageApplication/application-conf/application.properties

@@ -44,6 +44,17 @@ excludeFileList=.*,**/.*,**/*.properties,**/*.xml,**/*.groovy,**/*.json,**/*.md,
 # sample: skipImpactCalculationList=**/epsmtout.cpy,**/centralCopybooks/*.cpy
 skipImpactCalculationList=
 
+# Flag to expand impact analysis to automatically include all mandatory submodules to build list in pipeline builds with a DBB Metadatastore connection based on the intermediate build list after impact analysis
+# This functionality queries and relies on the DBB Output collection of the application
+# 
+# The feature is specifically useful for applications using static calls to increase the isolation of the build environment
+# If the property is set to true, if a main program undergoes a change, all statically called sub programs are recompiled
+# Default: false
+# This property is setup as a DBB file property. This allows to enable the feature just for a subset of your build files
+# Sample to inspect the intermediate build list add submodules for Cobol build files
+# addSubmodulesToBuildList=true :: **/*.cbl
+addSubmodulesToBuildList=true :: **/*.cbl
+
 
 ###############################################################
 # Build Property management

samples/application-conf/README.md

@@ -22,6 +22,7 @@ mainBuildBranch | The main build branch of the main application repository.  Use
 gitRepositoryURL | git repository URL of the application repository to establish links to the changed files in the build result properties | false
 excludeFileList | Files to exclude when scanning or running full build. | false
 skipImpactCalculationList | Files for which the impact analysis should be skipped in impact build | false
+addSubmodulesToBuildList  | Flag to include Static Sub module files in pipeline builds | false
 jobCard | JOBCARD for JCL execs | false
 **Build Property management** | | 
 loadFileLevelProperties | Flag to enable the zAppBuild capability to load individual artifact properties files for a build file | true

samples/application-conf/application.properties

@@ -61,6 +61,18 @@ excludeFileList=.*,**/.*,**/*.xml,**/*.groovy,**/*.json,**/*.md,**/application-c
 # sample: skipImpactCalculationList=**/epsmtout.cpy,**/centralCopybooks/*.cpy
 skipImpactCalculationList=
 
+#
+# Flag to expand impact analysis to automatically include all mandatory submodules to build list in pipeline builds with a DBB Metadatastore connection based on the intermediate build list after impact analysis
+# This functionality queries and relies on the DBB Output collection of the application
+# 
+# The feature is specifically useful for applications using static calls to increase the isolation of the build environment
+# If the property is set to true, if a main program undergoes a change, all statically called sub programs are recompiled
+# Default: false
+# This property is setup as a DBB file property. This allows to enable the feature just for a subset of your build files
+# Sample to inspect the intermediate build list add submodules for Cobol build files
+# addSubmodulesToBuildList=true :: **/*.cbl
+addSubmodulesToBuildList=false :: **/*.cbl
+
 #
 # Job card, please use \n to indicate a line break and use \ to break the line in this property file
 # Example: jobCard=//RUNZUNIT JOB ,MSGCLASS=H,CLASS=A,NOTIFY=&SYSUID,REGION=0M

utilities/ImpactUtilities.groovy

@@ -122,7 +122,23 @@ def createImpactBuildList() {
 				if (props.verbose) println "** Impact analysis for $changedFile has been skipped due to configuration."
 			}
 		}
-
+
+	    Set<String> buildLinkSet = new HashSet<String>() 
+	    buildSet.each { buildFile ->
+	         String addSubmodulesToBuildList = props.getFileProperty('addSubmodulesToBuildList', buildFile)
+
+	         //include statically called sub programs when the main program changes
+
+		    if (addSubmodulesToBuildList != null && addSubmodulesToBuildList.toBoolean()) {
+			   // Call addLinkDependencies to append link dependencies to buildSet
+			   if (props.verbose) println "** Perform analysis to add statically called sub modules to build list for ${buildFile}."
+			   buildLinkSet = addLinkDependencies(buildFile)
+		    }
+	    }
+	        if (buildLinkSet !=null) {
+            buildSet.addAll(buildLinkSet)
+	    }
+
 		// Perform impact analysis for property changes
 		if (props.impactBuildOnBuildPropertyChanges && props.impactBuildOnBuildPropertyChanges.toBoolean()){
 			if (props.verbose) println "*** Perform impacted analysis for property changes."
@@ -169,7 +185,7 @@ def createImpactBuildList() {
 		}
 
 	}
-
+	
 	return [buildSet, changedFiles, deletedFiles, renamedFiles, changedBuildProperties]
 }
 
@@ -867,4 +883,49 @@ def sortFileList(list) {
  */
 def isMappedAsZUnitConfigFile(String file) {
 	return (dependencyScannerUtils.getScanner(file).getClass() == com.ibm.dbb.dependency.ZUnitConfigScanner)
+}
+
+/*
+ *  addLinkDependencies -
+ *  method to identify all statically called sub module programs when the main program changes
+ *
+ *  @return list of statically called sub modules
+ *
+ */
+def addLinkDependencies(buildFile) {
+    Set<String> buildLinkSet = new HashSet<String>()	
+    MetadataStore metadataStore = MetadataStoreFactory.getMetadataStore()
+    def logicalFile = buildUtils.relativizePath(buildFile)
+    def logicalFiles = metadataStore.getCollection(props.applicationOutputsCollectionName).getLogicalFile(logicalFile)
+		if (logicalFiles) {
+            // Check if any logical files are found
+            // List all link dependencies for every main program that changes from the output collection. 
+            // This will return only the program name and not the absolute path
+            logicalFiles.each { logicalFileRecord ->
+                def dependencies = logicalFileRecord.getLogicalDependencies()
+
+                dependencies.each { logicalDep ->
+				 if (logicalDep.getCategory() == "LINK") { 
+                    def linkDepName = logicalDep.getLname()
+                    def linkDepLogicalFile = metadataStore.getCollection(props.applicationCollectionName).getLogicalFiles(linkDepName)
+
+                    // Get the logical path for all the link dependencies returned
+                    linkDepLogicalFile.each { filePath ->
+                        // Link Dependency Files to be added
+                        def linkDepFile = filePath.getFile()
+
+                        if (linkDepFile != logicalFile) {
+						    if (ScriptMappings.getScriptName(linkDepFile)) {
+                              buildLinkSet.add(linkDepFile)
+							  if (props.verbose) println "** $linkDepFile has a link dependency to $logicalFile. Adding to build list"
+						    }
+                        }
+                     }
+                   }
+                }
+            }
+        }
+
+
+    return buildLinkSet
 }