diff --git a/.gitattributes b/.gitattributes index 35e6af952..5704ecf1e 100644 --- a/.gitattributes +++ b/.gitattributes @@ -18,3 +18,6 @@ *.mac zos-working-tree-encoding=ibm-1047 git-encoding=utf-8 *.json zos-working-tree-encoding=utf-8 git-encoding=utf-8 *.yaml zos-working-tree-encoding=utf-8 git-encoding=utf-8 + +# TAZ recording file +*.rec binary diff --git a/README.md b/README.md index 096fbc7a0..7f3ef1386 100644 --- a/README.md +++ b/README.md @@ -27,10 +27,12 @@ The zAppBuild sample provides the following *language* build scripts by default: * DBDgen.groovy * PSBgen.groovy * MFS.groovy -* ZunitConfig.groovy +* TazUnitTest.groovy * CRB.groovy * Transfer.groovy (for transport non-buildable files like JCL or PROC into build libraries and register them as build output) * Easytrieve.groovy +* zCEE2.groovy +* zCEE3.groovy (for z/OS Connect OpenAPI 3 specification) All language scripts both compile and optionally link-edit programs. The language build scripts are intended to be useful out of the box but depending on the complexity of your applications' build requirements, may require modifications to meet your development team's needs. By following the examples used in the existing language build scripts of keeping all application specific references out of the build scripts and instead using configuration properties with strong default values, the zAppBuild sample can continue to be a generic build solution for all of your specific applications. diff --git a/build-conf/Easytrieve.properties b/build-conf/Easytrieve.properties index cbbba9371..ea29118b5 100644 --- a/build-conf/Easytrieve.properties +++ b/build-conf/Easytrieve.properties @@ -41,5 +41,5 @@ easytrieve_printTempOptions=cyl space(5,5) unit(vio) blksize(133) lrecl(133) rec # # List of output datasets to document deletions -easytrievel_outputDatasets=${easytrieve_loadPDS} +easytrieve_outputDatasets=${easytrieve_loadPDS} diff --git a/build-conf/GlobalOptions.properties b/build-conf/GlobalOptions.properties new file mode 100644 index 000000000..29032ead7 --- /dev/null +++ b/build-conf/GlobalOptions.properties @@ -0,0 +1,5 @@ +# Releng properties used by zAppBuild + +# Default allocation options for in-stream DD statements +global_instreamDataTempAllocation=tracks space(5,5) unit(vio) blksize(80) lrecl(80) recfm(f,b) new + diff --git a/build-conf/README.md b/build-conf/README.md index af677bbfb..95c1ec90d 100644 --- a/build-conf/README.md +++ b/build-conf/README.md @@ -30,7 +30,7 @@ SCSQLOAD | MQ Load Library. Example: CSQ.V9R1M0.SCSQLOAD SDSNLOAD | DB2 Load Library. Example: DB2.V9R1M0.SDSNLOAD SDSNEXIT | DB2 Exit Library. Example: DBC0CFG.SDSNEXIT SFELLOAD | Optional IDz Load Library. Example: FEL.V14R0M0.SFELLOAD -SBZUSAMP | Optional z/OS Dynamic Test Runner IDz zUnit / WAZI VTP library containing necessary copybooks. Example : FEL.V14R2.SBZUSAMP +SEQASAMP | Optional IBM Debug Tool library containing necessary TAZ copybooks. Example : DEBUG.V16.SEQASAMP EZETLOAD | Easytrieve Load Library. Example for IBM Migration Utility: SYS1.SFSYLOAD ### build.properties @@ -73,6 +73,13 @@ impactBuildOnBuildPropertyList | List of build property lists referencing which documentDeleteRecords | Option determine if the build framework should document deletions of outputs in DBB Build Report. Default: false. dbb.LinkEditScanner.excludeFilter | DBB configuration property used by the link edit scanner to exclude load module entries +### GlobalOptions.properties +General options that are reused by multiple language scripts + +Property | Description +--- | --- +global_instreamDataTempAllocation | Default allocation options for in-stream DD statements + ### dependencyReport.properties Properties used by the impact utilities to generate a report of external impacted files @@ -277,21 +284,21 @@ acbgen_outputDatasets | List of output datasets to document deletions ** Can be acbgen_pgm | MVS program name of the acbgen pgm acbgen_deployType | Deploy Type of build outputs -### ZunitConfig.properties -Build properties used by zAppBuild/language/ZunitConfig.groovy +### TazUnitTest.properties +Build properties used by zAppBuild/language/TazUnitTest.groovy Property | Description --- | --- -zunit_bzucfgPDS | Dataset to move BZUCFG files to from USS -zunit_bzureportPDS | Dataset where BZUCRPT files are stored -zunit_bzuplayPDS | Dataset to move zUnit Playback files to from USS -zunit_srcDatasets | Comma separated list of 'source' type data sets -zunit_srcOptions | BPXWDYN creation options for creating 'source' type data sets -zunit_loadDatasets | Comma separated list of 'load module' type data sets -zunit_loadOptions | BPXWDYN creation options for creating 'load module' type data sets -zunit_reportDatasets | Comma separated list of 'report' type data sets -zunit_reportOptions | BPXWDYN creation options for creating 'report' type data sets -zunit_dependenciesDatasetMapping | DBB property mapping to map dependencies to different target datasets +tazunittest_bzucfgPDS | Dataset to move BZUCFG files to from USS +tazunittest_bzureportPDS | Dataset where BZUCRPT files are stored +tazunittest_bzuplayPDS | Dataset to move zUnit Playback files to from USS +tazunittest_srcDatasets | Comma separated list of 'source' type data sets +tazunittest_srcOptions | BPXWDYN creation options for creating 'source' type data sets +tazunittest_loadDatasets | Comma separated list of 'load module' type data sets +tazunittest_loadOptions | BPXWDYN creation options for creating 'load module' type data sets +tazunittest_reportDatasets | Comma separated list of 'report' type data sets +tazunittest_reportOptions | BPXWDYN creation options for creating 'report' type data sets +tazunittest_dependenciesDatasetMapping | DBB property mapping to map dependencies to different target datasets ### zCEE3.properties Application properties used by zAppBuild/language/zCEE3.groovy @@ -300,7 +307,7 @@ Property | Description --- | --- zcee3_shellEnvironment | Shell environment used to run the gradle command zcee3_gradlePath | Path to gradle executable -zcee3_gradle_JAVA_OPTS | JAVA Options used with gradle +zcee3_gradle_debug | Flag to indicate if debug option should be added ### zCEE2.properties Application properties used by zAppBuild/language/zCEE2.groovy @@ -308,9 +315,8 @@ Application properties used by zAppBuild/language/zCEE2.groovy Property | Description --- | --- zcee2_zconbtPath | Absolute path to zconbt executable on z/OS UNIX System Services -zcee2_JAVA_HOME | Java installation used by the zconbt utility -zcee2_inputType | Mapping of input files with types of files -zcee2_ARA_PackageArtifacts | Flag to indicate if artifacts produced for the ARA processing should be packaged +zcee2_inputType | Mapping of input files with types of projects/files +zcee2_ARA_packageArtifacts | Flag to indicate if artifacts produced for the ARA processing should be packaged ### CRB.properties Application properties used by zAppBuild/language/CRB.groovy diff --git a/build-conf/REXX.properties b/build-conf/REXX.properties index 6277cfe7a..7f17b8709 100644 --- a/build-conf/REXX.properties +++ b/build-conf/REXX.properties @@ -4,8 +4,8 @@ # Comma separated list of required build properties for REXX.groovy rexx_requiredBuildProperties=rexx_srcPDS,rexx_objPDS,rexx_loadPDS,\ rexx_cexecPDS, rexx_compiler,rexx_linkEditor,rexx_tempOptions, \ - SFANLMD, \ - rexx_dependencySearch + SFANLMD, rexx_dependencySearch, \ + rexx_compileMaxRC, rexx_linkEditMaxRC, rexx_linkEdit # # rexx compiler name @@ -53,5 +53,5 @@ rexx_outputDatasets=${rexx_cexecPDS},${rexx_loadPDS} # rexx_dependenciesDatasetMapping = rexx_macroPDS :: **/rexxmacros/*.rexx # # default copies all dependencies into the dependency dataset definition which was previously passed to the utilities/BuildUitilities.copySourceFiles method -# rexx_dependenciesDatasetMapping = rexx_srcPDS :: **/* +# rexx_dependenciesDatasetMapping = rexx_srcPDS :: **/* rexx_dependenciesDatasetMapping = rexx_srcPDS :: **/* diff --git a/build-conf/TazUnitTest.properties b/build-conf/TazUnitTest.properties new file mode 100644 index 000000000..aeb540641 --- /dev/null +++ b/build-conf/TazUnitTest.properties @@ -0,0 +1,33 @@ +# Releng properties used by language/TazUnitTest.groovy + +tazunittest_requiredBuildProperties=tazunittest_srcDatasets,tazunittest_loadDatasets,tazunittest_reportDatasets,tazunittest_bzucfgPDS,\ + tazunittest_bzureportPDS,tazunittest_bzuplayPDS,tazunittest_srcOptions,tazunittest_loadOptions,tazunittest_reportOptions,\ + tazunittest_jobCard,tazunittest_maxPassRC,tazunittest_maxWarnRC,tazunittest_playbackFileExtension,\ + tazunittest_eqaplayParms,tazunittest_userDebugSessionTestParm, \ + tazunittest_dependencySearch + +tazunittest_bzucfgPDS=${hlq}.BZU.BZUCFG +tazunittest_bzureportPDS=${hlq}.BZU.BZURPT +tazunittest_bzuplayPDS=${hlq}.BZU.BZUPLAY + +# List the data sets that need to be created and their creation options +tazunittest_srcDatasets=${tazunittest_bzucfgPDS} +tazunittest_srcOptions=cyl space(1,1) lrecl(27998) dsorg(PO) recfm(V,B) dsntype(library) + +tazunittest_loadDatasets=${tazunittest_bzuplayPDS} +tazunittest_loadOptions=cyl space(1,1) dsorg(PO) lrecl(256) recfm(F,B) blksize(32512) dsntype(library) + +tazunittest_reportDatasets=${tazunittest_bzureportPDS} +tazunittest_reportOptions=tracks space(200,40) dsorg(PO) blksize(32760) lrecl(16383) recfm(v,b) dsntype(library) + +# tazunittest_dependenciesDatasetMapping - an optional dbb property mapping to map dependencies to different target datasets +# this property is used when dependencies are copied to the different build libraries, e.q dclgens going into to a dedicated library +# note, that a dependency file needs to match a single rule +# +# sample: +# tazunittest_dependenciesDatasetMapping = tazunittest_bzuplayPDS :: **/bzuplay/*.rec +# tazunittest_dependenciesDatasetMapping = tazunittest_bzuplayPDS :: **/bzuplay_vtp/*.rec +# +# default copies all dependencies into the dependency dataset definition which was previously passed to the utilities/BuildUitilities.copySourceFiles method +# tazunittest_dependenciesDatasetMapping = tazunittest_bzuplayPDS :: **/* +tazunittest_dependenciesDatasetMapping = tazunittest_bzuplayPDS :: **/* diff --git a/build-conf/ZunitConfig.properties b/build-conf/ZunitConfig.properties deleted file mode 100644 index c4c28fc9e..000000000 --- a/build-conf/ZunitConfig.properties +++ /dev/null @@ -1,33 +0,0 @@ -# Releng properties used by language/ZuntConfig.groovy - -zunit_requiredBuildProperties=zunit_srcDatasets,zunit_loadDatasets,zunit_reportDatasets,zunit_bzucfgPDS,\ - zunit_bzureportPDS,zunit_bzuplayPDS,zunit_srcOptions,zunit_loadOptions,zunit_reportOptions,\ - jobCard,zunit_maxPassRC,zunit_maxWarnRC,zunit_playbackFileExtension,\ - zunit_bzuplayParms,zunit_userDebugSessionTestParm, \ - zunit_dependencySearch - -zunit_bzucfgPDS=${hlq}.BZU.BZUCFG -zunit_bzureportPDS=${hlq}.BZU.BZURPT -zunit_bzuplayPDS=${hlq}.BZU.BZUPLAY - -# List the data sets that need to be created and their creation options -zunit_srcDatasets=${zunit_bzucfgPDS} -zunit_srcOptions=cyl space(1,1) lrecl(27998) dsorg(PO) recfm(V,B) dsntype(library) - -zunit_loadDatasets=${zunit_bzuplayPDS} -zunit_loadOptions=cyl space(1,1) dsorg(PO) lrecl(256) recfm(F,B) blksize(32512) dsntype(library) - -zunit_reportDatasets=${zunit_bzureportPDS} -zunit_reportOptions=tracks space(200,40) dsorg(PO) blksize(32760) lrecl(16383) recfm(v,b) dsntype(library) - -# zunit_dependenciesDatasetMapping - an optional dbb property mapping to map dependencies to different target datasets -# this property is used when dependencies are copied to the different build libraries, e.q dclgens going into to a dedicated library -# note, that a dependency file needs to match a single rule -# -# sample: -# zunit_dependenciesDatasetMapping = zunit_bzuplayPDS :: **/bzuplay/*.bzuplay -# zunit_dependenciesDatasetMapping = zunit_bzuplayPDS :: **/bzuplay_vtp/*.bzuplay -# -# default copies all dependencies into the dependency dataset definition which was previously passed to the utilities/BuildUitilities.copySourceFiles method -# zunit_dependenciesDatasetMapping = zunit_bzuplayPDS :: **/* -zunit_dependenciesDatasetMapping = zunit_bzuplayPDS :: **/* diff --git a/build-conf/build.properties b/build-conf/build.properties index 97c5976a2..c1c2adc62 100644 --- a/build-conf/build.properties +++ b/build-conf/build.properties @@ -16,9 +16,10 @@ # These properties files expect to contain centrally managed defaults # such as system datasets, language script specific settings # -buildPropFiles=${systemDatasets},dependencyReport.properties,Assembler.properties,BMS.properties,\ -MFS.properties,PSBgen.properties,DBDgen.properties,ACBgen.properties,Cobol.properties,\ -LinkEdit.properties,PLI.properties,REXX.properties,ZunitConfig.properties,Transfer.properties,\ +buildPropFiles=${systemDatasets},GlobalOptions.properties,dependencyReport.properties,\ +Assembler.properties,BMS.properties,Cobol.properties,Transfer.properties,\ +MFS.properties,PSBgen.properties,DBDgen.properties,ACBgen.properties,\ +LinkEdit.properties,PLI.properties,REXX.properties,TazUnitTest.properties,\ CRB.properties,zCEE3.properties,zCEE2.properties,Easytrieve.properties # @@ -50,7 +51,7 @@ systemDatasets=datasets.properties # default-application-conf/PLI.properties,\ # default-application-conf/Transfer.properties,\ # default-application-conf/LinkEdit.properties,\ -# default-application-conf/ZunitConfig.properties +# default-application-conf/TazUnitTest.properties # applicationDefaultPropFiles=defaultzAppBuildConf.properties @@ -110,7 +111,7 @@ createBuildOutputSubfolder=true # # Minimum required DBB ToolkitVersion to run this version of zAppBuild # Build initialization process validates the DBB Toolkit Version in use and matches that against this setting -requiredDBBToolkitVersion=2.0.0 +requiredDBBToolkitVersion=2.0.2 # # Comma separated list of required build properties for zAppBuild/build.groovy @@ -142,6 +143,7 @@ metadataStoreType=file # The dbb.gateway.type property determines which gateway type is used for the entire build process # Possible values are 'legacy' and 'interactive'. Default if not indicated is 'legacy' +# Please note that the ISPF client gateway is not used by language scripts provided by zAppBuild, but can be used in custom implementations. dbb.gateway.type=legacy # Procedure Name - specified with the procname parameter diff --git a/build-conf/datasets.properties b/build-conf/datasets.properties index abd099157..b6fa6f124 100644 --- a/build-conf/datasets.properties +++ b/build-conf/datasets.properties @@ -69,8 +69,8 @@ SEQAMOD= # Optional IDz Load Library. Example: FEL.V14R0M0.SFELLOAD SFELLOAD= -# Optional IDZ zUnit / WAZI VTP library containing necessary copybooks. Example : FEL.V14R2.SBZUSAMP -SBZUSAMP= +# Optional IBM Debug Tool library containing necessary TAZ copybooks. Example : DEBUG.V16.SEQASAMP +SEQASAMP= # REXX Compiler Data Sets. Example: REXX.V1R4.SFANLMD SFANLMD= diff --git a/build-conf/zCEE2.properties b/build-conf/zCEE2.properties index 5de489098..23c3b3ce8 100644 --- a/build-conf/zCEE2.properties +++ b/build-conf/zCEE2.properties @@ -1,20 +1,20 @@ # Releng properties used by language/zCEE2.groovy +# +# Important considerations when using the z/OS Connect Build Toolkit (zconbt) +# Starting with z/OS Connect v3.0.81, zconbt supports the use of Java 11 +# Which Java version is used with zconbt is controlled through the JAVA_HOME environement variable +# It is recommended to set this environement variable in the .profile file of the user that runs the build + # # Comma separated list of required build properties for zCEE3.groovy zcee2_requiredBuildProperties=zcee2_zconbtPath # # Absolute path to zconbt executable on z/OS UNIX System Services -# for instance: /var/zosconnect/v359/bin/zconbt.zos +# for instance: /var/zosconnect/v384/bin/zconbt.zos zcee2_zconbtPath= -# -# Java installation used by the zconbt utility -# for instance: /usr/lpp/java/J8.0_64 -# if not set, the value of the JAVA_HOME environment variable will be used -zcee2_JAVA_HOME= - # # Mapping of input files with types of files # PROJECT can be used for SAR and AAR projects @@ -23,7 +23,7 @@ zcee2_JAVA_HOME= zcee2_inputType=PROJECT # -# Flag to indicate if artifacts produced for the ARA processing should be packaged (generated copybooks, API information copybook and logs) +# Flag to indicate if artifacts produced by the ARA processing should be packaged (generated copybooks, API information copybook and logs) # When set to true, the artifacts are located based on the dataStructuresLocation, apiInfoFileLocation and logFileDirectory properties of the ARA properties files -# When not defined, the default value is false and artifacts are packaged -zcee2_ARA_PackageArtifacts=true +# When not defined, the default value is false and generated artifacts are not packaged +zcee2_ARA_packageArtifacts=true diff --git a/build-conf/zCEE3.properties b/build-conf/zCEE3.properties index 3463bc067..fc2c1a39c 100644 --- a/build-conf/zCEE3.properties +++ b/build-conf/zCEE3.properties @@ -1,8 +1,13 @@ # Releng properties used by language/zCEE3.groovy +# +# Important considerations when using Gradle with Java 11 on z/OS Unix System Services +# IPv6 needs to be disabled by setting the IBM_JAVA_OPTIONS environment variable as follows: +# IBM_JAVA_OPTS="-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -Dfile.encoding=UTF-8" +# It is recommended to set this globally in the .profile file of the user that runs the build # # Comma separated list of required build properties for zCEE3.groovy -zcee3_requiredBuildProperties=zcee3_shellEnvironment, zcee3_gradlePath, zcee3_gradle_JAVA_OPTS +zcee3_requiredBuildProperties=zcee3_shellEnvironment, zcee3_gradlePath # # Shell environment used to run the gradle command @@ -11,10 +16,12 @@ zcee3_shellEnvironment=bash # # Absolute path to gradle executable on z/OS UNIX System Services # for instance: /var/gradle/gradle-7.6/bin/gradle -zcee3_gradlePath= +zcee3_gradlePath=/var/gradle/gradle-8.10/bin/gradle # -# JAVA Options used with gradle -# for instance: zCEE_gradle_JAVA_OPTS=-Xmx512m -XX:MaxMetaspaceSize=256m -Dfile.encoding=UTF-8 -zcee3_gradle_JAVA_OPTS=-Dfile.encoding=UTF-8 +# Flag to indicate if --debug option should be added to the gradle command +# Setting the debug option can be helpful to diagnose issues with the gradle daemon +# Default is false, can be set to true +zcee3_gradle_debug=true + diff --git a/docs/BUILD.md b/docs/BUILD.md index af75f0585..e5f333d0d 100644 --- a/docs/BUILD.md +++ b/docs/BUILD.md @@ -95,7 +95,7 @@ $DBB_HOME/bin/groovyz build.groovy \ --debug \ --impactBuild ``` -**Use Code Coverage Headless Collector in zUnit Tests and specify parameters through command-line options (which override properties defined in ZunitConfig.properties)** +**Use Code Coverage Headless Collector in zUnit Tests and specify parameters through command-line options (which override properties defined in TazUnitTest.properties)** ``` $DBB_HOME/bin/groovyz build.groovy \ --workspace /u/build/repos \ @@ -300,7 +300,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -379,7 +379,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -483,7 +483,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -723,7 +723,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -882,7 +882,7 @@ Please see the output provided in verbose mode when setting up the collections a ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -1014,7 +1014,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/PLI.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/REXX.properties -** Loading property file /var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -1174,7 +1174,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/PLI.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/REXX.properties -** Loading property file /var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -1305,7 +1305,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/PLI.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/REXX.properties -** Loading property file /var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** Loading property file /var/dbb/dbb-zappbuild/build-conf/Transfer.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties @@ -1652,7 +1652,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -1796,7 +1796,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -1975,7 +1975,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -2135,7 +2135,7 @@ groovyz dbb-zappbuild/build.groovy \ ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/Cobol.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/PLI.properties -** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/dbb/dbb-zappbuild/build-conf/TazUnitTest.properties ** appConf = /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/file.properties ** Loading property file /var/dbb/dbb-zappbuild/samples/MortgageApplication/application-conf/BMS.properties @@ -2164,7 +2164,7 @@ groovyz dbb-zappbuild/build.groovy \ ** The dataset DEBUG.V14R1.SEQAMOD referenced for property SEQAMOD was found. ** The dataset DBC0CFG.SDSNEXIT referenced for property SDSNEXIT was found. ** The dataset IMS.V15R1.SDFSRESL referenced for property SDFSRESL was found. -** The dataset RATCFG.ZUNIT.SBZUSAMP referenced for property SBZUSAMP was found. +** The dataset DEBUG.V16.SEQASAMP referenced for property SEQASAMP was found. ** The dataset CEE.SCEELKED referenced for property SCEELKED was found. ..... // lists of all build properties ... diff --git a/docs/HOWTOS.md b/docs/HOWTOS.md index 6bec65fcf..79e6f7eef 100644 --- a/docs/HOWTOS.md +++ b/docs/HOWTOS.md @@ -118,7 +118,7 @@ default-application-conf/BMS.properties,\ default-application-conf/PLI.properties,\ default-application-conf/Transfer.properties,\ default-application-conf/LinkEdit.properties,\ -default-application-conf/ZunitConfig.properties +default-application-conf/TazUnitTest.properties .. ``` This allows to reduce the necessary definitions within the `application-conf` directory of the application repository. This `application-conf` directory now only contains the `application.properties` and `file.properties` files to define the application-specific settings and exceptions, such as [file properties](../docs/FilePropertyManagement.md#dbb-file-properties) for particular build files: diff --git a/languages/Assembler.groovy b/languages/Assembler.groovy index 0f712532e..ce63fdbf1 100644 --- a/languages/Assembler.groovy +++ b/languages/Assembler.groovy @@ -314,7 +314,7 @@ def createAssemblerCommand(String buildFile, LogicalFile logicalFile, String mem assembler.dd(new DDStatement().name("SYSUT1").options(props.assembler_tempOptions)) // define object dataset allocation - assembler.dd(new DDStatement().name("SYSLIN").dsn("${props.assembler_objPDS}($member)").options('shr').output(true)) + assembler.dd(new DDStatement().name("SYSLIN").dsn("${props.assembler_objPDS}($member)").options('shr').output(true).deployType("OBJ")) // create a SYSLIB concatenation with optional MACLIB and MODGEN assembler.dd(new DDStatement().name("SYSLIB").dsn(props.assembler_macroPDS).options("shr")) diff --git a/languages/CRB.groovy b/languages/CRB.groovy index a33a8e6be..ab24051d8 100644 --- a/languages/CRB.groovy +++ b/languages/CRB.groovy @@ -59,30 +59,24 @@ buildList.each { buildFile -> if (!outputDir.exists()) outputDir.mkdirs() - // Build the shell command to execute - def applicationParm = "" - if (applicationConstraintsFile) { - fileExists(applicationConstraintsFile) - applicationParm = "--application $applicationConstraintsFile" + String optionsString = "build --model ${resourceModelFile} --resources ${props.workspace}/${buildFile}" + optionsString += " --output ${props.buildOutDir}/${outputFile}" + if (applicationConstraintsFile && fileExists(applicationConstraintsFile)) { + optionsString += " --application ${applicationConstraintsFile}" } - def commandString = zrbPath + " build --model $resourceModelFile $applicationParm --resources ${props.workspace}/${buildFile} --output ${props.buildOutDir}/$outputFile" + ArrayList optionsList = new ArrayList(Arrays.asList(optionsString.split(" "))) if (props.verbose) - println("*** Executing zrb command: $commandString") - - // Execute the command and direct console output and error streams to buffer - def process = commandString.execute() - StringBuffer zrbOut = new StringBuffer() - StringBuffer zrbErr = new StringBuffer() - process.waitForProcessOutput(zrbOut, zrbErr) - def returnCode = process.exitValue() - - // write outputs to log file - String enc = props.logEncoding ?: 'IBM-1047' - logFile.withWriter(enc) { writer -> - writer.append(zrbOut) - writer.append(zrbErr) - } + println("*** Executing command '$zrbPath' with options '${optionsList}'") + String encoding = props.logEncoding ?: 'IBM-1047' + + UnixExec zrbExecution = new UnixExec().command(zrbPath) + zrbExecution.setOptions(optionsList) + zrbExecution.output(logFile.getAbsolutePath()).mergeErrors(true); + zrbExecution.setFile(buildFile) + zrbExecution.setOutputEncoding(encoding) + zrbExecution.addOutput(props.buildOutDir, outputFile, "CSD") + int returnCode = zrbExecution.execute() // evaluate return code if (returnCode > maxRC) { @@ -93,18 +87,6 @@ buildList.each { buildFile -> } else { if (props.verbose) println("*** zrb return code: $returnCode") println("*** Output file is ${props.buildOutDir}/$outputFile.") - - // Create a new record of type AnyTypeRecord - AnyTypeRecord CRBRecord = new AnyTypeRecord("USS_RECORD") - CRBRecord.setAttribute("file", buildFile) - CRBRecord.setAttribute("label", "CSD file created with CICS Resource Builder") - CRBRecord.setAttribute("outputs", "[${props.buildOutDir}, $outputFile, CSD]") - CRBRecord.setAttribute("command", commandString); - - // Add new record to build report - if (props.verbose) - println("*** Adding USS_RECORD for $buildFile") - BuildReportFactory.getBuildReport().addRecord(CRBRecord) } } } diff --git a/languages/Cobol.groovy b/languages/Cobol.groovy index 345b1526a..786553f1a 100644 --- a/languages/Cobol.groovy +++ b/languages/Cobol.groovy @@ -13,7 +13,7 @@ import com.ibm.dbb.build.report.records.* @Field def buildUtils= loadScript(new File("${props.zAppBuildDir}/utilities/BuildUtilities.groovy")) @Field def impactUtils= loadScript(new File("${props.zAppBuildDir}/utilities/ImpactUtilities.groovy")) @Field def bindUtils= loadScript(new File("${props.zAppBuildDir}/utilities/BindUtilities.groovy")) - + println("** Building ${argMap.buildList.size()} ${argMap.buildList.size() == 1 ? 'file' : 'files'} mapped to ${this.class.getName()}.groovy script") // verify required build properties @@ -37,14 +37,14 @@ sortedList.each { buildFile -> println "*** (${currentBuildFileNumber++}/${sortedList.size()}) Building file $buildFile" // Check if this a testcase - isZUnitTestCase = buildUtils.isGeneratedzUnitTestCaseProgram(buildFile) + isTazUnitTestCase = buildUtils.isGeneratedTazTestCaseProgram(buildFile) // configure dependency resolution and create logical file String dependencySearch = props.getFileProperty('cobol_dependencySearch', buildFile) SearchPathDependencyResolver dependencyResolver = new SearchPathDependencyResolver(dependencySearch) // copy build file and dependency files to data sets - if(isZUnitTestCase){ + if(isTazUnitTestCase){ buildUtils.copySourceFiles(buildFile, props.cobol_testcase_srcPDS, null, null, null) }else{ buildUtils.copySourceFiles(buildFile, props.cobol_srcPDS, 'cobol_dependenciesDatasetMapping', props.cobol_dependenciesAlternativeLibraryNameMapping, dependencyResolver) @@ -106,7 +106,7 @@ sortedList.each { buildFile -> buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}.log":logFile]) } else { - if(!props.userBuild && !isZUnitTestCase){ + if(!props.userBuild && !isTazUnitTestCase){ // only scan the load module if load module scanning turned on for file String scanLoadModule = props.getFileProperty('cobol_scanLoadModule', buildFile) if (scanLoadModule && scanLoadModule.toBoolean()) @@ -116,20 +116,29 @@ sortedList.each { buildFile -> } } - //perform Db2 Bind only on User Build and perfromBindPackage property - if (props.userBuild && bindFlag && logicalFile.isSQL() && props.bind_performBindPackage && props.bind_performBindPackage.toBoolean() ) { + //perform Db2 Bind Pkg only on User Build and perfromBindPackage property + bind_performBindPackage = props.getFileProperty('bind_performBindPackage', buildFile) + if (props.userBuild && bindFlag && logicalFile.isSQL() && bind_performBindPackage && bind_performBindPackage.toBoolean()) { int bindMaxRC = props.getFileProperty('bind_maxRC', buildFile).toInteger() - - // if no owner is set, use the user.name as package owner - def owner = ( !props.bind_packageOwner ) ? System.getProperty("user.name") : props.bind_packageOwner - - def (bindRc, bindLogFile) = bindUtils.bindPackage(buildFile, props.cobol_dbrmPDS, props.buildOutDir, props.bind_runIspfConfDir, - props.bind_db2Location, props.bind_collectionID, owner, props.bind_qualifier, props.verbose && props.verbose.toBoolean()); + def (bindRc, bindLogFile) = bindUtils.bindPackage(buildFile, props.cobol_dbrmPDS); if ( bindRc > bindMaxRC) { String errorMsg = "*! The bind package return code ($bindRc) for $buildFile exceeded the maximum return code allowed ($props.bind_maxRC)" println(errorMsg) props.error = "true" - buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_bind.log":bindLogFile]) + buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_bind_pkg.log":bindLogFile]) + } + } + + //perform Db2 Bind Pkg only on User Build and perfromBindPackage property + bind_performBindPlan = props.getFileProperty('bind_performBindPlan', buildFile) + if (props.userBuild && bindFlag && logicalFile.isSQL() && bind_performBindPlan && bind_performBindPlan.toBoolean()) { + int bindMaxRC = props.getFileProperty('bind_maxRC', buildFile).toInteger() + def (bindRc, bindLogFile) = bindUtils.bindPlan(buildFile); + if ( bindRc > bindMaxRC) { + String errorMsg = "*! The bind plan return code ($bindRc) for $buildFile exceeded the maximum return code allowed ($props.bind_maxRC)" + println(errorMsg) + props.error = "true" + buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_bind_plan.log":bindLogFile]) } } @@ -186,23 +195,20 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe MVSExec compile = new MVSExec().file(buildFile).pgm(compiler).parm(parms) // add DD statements to the compile command - - if (isZUnitTestCase){ - compile.dd(new DDStatement().name("SYSIN").dsn("${props.cobol_testcase_srcPDS}($member)").options('shr').report(true)) - } - else - { + if (isTazUnitTestCase){ + compile.dd(new DDStatement().name("SYSIN").dsn("${props.cobol_testcase_srcPDS}($member)").options('shr').report(true)) + } else { compile.dd(new DDStatement().name("SYSIN").dsn("${props.cobol_srcPDS}($member)").options('shr').report(true)) } compile.dd(new DDStatement().name("SYSPRINT").options(props.cobol_printTempOptions)) compile.dd(new DDStatement().name("SYSMDECK").options(props.cobol_tempOptions)) (1..15).toList().each { num -> - compile.dd(new DDStatement().name("SYSUT$num").options(props.cobol_tempOptions)) + compile.dd(new DDStatement().name("SYSUT$num").options("cyl space(1,1) unit(sysallda) new")) } // define object dataset allocation - compile.dd(new DDStatement().name("SYSLIN").dsn("${props.cobol_objPDS}($member)").options('shr').output(true)) + compile.dd(new DDStatement().name("SYSLIN").dsn("${props.cobol_objPDS}($member)").options('shr').output(true).deployType("OBJ")) // add a syslib to the compile command with optional bms output copybook and CICS concatenation compile.dd(new DDStatement().name("SYSLIB").dsn(props.cobol_cpyPDS).options("shr")) @@ -235,9 +241,9 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe if (buildUtils.isMQ(logicalFile)) compile.dd(new DDStatement().dsn(props.SCSQCOBC).options("shr")) - // add additional zunit libraries - if (isZUnitTestCase) - compile.dd(new DDStatement().dsn(props.SBZUSAMP).options("shr")) + // add additional TAZ libraries + if (isTazUnitTestCase) + compile.dd(new DDStatement().dsn(props.SEQASAMP).options("shr")) // adding alternate library definitions if (props.cobol_dependenciesAlternativeLibraryNameMapping) { @@ -342,7 +348,7 @@ def createLinkEditCommand(String buildFile, LogicalFile logicalFile, String memb // Define SYSIN dd as instream data if (sysin_linkEditInstream) { if (props.verbose) println("*** Generated linkcard input stream: \n $sysin_linkEditInstream") - linkedit.dd(new DDStatement().name("SYSIN").instreamData(sysin_linkEditInstream)) + linkedit.dd(new DDStatement().name("SYSIN").instreamData(sysin_linkEditInstream).options(props.global_instreamDataTempAllocation)) } // add SYSLIN along the reference to SYSIN if configured through sysin_linkEditInstream @@ -351,7 +357,7 @@ def createLinkEditCommand(String buildFile, LogicalFile logicalFile, String memb // add DD statements to the linkedit command String deployType = buildUtils.getDeployType("cobol", buildFile, logicalFile) - if(isZUnitTestCase){ + if(isTazUnitTestCase){ linkedit.dd(new DDStatement().name("SYSLMOD").dsn("${props.cobol_testcase_loadPDS}($member)").options('shr').output(true).deployType('ZUNIT-TESTCASE')) } else { diff --git a/languages/Easytrieve.groovy b/languages/Easytrieve.groovy index 16f9c80b2..57df00ad8 100644 --- a/languages/Easytrieve.groovy +++ b/languages/Easytrieve.groovy @@ -143,7 +143,7 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe } // define object dataset allocation - compile.dd(new DDStatement().name("SYSLIN").dsn("${props.easytrieve_objPDS}($member)").options('shr').output(true)) + compile.dd(new DDStatement().name("SYSLIN").dsn("${props.easytrieve_objPDS}($member)").options('shr').output(true).deployType("OBJ")) // add a syslib to the compile command compile.dd(new DDStatement().name("PANDD").dsn(props.easytrieve_cpyPDS).options("shr")) diff --git a/languages/PLI.groovy b/languages/PLI.groovy index 9264abde6..b888bdd74 100644 --- a/languages/PLI.groovy +++ b/languages/PLI.groovy @@ -184,7 +184,7 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe } // define object dataset allocation - compile.dd(new DDStatement().name("SYSLIN").dsn("${props.pli_objPDS}($member)").options('shr').output(true)) + compile.dd(new DDStatement().name("SYSLIN").dsn("${props.pli_objPDS}($member)").options('shr').output(true).deployType("OBJ")) // add a syslib to the compile command with optional bms output copybook and CICS concatenation compile.dd(new DDStatement().name("SYSLIB").dsn(props.pli_incPDS).options("shr")) @@ -219,7 +219,7 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe // add additional zunit libraries if (isZUnitTestCase) - compile.dd(new DDStatement().dsn(props.SBZUSAMP).options("shr")) + compile.dd(new DDStatement().dsn(props.SEQASAMP).options("shr")) // add a tasklib to the compile command with optional CICS, DB2, and IDz concatenations String compilerVer = props.getFileProperty('pli_compilerVersion', buildFile) @@ -334,7 +334,7 @@ def createLinkEditCommand(String buildFile, LogicalFile logicalFile, String memb // Define SYSIN dd if (sysin_linkEditInstream) { if (props.verbose) println("*** Generated linkcard input stream: \n $sysin_linkEditInstream") - linkedit.dd(new DDStatement().name("SYSIN").instreamData(sysin_linkEditInstream)) + linkedit.dd(new DDStatement().name("SYSIN").instreamData(sysin_linkEditInstream).options(props.global_instreamDataTempAllocation)) } // add SYSLIN along the reference to SYSIN if configured through sysin_linkEditInstream diff --git a/languages/README.md b/languages/README.md index 0bff627ff..e23140b6b 100644 --- a/languages/README.md +++ b/languages/README.md @@ -10,8 +10,9 @@ zAppBuild comes with a number of language specific build scripts. These script * DBDgen.groovy * PSBgen.groovy * MFS.groovy +* zCEE2.groovy * zCEE3.groovy -* ZunitConfig.groovy +* TazUnitTest.groovy * Easytrieve.groovy All language scripts both compile and optionally link-edit programs. The language build scripts are intended to be useful out of the box but depending on the complexity of your applications' build requirements, may require modifications to meet your development team's needs. By following the examples used in the existing language build scripts of keeping all application specific references out of the build scripts and instead using configuration properties with strong default values, the zAppBuild sample can continue to be a generic build solution for all of your specific applications. diff --git a/languages/REXX.groovy b/languages/REXX.groovy index ad4bd1dfe..0d4fe591e 100644 --- a/languages/REXX.groovy +++ b/languages/REXX.groovy @@ -124,7 +124,7 @@ def createCompileCommand(String buildFile, LogicalFile logicalFile, String membe String linkEditStream = props.getFileProperty('rexx_linkEditStream', buildFile) String linkDebugExit = props.getFileProperty('rexx_linkDebugExit', buildFile) - compile.dd(new DDStatement().name("SYSPUNCH").dsn("${props.rexx_objPDS}($member)").options('shr').output(true)) + compile.dd(new DDStatement().name("SYSPUNCH").dsn("${props.rexx_objPDS}($member)").options('shr').output(true).deployType("OBJ")) String deployType = buildUtils.getDeployType("rexx_cexec", buildFile, null) compile.dd(new DDStatement().name("SYSCEXEC").dsn("${props.rexx_cexecPDS}($member)").options('shr').output(true).deployType(deployType)) diff --git a/languages/ZunitConfig.groovy b/languages/TazUnitTest.groovy similarity index 56% rename from languages/ZunitConfig.groovy rename to languages/TazUnitTest.groovy index 106217ba9..19d048a8b 100644 --- a/languages/ZunitConfig.groovy +++ b/languages/TazUnitTest.groovy @@ -3,6 +3,7 @@ import com.ibm.dbb.metadata.* import com.ibm.dbb.dependency.* import com.ibm.dbb.build.* import groovy.transform.* +import com.ibm.dbb.build.JobExec import groovy.xml.* @@ -15,36 +16,36 @@ println("** Building ${argMap.buildList.size()} ${argMap.buildList.size() == 1 ? // verify required build properties buildUtils.assertBuildProperties(props.cobol_requiredBuildProperties) -buildUtils.assertBuildProperties(props.zunit_requiredBuildProperties) +buildUtils.assertBuildProperties(props.tazunittest_requiredBuildProperties) -def langQualifier = "zunit" +def langQualifier = "tazunittest" buildUtils.createLanguageDatasets(langQualifier) int currentBuildFileNumber = 1 // iterate through build list (argMap.buildList.sort()).each { buildFile -> - println "*** (${currentBuildFileNumber++}/${argMap.buildList.size()}) Building file $buildFile" + println "*** (${currentBuildFileNumber++}/${argMap.buildList.size()}) Executing TAZ test case $buildFile" String member = CopyToPDS.createMemberName(buildFile) - File logFile = new File("${props.buildOutDir}/${member}.zunit.jcl.log") - File reportLogFile = new File("${props.buildOutDir}/${member}.zunit.report.log") + File logFile = new File("${props.buildOutDir}/${member}.tazunittest.jcl.log") + File reportLogFile = new File("${props.buildOutDir}/${member}.tazunittest.report.log") - - String dependencySearch = props.getFileProperty('zunit_dependencySearch', buildFile) + + String dependencySearch = props.getFileProperty('tazunittest_dependencySearch', buildFile) SearchPathDependencyResolver dependencyResolver = new SearchPathDependencyResolver(dependencySearch) - + // copy build file and dependency files to data sets - buildUtils.copySourceFiles(buildFile, props.zunit_bzucfgPDS, 'zunit_dependenciesDatasetMapping', null, dependencyResolver) + buildUtils.copySourceFiles(buildFile, props.tazunittest_bzucfgPDS, 'tazunittest_dependenciesDatasetMapping', null, dependencyResolver) // get logical file LogicalFile logicalFile = buildUtils.createLogicalFile(dependencyResolver, buildFile) - + // get playback dependency for bzucfg file from logicalFile - LogicalDependency playbackFile = getPlaybackFile(logicalFile); - + LogicalDependency playbackFile = getPlaybackFile(logicalFile); + // Create JCLExec String - String jobcard = props.jobCard.replace("\\n", "\n") + String jobcard = props.getFileProperty('tazunittest_jobCard', buildFile).replace("\\n", "\n") String jcl = jobcard jcl += """\ \n//* @@ -54,55 +55,55 @@ int currentBuildFileNumber = 1 // SPACE=(TRK,(1,1),RLSE) //* //* Action: Run Test Case... -//RUNNER EXEC PROC=BZUPPLAY, -// BZUCFG=${props.zunit_bzucfgPDS}(${member}), +//RUNNER EXEC PROC=EQAPPLAY, +// BZUCFG=${props.tazunittest_bzucfgPDS}(${member}), // BZUCBK=${props.cobol_testcase_loadPDS}, // BZULOD=${props.cobol_loadPDS}, """ -// Add parms for bzupplay proc / zUnit Runner -zunitParms = props.getFileProperty('zunit_bzuplayParms', buildFile) -jcl += """\ -// PARM=('$zunitParms') + // Add parms for eqapplay proc / TAZ Runner + unitTestParms = props.getFileProperty('tazunittest_eqaplayParms', buildFile) + jcl += """\ +// PARM=('$unitTestParms') """ if (playbackFile != null) { // bzucfg contains reference to a playback file jcl += - "//REPLAY.BZUPLAY DD DISP=SHR, \n" + - "// DSN=${props.zunit_bzuplayPDS}(${playbackFile.getLname()}) \n" + "//REPLAY.BZUPLAY DD DISP=SHR, \n" + + "// DSN=${props.tazunittest_bzuplayPDS}(${playbackFile.getLname()}) \n" } else { // no playbackfile referenced jcl += - "//REPLAY.BZUPLAY DD DUMMY \n" + "//REPLAY.BZUPLAY DD DUMMY \n" } jcl += """\ //REPLAY.BZURPT DD DISP=SHR, -// DSN=${props.zunit_bzureportPDS}(${member}) +// DSN=${props.tazunittest_bzureportPDS}(${member}) """ -// Add parms for bzupplay proc / zUnit Runner -zunitDebugParm = props.getFileProperty('zunit_userDebugSessionTestParm', buildFile) + // Add debugger parameters + debugParms = props.getFileProperty('tazunittest_userDebugSessionTestParm', buildFile) -// if code coverage collection is activated + // add code coverage collection if activated if (props.codeZunitCoverage && props.codeZunitCoverage.toBoolean()) { - // codeCoverageHost + // codeCoverageHost if (props.codeCoverageHeadlessHost != null) codeCoverageHost = props.codeCoverageHeadlessHost - else - codeCoverageHost = props.getFileProperty('zunit_CodeCoverageHost', buildFile) - // codeCoveragePort + else + codeCoverageHost = props.getFileProperty('tazunittest_CodeCoverageHost', buildFile) + // codeCoveragePort if (props.codeCoverageHeadlessPort != null) codeCoveragePort = props.codeCoverageHeadlessPort - else - codeCoveragePort = props.getFileProperty('zunit_CodeCoveragePort', buildFile) + else + codeCoveragePort = props.getFileProperty('tazunittest_CodeCoveragePort', buildFile) // codeCoverageOptions if (props.codeCoverageOptions != null) codeCoverageOptions = props.codeCoverageOptions - else - codeCoverageOptions = props.getFileProperty('zunit_CodeCoverageOptions', buildFile) - + else + codeCoverageOptions = props.getFileProperty('tazunittest_CodeCoverageOptions', buildFile) + jcl += - "//CEEOPTS DD * \n" + - ( ( codeCoverageHost != null && codeCoveragePort != null && !props.userBuild ) ? "TEST(,,,TCPIP&${codeCoverageHost}%${codeCoveragePort}:*) \n" : "${zunitDebugParm} \n" ) + - "ENVAR(\n" + "//CEEOPTS DD * \n" + + ( ( codeCoverageHost != null && codeCoveragePort != null && !props.userBuild ) ? "TEST(,,,TCPIP&${codeCoverageHost}%${codeCoveragePort}:*) \n" : "${debugParms} \n" ) + + "ENVAR(\n" if (codeCoverageOptions != null) { optionsParms = splitCCParms('"' + "EQA_STARTUP_KEY=CC,${member},t=${member},i=${member}," + codeCoverageOptions + '")'); optionsParms.each { optionParm -> @@ -111,12 +112,12 @@ zunitDebugParm = props.getFileProperty('zunit_userDebugSessionTestParm', buildFi } else { jcl += '"' + "EQA_STARTUP_KEY=CC,${member},t=${member},i=${member}" +'")' + "\n" } - jcl += "/* \n" + jcl += "/* \n" } else if (props.debugzUnitTestcase && props.userBuild) { - // initiate debug session of test case + // initiate debug session of test case jcl += - "//CEEOPTS DD * \n" + - "${zunitDebugParm} \n" + "//CEEOPTS DD * \n" + + "${debugParms} \n" } jcl += """\ //* @@ -132,18 +133,15 @@ zunitDebugParm = props.getFileProperty('zunit_userDebugSessionTestParm', buildFi def dbbConf = System.getenv("DBB_CONF") // Create jclExec - def zUnitRunJCL = new JCLExec().text(jcl) - zUnitRunJCL.confDir(dbbConf) - - // Execute jclExec - zUnitRunJCL.execute() + def tazUnitTestRunJcl = new JobExec().text(jcl).buildFile(buildFile) + def rc = tazUnitTestRunJcl.execute() /** * Store results */ // Save Job Spool to logFile - zUnitRunJCL.saveOutput(logFile, props.logEncoding) + tazUnitTestRunJcl.saveOutput(logFile, props.logEncoding) // // Extract Job BZURPT as XML // def logEncoding = "UTF-8" @@ -173,46 +171,29 @@ zunitDebugParm = props.getFileProperty('zunit_userDebugSessionTestParm', buildFi // Evaluate if running in preview build mode if (!props.preview) { - - // Splitting the String into a StringArray using CC as the seperator - def jobRcStringArray = zUnitRunJCL.maxRC.split("CC") - - // This evals the number of items in the ARRAY! Dont get confused with the returnCode itself - if ( jobRcStringArray.length > 1 ){ - // Ok, the string can be splitted because it contains the keyword CC : Splitting by CC the second record contains the actual RC - rc = zUnitRunJCL.maxRC.split("CC")[1].toInteger() - - // manage processing the RC, up to your logic. You might want to flag the build as failed. - if (rc <= props.zunit_maxPassRC.toInteger()){ - println "*** zUnit Test Job ${zUnitRunJCL.submittedJobId} completed with $rc " - // Store Report in Workspace - new CopyToHFS().dataset(props.zunit_bzureportPDS).member(member).file(reportLogFile).hfsEncoding(props.logEncoding).append(false).copy() - // printReport - printReport(reportLogFile) - } else if (rc <= props.zunit_maxWarnRC.toInteger()){ - String warningMsg = "*! The zunit test returned a warning ($rc) for $buildFile" - // Store Report in Workspace - new CopyToHFS().dataset(props.zunit_bzureportPDS).member(member).file(reportLogFile).hfsEncoding(props.logEncoding).append(false).copy() - // print warning and report - println warningMsg - printReport(reportLogFile) - buildUtils.updateBuildResult(warningMsg:warningMsg,logs:["${member}_zunit.log":logFile]) - } else { // rc > props.zunit_maxWarnRC.toInteger() - props.error = "true" - String errorMsg = "*! The zunit test failed with RC=($rc) for $buildFile " - println(errorMsg) - buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_zunit.log":logFile]) - } - } - else { - // We don't see the CC, assume an exception + // manage processing the RC, up to your logic. You might want to flag the build as failed. + if (rc <= props.tazunittest_maxPassRC.toInteger()){ + println "*** TAZ Unit Test job ${tazUnitTestRunJcl.submittedJobId} completed with $rc " + // Store Report in Workspace + new CopyToHFS().dataset(props.tazunittest_bzureportPDS).member(member).file(reportLogFile).copyMode(DBBConstants.CopyMode.valueOf("BINARY")).append(false).copy() + // printReport + printReport(reportLogFile) + } else if (rc <= props.tazunittest_maxWarnRC.toInteger()){ + String warningMsg = "*! TAZ Unit Test returned a warning ($rc) for $buildFile" + // Store Report in Workspace + new CopyToHFS().dataset(props.tazunittest_bzureportPDS).member(member).file(reportLogFile).copyMode(DBBConstants.CopyMode.valueOf("BINARY")).append(false).copy() + // print warning and report + println warningMsg + printReport(reportLogFile) + buildUtils.updateBuildResult(warningMsg:warningMsg,logs:["${member}_tazunittest.log":logFile]) + } else { // rc > props.tazunittest_maxWarnRC.toInteger() props.error = "true" - String errorMsg = "*! zUnit Test Job ${zUnitRunJCL.submittedJobId} failed with ${zUnitRunJCL.maxRC}" + String errorMsg = "*! TAZ Unit Test failed with RC=($rc) for $buildFile " println(errorMsg) - buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_zunit.log":logFile]) + buildUtils.updateBuildResult(errorMsg:errorMsg,logs:["${member}_tazunittest.log":logFile]) } } else { // skip evaluating Unit tests result - if (props.verbose) println "*** Evaluation of zUnit test result skipped, because running in preview mode." + if (props.verbose) println "*** Evaluation of TAZ Unit Test result skipped, because running in preview mode." } } @@ -225,12 +206,12 @@ zunitDebugParm = props.getFileProperty('zunit_userDebugSessionTestParm', buildFi * returns the LogicalDependency of the playbackfile */ def getPlaybackFile(LogicalFile logicalFile) { - // find playback file dependency - LogicalDependency playbackDependency = logicalFile.getLogicalDependencies().find { - it.getLibrary() == "SYSPLAY" - } + // find playback file dependency + LogicalDependency playbackDependency = logicalFile.getLogicalDependencies().find { + it.getLibrary() == "SYSPLAY" + } return playbackDependency - } +} /** * Parsing the result file and prints summary of the result @@ -244,7 +225,6 @@ def printReport(File resultFile) { reportString = new FileInputStream(resultFile).getText("IBM-1047") try { - def runnerResult = new XmlParser().parseText(reportString) def testCase = runnerResult.testCase println "****************** Module ${testCase.@moduleName} ******************" @@ -257,7 +237,7 @@ def printReport(File resultFile) { } println "****************** Module ${testCase.@moduleName} ****************** \n" } catch (Exception e) { - print "! Reading zUnit result failed." + print "! Reading TAZ Unit Test result failed." } } diff --git a/languages/zCEE2.groovy b/languages/zCEE2.groovy index 7eabb54d6..5da8cc5f4 100644 --- a/languages/zCEE2.groovy +++ b/languages/zCEE2.groovy @@ -4,8 +4,8 @@ import com.ibm.dbb.build.* import groovy.transform.* import com.ibm.dbb.build.report.* import com.ibm.dbb.build.report.records.* -import java.nio.file.*; - +import java.nio.file.* +import groovy.io.FileType // define script properties @Field BuildProperties props = BuildProperties.getInstance() @@ -20,31 +20,30 @@ HashMap updatedBuildList = new HashMap() println("** Streamlining the build list to remove duplicates") argMap.buildList.each { buildFile -> PropertyMappings inputTypeMappings = new PropertyMappings("zcee2_inputType") - inputType = inputTypeMappings.getValue(buildFile) - - if (inputType) { - if (inputType == "PROJECT") { - File changedBuildFile = new File(buildFile); - File projectDir = changedBuildFile.getParentFile() - boolean projectDirFound = false - while (projectDir != null && !projectDirFound) { - File projectFile = new File(projectDir.getPath() + '/.project') - if (projectFile.exists()) { - projectDirFound = true - } else { - projectDir = projectDir.getParentFile() - } - } - if (projectDirFound) { - updatedBuildList.putIfAbsent(projectDir.getPath(), "PROJECT") + String inputType = inputTypeMappings.getValue(buildFile) + if (!inputType || inputType.isEmpty()) { + inputType = props.zcee2_inputType + println("*! [WARNING] No Input Type mapping for file ${buildFile}, using default type '${inputType}'") + } + if (inputType == "PROJECT") { + File changedBuildFile = new File(buildFile); + File projectDir = changedBuildFile.getParentFile() + boolean projectDirFound = false + while (projectDir != null && !projectDirFound) { + File projectFile = new File(projectDir.getPath() + '/.project') + if (projectFile.exists()) { + projectDirFound = true } else { - if (props.verbose) println("!* No project directory found for file '${buildFile}'. Skipping...") + projectDir = projectDir.getParentFile() } + } + if (projectDirFound) { + updatedBuildList.putIfAbsent(projectDir.getPath(), "PROJECT") } else { - updatedBuildList.put(buildFile, inputType); + if (props.verbose) println("!* No project directory found for file '${buildFile}'. Skipping...") } } else { - println("!* No Input Type mapping for file ${buildFile}, skipping it...") + updatedBuildList.put(buildFile, inputType); } } @@ -59,22 +58,33 @@ sortedList.each { buildFile, inputType -> println "*** (${currentBuildFileNumber++}/${sortedList.size()}) Building ${inputType == "PROJECT" ? 'project' : 'properties file'} $buildFile" String parameters = "" - String outputDir = "" String outputFile = "" + String deployType if (inputType == "PROJECT") { - outputDir = "${props.buildOutDir}/zCEE2/$buildFile" - parameters = "-od $outputDir -pd $buildFile" + String outputFileName = Paths.get("zCEE2/$buildFile").getFileName() + // test if package.xml exsist to determine if it's an API project or a SAR project + File packageXMLFile = new File("${props.workspace}/$buildFile/package.xml") + if (packageXMLFile.exists()) { + // It's an API project (AAR) + outputFile = "zCEE2/$buildFile/${outputFileName}.aar" + deployType = "zCEE2-AAR" + } else { + // It's a Service project (SAR) + outputFile = "zCEE2/$buildFile/${outputFileName}.sar" + deployType = "zCEE2-SAR" + } + parameters = "-f ${props.buildOutDir}/$outputFile -pd $buildFile" } else { File changedBuildFile = new File(buildFile); String outputFileName = changedBuildFile.getName().split("\\.")[0] + "." + inputType.toLowerCase() File projectDir = changedBuildFile.getParentFile() - outputFile = "${props.buildOutDir}/zCEE2/${projectDir.getPath()}/${outputFileName}" - outputDir = "${props.buildOutDir}/zCEE2/${projectDir.getPath()}" - parameters = "-f $outputFile -p $buildFile" + outputFile = "zCEE2/${projectDir.getPath()}/${outputFileName}" + deployType = "zCEE2-$inputType" + parameters = "-f ${props.buildOutDir}/$outputFile -p $buildFile" } - File outputDirectory = new File(outputDir) - outputDirectory.mkdirs() - + Path outputDirPath = Paths.get("${props.buildOutDir}/${outputFile}").getParent() + File outputDir = outputDirPath.toFile() + outputDir.mkdirs() Properties ARAproperties = new Properties() File dataStructuresLocation @@ -85,15 +95,17 @@ sortedList.each { buildFile, inputType -> ARApropertiesFile.withInputStream { ARAproperties.load(it) } - println("*** dataStructuresLocation: ${ARAproperties.dataStructuresLocation}") - println("*** apiInfoFileLocation: ${ARAproperties.apiInfoFileLocation}") - println("*** logFileDirectory: ${ARAproperties.logFileDirectory}") + if (props.verbose) { + println("*** dataStructuresLocation: ${ARAproperties.dataStructuresLocation}") + println("*** apiInfoFileLocation: ${ARAproperties.apiInfoFileLocation}") + println("*** logFileDirectory: ${ARAproperties.logFileDirectory}") + } dataStructuresLocation = new File(ARAproperties.dataStructuresLocation) dataStructuresLocation.mkdirs() apiInfoFileLocation = new File(ARAproperties.apiInfoFileLocation) apiInfoFileLocation.mkdirs() logFileDirectory = new File(ARAproperties.logFileDirectory) - logFileDirectory.mkdirs() + logFileDirectory.mkdirs() } @@ -102,121 +114,78 @@ sortedList.each { buildFile, inputType -> File logFile = new File("${props.buildOutDir}/${logFilePath}.zCEE2.log") if (logFile.exists()) logFile.delete() - - String zconbtPath = props.getFileProperty('zcee2_zconbtPath', buildFile) - - File zconbt = new File(zconbtPath) - if (!zconbt.exists()) { - def errorMsg = "*! zconbt wasn't find at location '$zconbtPath'" - println(errorMsg) - props.error = "true" - buildUtils.updateBuildResult(errorMsg:errorMsg) - } else { - String[] command; - command = [zconbtPath, parameters] - String commandString = command.join(" ") - if (props.verbose) - println("** Executing command '${commandString}'...") + String zconbtPath = props.zcee2_zconbtPath - StringBuffer shellOutput = new StringBuffer() - StringBuffer shellError = new StringBuffer() + if (buildUtils.fileExists(zconbtPath, "z/OS Connect EE OpenAPI 2 processing")) { + String encoding = props.logEncoding ?: 'IBM-1047' + ArrayList optionsList = new ArrayList(Arrays.asList(parameters)) + if (props.verbose) + println("*** Executing command '${zconbtPath}' with options '${optionsList}'") - String javaHome = props.getFileProperty('zcee2_JAVA_HOME', buildFile) - if (!javaHome) { - javaHome = System.getenv("JAVA_HOME") - } + UnixExec zCEE2Execution = new UnixExec().command(zconbtPath) + zCEE2Execution.setOptions(optionsList) + zCEE2Execution.output(logFile.getAbsolutePath()).mergeErrors(true) + zCEE2Execution.setFile(buildFile) + zCEE2Execution.setOutputEncoding(encoding) - ProcessBuilder cmd = new ProcessBuilder(zconbtPath, parameters); - Map env = cmd.environment(); - env.put("JAVA_HOME", javaHome); - env.put("PATH", javaHome + "/bin" + ":" + env.get("PATH")) - Process process = cmd.start() - process.consumeProcessOutput(shellOutput, shellError) - process.waitFor() - if (props.verbose) - println("** Exit value for the zconbt process: ${process.exitValue()}"); - - // write outputs to log file - String enc = props.logEncoding ?: 'IBM-1047' - logFile.withWriter(enc) { writer -> - writer.append(shellOutput) - writer.append(shellError) - } - - if (process.exitValue() != 0) { - def errorMsg = "*! Error during the zconbt process" - println(errorMsg) - if (props.verbose) - println("*! zconbt error message:\n${shellError}") - props.error = "true" - buildUtils.updateBuildResult(errorMsg:errorMsg) - } else { - if (props.verbose) - println("** zconbt output:\n${shellOutput}") - - ArrayList outputProperty = [] - Path outputDirectoryPath = Paths.get(props.buildOutDir) - if (inputType == "PROJECT") { - String[] outputFiles = outputDirectory.list() - for (int i=0; i + record.getType().equals("UNIX") } - } else { - Path outputFilePath = Paths.get(outputFile) - Path relativeOutputFilePath = outputDirectoryPath.relativize(outputFilePath) - outputProperty.add("[${props.buildOutDir}, ${relativeOutputFilePath.toString()}, zCEE2]") - - if (inputType == "ARA") { - def ARA_PackageArtifacts = props.getFileProperty('zcee2_ARA_PackageArtifacts', buildFile) - if (ARA_PackageArtifacts && ARA_PackageArtifacts.toBoolean()) { - - String[] outputFiles - Path finalOutputFilePath - - outputFiles = dataStructuresLocation.list() - File dataStructuresLocationDir = new File("${props.buildOutDir}/zCEE2/dataStructures") - dataStructuresLocationDir.mkdirs() - for (int i=0; i + record.getFile().equals(buildFile) + } + if (unixRecord) { + // Captures the generated data structures + File targetDataStructuresLocationDir = new File("${props.buildOutDir}/zCEE2/${buildFile.replace(".properties", "")}/dataStructures") + targetDataStructuresLocationDir.mkdirs() + dataStructuresLocation.eachFile(FileType.FILES) { file -> + Path targetFile = Paths.get(targetDataStructuresLocationDir.getPath() + "/" + file.getName()) + Files.copy(file.toPath(), targetFile, StandardCopyOption.COPY_ATTRIBUTES) + Path relativeOutputFilePath = Paths.get(props.buildOutDir).relativize(targetFile) + unixRecord.addOutput(props.buildOutDir, relativeOutputFilePath.toString(), "zCEE2-Copy") } - outputFiles = apiInfoFileLocation.list() - File apiInfoFileLocationDir = new File("${props.buildOutDir}/zCEE2/apiInfoFiles") - apiInfoFileLocationDir.mkdirs() - for (int i=0; i + Path targetFile = Paths.get(targetApiInfoFileLocationDir.getPath() + "/" + file.getName()) + Files.copy(file.toPath(), targetFile, StandardCopyOption.COPY_ATTRIBUTES) + Path relativeOutputFilePath = Paths.get(props.buildOutDir).relativize(targetFile) + unixRecord.addOutput(props.buildOutDir, relativeOutputFilePath.toString(), "zCEE2-Info") } - outputFiles = logFileDirectory.list() - File logFileDirectoryDir = new File("${props.buildOutDir}/zCEE2/logs") - logFileDirectoryDir.mkdirs() - for (int i=0; i + Path targetFile = Paths.get(targetLogFileDirectoryDir.getPath() + "/" + file.getName()) + Files.copy(file.toPath(), targetFile, StandardCopyOption.COPY_ATTRIBUTES) + Path relativeOutputFilePath = Paths.get(props.buildOutDir).relativize(targetFile) + unixRecord.addOutput(props.buildOutDir, relativeOutputFilePath.toString(), "zCEE2-Logs") } } } } - AnyTypeRecord zCEEWARRecord = new AnyTypeRecord("USS_RECORD") - zCEEWARRecord.setAttribute("file", buildFile) - zCEEWARRecord.setAttribute("label", "z/OS Connect EE OpenAPI 2 definition") - zCEEWARRecord.setAttribute("outputs", outputProperty.join(";")) - zCEEWARRecord.setAttribute("command", commandString); - BuildReportFactory.getBuildReport().addRecord(zCEEWARRecord) + } + + if (props.verbose) + println("** Exit value for the zconbt process: $returnCode") + + if (returnCode != 0) { + def errorMsg = "*! Error during the zconbt process - Please check the log file at '${logFile.getAbsolutePath()}'." + println(errorMsg) + props.error = "true" + buildUtils.updateBuildResult(errorMsg:errorMsg) } } } \ No newline at end of file diff --git a/languages/zCEE3.groovy b/languages/zCEE3.groovy index 8c3461f4e..c0a2672dd 100644 --- a/languages/zCEE3.groovy +++ b/languages/zCEE3.groovy @@ -32,8 +32,6 @@ sortedList.each { buildFile -> if (props.verbose) println("** Expected location of the 'api.war' file: ${WarLocation}") File WarFile = new File(WarLocation) - String[] command; - String commandString; // log file - Changing slashes with dots to avoid conflicts String member = buildFile.replace("/", ".") @@ -41,71 +39,49 @@ sortedList.each { buildFile -> if (logFile.exists()) logFile.delete() - String JAVA_OPTS = props.getFileProperty('zcee3_gradle_JAVA_OPTS', buildFile) - String gradlePath = props.getFileProperty('zcee3_gradlePath', buildFile) + String gradlePath = props.zcee3_gradlePath - File gradleExecutable = new File(gradlePath) - if (!gradleExecutable.exists()) { - def errorMsg = "*! gradle wasn't find at location '$gradlePath'" - println(errorMsg) - props.error = "true" - buildUtils.updateBuildResult(errorMsg:errorMsg) - } else { - String shellEnvironment = props.getFileProperty('zcee3_shellEnvironment', buildFile) + if (buildUtils.fileExists(gradlePath, "z/OS Connect EE OpenAPI 3 processing")) { + String shellEnvironment = props.zcee3_shellEnvironment + String encoding = props.logEncoding ?: 'IBM-1047' + ArrayList optionsList = new ArrayList() + optionsList.add(gradlePath) + optionsList.add(gradleBuildLocation) + if (props.zcee3_gradle_debug && props.zcee3_gradle_debug.toBoolean()) + optionsList.add("--debug") - command = [shellEnvironment, gradlePath, gradleBuildLocation] - commandString = command.join(" ") if (props.verbose) - println("** Executing command '${commandString}' in working directory '${workingDir}'...") - StringBuffer shellOutput = new StringBuffer() - StringBuffer shellError = new StringBuffer() + println("*** Executing command '${shellEnvironment}' with options '${optionsList}'") - ProcessBuilder cmd = new ProcessBuilder(shellEnvironment, gradlePath, gradleBuildLocation); - Map env = cmd.environment(); - env.put("JAVA_OPTS", JAVA_OPTS); - cmd.directory(new File(workingDir)); - Process process = cmd.start() - process.consumeProcessOutput(shellOutput, shellError) - process.waitFor() - if (props.verbose) - println("** Exit value for the gradle build: ${process.exitValue()}"); - - // write outputs to log file - String enc = props.logEncoding ?: 'IBM-1047' - logFile.withWriter(enc) { writer -> - writer.append(shellOutput) - writer.append(shellError) - } - - if (process.exitValue() != 0) { - def errorMsg = "*! Error during the gradle process" + UnixExec zCEE3Execution = new UnixExec().command(shellEnvironment) + zCEE3Execution.setOptions(optionsList) + zCEE3Execution.output(logFile.getAbsolutePath()).mergeErrors(true); + zCEE3Execution.setWorkingDirectory(workingDir) + zCEE3Execution.setFile(buildFile) + zCEE3Execution.setOutputEncoding(encoding) + zCEE3Execution.addOutput(props.buildOutDir, "zCEE3/$WarLocation", "zCEE3") + int returnCode = zCEE3Execution.execute() + + if (returnCode != 0) { + String errorMsg = "*! Error during the gradle process. Please check the gradle log file at '${logFile.getAbsolutePath()}'." println(errorMsg) - if (props.verbose) - println("*! gradle error message:\n${shellError}") props.error = "true" buildUtils.updateBuildResult(errorMsg:errorMsg) } else { if (props.verbose) - println("** gradle output:\n${shellOutput}") + println("** gradle return code: ${returnCode}") if (WarFile.exists()) { // Copy api.war to the buildOutDir directory File WarFileTarget = new File(props.buildOutDir + '/zCEE3/' + WarLocation); File WarTargetDir = WarFileTarget.getParentFile(); WarTargetDir.mkdirs(); - Files.copy(WarFile.toPath(), WarFileTarget.toPath(), StandardCopyOption.COPY_ATTRIBUTES); - - AnyTypeRecord zCEEWARRecord = new AnyTypeRecord("USS_RECORD") - zCEEWARRecord.setAttribute("file", buildFile) - zCEEWARRecord.setAttribute("label", "z/OS Connect EE OpenAPI 3 YAML definition") - zCEEWARRecord.setAttribute("outputs", "[${props.buildOutDir}, zCEE3/$WarLocation, zCEE3]") - zCEEWARRecord.setAttribute("command", commandString); - BuildReportFactory.getBuildReport().addRecord(zCEEWARRecord) + Files.copy(WarFile.toPath(), WarFileTarget.toPath(), StandardCopyOption.COPY_ATTRIBUTES); } else { - def errorMsg = "*! Error when searching for the 'api.war' file at location '${WarLocation}'" + String errorMsg = "*! Error when searching for the 'api.war' file at location '${WarLocation}'" println(errorMsg) props.error = "true" buildUtils.updateBuildResult(errorMsg:errorMsg) } } } -} +} \ No newline at end of file diff --git a/samples/MortgageApplication/application-conf/README.md b/samples/MortgageApplication/application-conf/README.md index 0b063de34..9112f5b35 100644 --- a/samples/MortgageApplication/application-conf/README.md +++ b/samples/MortgageApplication/application-conf/README.md @@ -19,6 +19,7 @@ mainBuildBranch | The main build branch of the main application repository. Use gitRepositoryURL | git repository URL of the application repository to establish links to the changed files in the build result properties | false excludeFileList | Files to exclude when scanning or running full build. skipImpactCalculationList | Files for which the impact analysis should be skipped in impact build +addSubmodulesToBuildList | Flag to include Static Sub module files in pipeline builds jobCard | JOBCARD for JCL execs resolveSubsystems | boolean flag to configure the SearchPathDependencyResolver to evaluate if resolved dependencies impact the file flags isCICS, isSQL, isDLI, isMQ when creating the LogicalFile impactSearch | Impact finder resolution search configuration leveraging the SearchPathImpactFinder API. Sample configurations are inlcuded below, next to the previous rule definitions. diff --git a/samples/MortgageApplication/application-conf/application.properties b/samples/MortgageApplication/application-conf/application.properties index 53e39bc14..e52a1da85 100644 --- a/samples/MortgageApplication/application-conf/application.properties +++ b/samples/MortgageApplication/application-conf/application.properties @@ -44,6 +44,17 @@ excludeFileList=.*,**/.*,**/*.properties,**/*.xml,**/*.groovy,**/*.json,**/*.md, # sample: skipImpactCalculationList=**/epsmtout.cpy,**/centralCopybooks/*.cpy skipImpactCalculationList= +# Flag to expand impact analysis to automatically include all mandatory submodules to build list in pipeline builds with a DBB Metadatastore connection based on the intermediate build list after impact analysis +# This functionality queries and relies on the DBB Output collection of the application +# +# The feature is specifically useful for applications using static calls to increase the isolation of the build environment +# If the property is set to true, if a main program undergoes a change, all statically called sub programs are recompiled +# Default: false +# This property is setup as a DBB file property. This allows to enable the feature just for a subset of your build files +# Sample to inspect the intermediate build list add submodules for Cobol build files +# addSubmodulesToBuildList=true :: **/*.cbl +addSubmodulesToBuildList=true :: **/*.cbl + ############################################################### # Build Property management diff --git a/samples/application-conf/README.md b/samples/application-conf/README.md index a7e4ad4c1..3152dd3e6 100644 --- a/samples/application-conf/README.md +++ b/samples/application-conf/README.md @@ -22,6 +22,7 @@ mainBuildBranch | The main build branch of the main application repository. Use gitRepositoryURL | git repository URL of the application repository to establish links to the changed files in the build result properties | false excludeFileList | Files to exclude when scanning or running full build. | false skipImpactCalculationList | Files for which the impact analysis should be skipped in impact build | false +addSubmodulesToBuildList | Flag to include Static Sub module files in pipeline builds | false jobCard | JOBCARD for JCL execs | false **Build Property management** | | loadFileLevelProperties | Flag to enable the zAppBuild capability to load individual artifact properties files for a build file | true @@ -206,15 +207,18 @@ pli_compileSyslibConcatenation | A comma-separated list of libraries to be conca pli_linkEditSyslibConcatenation | A comma-separated list of libraries to be concatenated in syslib during linkEdit step | true ### bind.properties -Application properties used by zAppBuild/language/COBOL.groovy +Application properties used by zAppBuild/utilities/BindUtilities.groovy Property | Description | Overridable --- | --- | --- -bind_performBindPackage | Default variable to perform DB2 bind as part of a DBB User Build (default value:false) | true -bind_runIspfConfDir | | +bind_performBindPackage | Flag to perform DB2 bind package as part of a DBB User Build (default value:false) | true +bind_performBindPlan | Flag to perform DB2 bind plan as part of a DBB User Build (default value:false) | true +bind_jobCard | The Jobcard used for Bind jobs bind_db2Location | The name of the DB2 subsystem | true -bind_collectionID | The DB2 collection (Package) name | true -bind_packageOwner | The owner of the package, if left empty the use executing the command will be used | true +bind_collectionID | The Db2 collection (package) name | true +bind_plan | The Db2 plan name | true +bind_plan_pklist | The package list parameter for the Db2 plan | true +bind_packageOwner | The owner of the package, if left empty the user executing the command will be used | true bind_qualifier | The value of the implicit qualifier | true bind_maxRC | Default bind maximum RC allowed. | true @@ -271,20 +275,21 @@ acbgen_pgmParms | Default ACBgen parameters. | true acbgen_pgmMaxRC | Default ACBgen maximum RC allowed. | true acbgen_deployType | default deployType for build output | true -### ZunitConfig.properties -Application properties used by zAppBuild/language/ZunitConfig.groovy +### TazUnitTest.properties +Application properties used by zAppBuild/language/TazUnitTest.groovy Property | Description | Overridable --- | --- | --- -zunit_maxPassRC | Default zUnit maximum RC allowed for a Pass. | true -zunit_maxWarnRC | Default zUnit maximum RC allowed for a Warninig (everything beyond this value will Fail). | true -zunit_playbackFileExtension | Default zUnit Playback File Extension. | true -zunit_dependencySearch | Default zUnit dependencySearch configuration to configure the SearchPathDependencyResolver. Format is a concatenated string of searchPath configurations. Strings representing the SearchPaths defined in `application-conf/application.properties`. | true -zunit_bzuplayParms | Default options passed to the zUnit runner BZUPLAY | true -zunit_userDebugSessionTestParm | Debug Tool Test parameter to initiate the debug session | true -zunit_CodeCoverageHost | Headless Code Coverage Collector host (if not specified IDz will be used for reporting) | true -zunit_CodeCoveragePort | Headless Code Coverage Collector port (if not specified IDz will be used for reporting) | true -zunit_CodeCoverageOptions | Headless Code Coverage Collector Options | true +tazunittest_maxPassRC | Default zUnit maximum RC allowed for a Pass. | true +tazunittest_maxWarnRC | Default zUnit maximum RC allowed for a Warninig (everything beyond this value will Fail). | true +tazunittest_jobCard | Jobcard for TAZ Unit Test execution jobs | true +tazunittest_playbackFileExtension | Default zUnit Playback File Extension. | true +tazunittest_dependencySearch | Default zUnit dependencySearch configuration to configure the SearchPathDependencyResolver. Format is a concatenated string of searchPath configurations. Strings representing the SearchPaths defined in `application-conf/application.properties`. | true +tazunittest_eqaplayParms | Default options passed to the TAZ runner procedure EQAPPLAY | true +tazunittest_userDebugSessionTestParm | Debug Tool Test parameter to initiate the debug session | true +tazunittest_CodeCoverageHost | Headless Code Coverage Collector host (if not specified IDz will be used for reporting) | true +tazunittest_CodeCoveragePort | Headless Code Coverage Collector port (if not specified IDz will be used for reporting) | true +tazunittest_CodeCoverageOptions | Headless Code Coverage Collector Options | true ### CRB.properties Application properties used by zAppBuild/language/CRB.groovy diff --git a/samples/application-conf/TazUnitTest.properties b/samples/application-conf/TazUnitTest.properties new file mode 100755 index 000000000..02166f4a7 --- /dev/null +++ b/samples/application-conf/TazUnitTest.properties @@ -0,0 +1,41 @@ +# Application properties used by zAppBuild/language/TazUnitTest.groovy + +# default TAZ unit test maximum return codes +# can be overridden by file properties +tazunittest_maxPassRC=4 +tazunittest_maxWarnRC=8 + +# +# file extension of taz recording files +tazunittest_playbackFileExtension=rec + +# +# Job card, please use \n to indicate a line break and use \ to break the line in this property file +# Example: jobCard=//TAZTEST JOB ,MSGCLASS=H,CLASS=A,NOTIFY=&SYSUID,REGION=0M +tazunittest_jobCard=//TAZTEST JOB ,MSGCLASS=H,CLASS=A,NOTIFY=&SYSUID,REGION=0M + +# +# TAZ unit test dependencySearch configuration +# searchPath defined in application.properties +tazunittest_dependencySearch=${tazRecordingFileSearch} + +# +# Options passed to the EQAPPLAY procedure +# can be overridden by file properties +tazunittest_eqaplayParms=STOP=E,REPORT=XML + +# Code Coverage parameters +#tazunittest_CodeCoverageHost= +#tazunittest_CodeCoveragePort= + +# Code Coverage options for the Headless Collector +# As documented in: https://www.ibm.com/docs/en/developer-for-zos/17.0?topic=applications-specifying-code-coverage-options-in-startup-key +# Sample value - To generate PDF and SonarQube reports located in the build output directory, specify the value "e='CCPDF,CCSONARQUBE',o=${buildOutDir}" (without double quotes) +#tazunittest_CodeCoverageOptions= + +# +# Debug Tool Test parameter to initiate the debug session +# Default uses Debug Manager +# Example: Test Runtime Options: https://www.ibm.com/docs/en/developer-for-zos/15.0.0?topic=string-example-test-runtime-options +tazunittest_userDebugSessionTestParm=TEST(ERROR,,PROMPT,DBMDT:*) + diff --git a/samples/application-conf/ZunitConfig.properties b/samples/application-conf/ZunitConfig.properties deleted file mode 100755 index d3f129086..000000000 --- a/samples/application-conf/ZunitConfig.properties +++ /dev/null @@ -1,37 +0,0 @@ -# Application properties used by zAppBuild/language/ZunitConfig.groovy - -# default zUnit maximum RCs allowed -# can be overridden by file properties -zunit_maxPassRC=4 -zunit_maxWarnRC=8 - -# -# file extension of zunit playback files -zunit_playbackFileExtension=plbck - -# -# zUnit dependencySearch configuration -# searchPath defined in application.properties -zunit_dependencySearch=${zunitPlayfileSearch} - -# -# Options passed to the zUnit runner BZUPLAY -# https://www.ibm.com/docs/en/zvtp/2.0?topic=playback-setup-execution-parameters -# can be overridden by file properties -zunit_bzuplayParms=STOP=E,REPORT=XML - -# zUnit Code Coverage paramaters -#zunit_CodeCoverageHost= -#zunit_CodeCoveragePort= - -# zUnit Code Coverage options for the Headless Collector -# As documented in: https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_15.0.0/com.ibm.debug.pdt.codecoverage.zpcl.doc/topics/tccstartup.html -# Sample value - To generate PDF and SonarQube reports located in the build output directory, specify the value "e='CCPDF,CCSONARQUBE',o=${buildOutDir}" (without double quotes) -#zunit_CodeCoverageOptions= - -# -# Debug Tool Test parameter to initiate the debug session -# Default uses Debug Manager -# Example: Test Runtime Options: https://www.ibm.com/docs/en/developer-for-zos/15.0.0?topic=string-example-test-runtime-options -zunit_userDebugSessionTestParm=TEST(ERROR,,PROMPT,DBMDT:*) - diff --git a/samples/application-conf/application.properties b/samples/application-conf/application.properties index 2ad1a2ba3..e2f2519db 100644 --- a/samples/application-conf/application.properties +++ b/samples/application-conf/application.properties @@ -11,7 +11,7 @@ applicationPropFiles=file.properties,bind.properties,Assembler.properties,\ BMS.properties,Cobol.properties,LinkEdit.properties,bind.properties,\ PLI.properties,MFS.properties,PSBgen.properties,DBDgen.properties,\ -ACBgen.properties,REXX.properties,ZunitConfig.properties,\ +ACBgen.properties,REXX.properties,TazUnitTest.properties,\ Transfer.properties,CRB.properties,\ reports.properties,languageConfigurationMapping.properties @@ -28,7 +28,7 @@ LinkEdit.groovy,DBDgen.groovy,PSBgen.groovy,Transfer.groovy,CRB.groovy,zCEE2.gro # # Comma seperated list of the test script processing order -testOrder=ZunitConfig.groovy +testOrder=TazUnitTest.groovy # # Flag to log output in table views instead of printing raw JSON data @@ -61,6 +61,18 @@ excludeFileList=.*,**/.*,**/*.xml,**/*.groovy,**/*.json,**/*.md,**/application-c # sample: skipImpactCalculationList=**/epsmtout.cpy,**/centralCopybooks/*.cpy skipImpactCalculationList= +# +# Flag to expand impact analysis to automatically include all mandatory submodules to build list in pipeline builds with a DBB Metadatastore connection based on the intermediate build list after impact analysis +# This functionality queries and relies on the DBB Output collection of the application +# +# The feature is specifically useful for applications using static calls to increase the isolation of the build environment +# If the property is set to true, if a main program undergoes a change, all statically called sub programs are recompiled +# Default: false +# This property is setup as a DBB file property. This allows to enable the feature just for a subset of your build files +# Sample to inspect the intermediate build list add submodules for Cobol build files +# addSubmodulesToBuildList=true :: **/*.cbl +addSubmodulesToBuildList=false :: **/*.cbl + # # Job card, please use \n to indicate a line break and use \ to break the line in this property file # Example: jobCard=//RUNZUNIT JOB ,MSGCLASS=H,CLASS=A,NOTIFY=&SYSUID,REGION=0M @@ -114,7 +126,7 @@ resolveSubsystems=false # # this configuration is used when running zAppBuild with the --impactBuild option # to calculate impacted files based on the identified changed files -impactSearch=${copybookSearch}${pliincludeSearch}${bmsSearch}${linkSearch}${rexxCopySearch}${zunitTestConfigSearch}${zunitTestcasePgmSearch}${eztMacSearch} +impactSearch=${copybookSearch}${pliincludeSearch}${bmsSearch}${linkSearch}${rexxCopySearch}${tazTestConfigSearch}${tazTestcasePgmSearch}${eztMacSearch} # # copybookSearch @@ -193,23 +205,23 @@ rexxCopySearch = search:[SYSLIB:COPY]${workspace}/?path=${application}/rexx/*.re # linkSearch = search:[:LINK]${workspace}/?path=${application}/cobol/*.cbl -# zunitTestConfigSearch -# searchPath to locate zunit config files +# tazTestConfigSearch +# searchPath to locate TAZ test configuration files # used in impact analysis -zunitTestConfigSearch = search:[:ZUNITINC]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli;${application}/testcase/*.cbl;${application}/testcase/*.pli +tazTestConfigSearch = search:[:ZUNITINC]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli;${application}/tests/*.cbl;${application}/tests/*.pli # -# zunitPlayfileSearch -# searchPath to locate zunit playback files +# tazRecordingFileSearch +# searchPath to locate TAZ recording files # used in dependency resolution -zunitPlayfileSearch = search:[SYSPLAY:]${workspace}/?path=${application}/testplayfiles/*.bzuplay +tazRecordingFileSearch = search:[SYSPLAY:]${workspace}/?path=${application}/tests/*.rec # -# zunitTestcasePgmSearch +# tazTestcasePgmSearch # searchPath to locate impacted test case programs # see also build-conf/build.properties -> createTestcaseDependency # used in impact analysis -zunitTestcasePgmSearch = search:[SYSPROG:PROGRAMDEPENDENCY]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli +tazTestcasePgmSearch = search:[SYSPROG:PROGRAMDEPENDENCY]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli # # eztMacSearch diff --git a/samples/application-conf/bind.properties b/samples/application-conf/bind.properties index da97ba1be..b6f729023 100644 --- a/samples/application-conf/bind.properties +++ b/samples/application-conf/bind.properties @@ -1,30 +1,49 @@ # # DB2 Bind Options # Build properties used by BindUtilities.groovy to build a DB2 application package for -# SQL programs in the application. +# SQL programs in the application +# +# Only enabled for user builds # -# Please provide values for the properties listed below. -# Ex: -# SUBSYS = DBB1 # Max Bind RC bind_maxRC=4 # Flag indicating to run DB2 BIND PACKAGE. Set to 'true' to bind SQL programs +# can be overridden by file properties +# Default: false bind_performBindPackage=false -# DBB Toolkit conf (configuration) directory location on zFS. Default value is preset -# This directory may have been relocated to a writable disk by your SYSADMIN -bind_runIspfConfDir= +# Flag indicating to run DB2 BIND PLAN. Set to 'true' to bind SQL programs +# can be overridden by file properties +# Default: false +bind_performBindPlan=false + +# Bind jobcard for the JCL that is generated +# can be overridden by file properties +bind_jobCard=//DB2BIND JOB 'DBB-DB2BIND',MSGLEVEL=(1,1),MSGCLASS=R,NOTIFY=&SYSUID # Specify the name of the DB2 subsystem +# can be overridden by file properties bind_db2Location= # Specify the DB2 collection (Package) +# can be overridden by file properties bind_collectionID= +# Specify the DB2 Plan name +# can be overridden by file properties +bind_plan= + +# Specify the DB2 Plan pklist +# can be overridden by file properties +# example: *.MORTAGE.* +bind_plan_pklist= + # Specify the owner of the package, if empty user.name will be used +# can be overridden by file properties bind_packageOwner= -# Specify the value of the implicit qualifier. +# Specify the value of the implicit qualifier. +# can be overridden by file properties bind_qualifier= diff --git a/samples/application-conf/file.properties b/samples/application-conf/file.properties index e3df501f6..8086c063e 100644 --- a/samples/application-conf/file.properties +++ b/samples/application-conf/file.properties @@ -11,11 +11,11 @@ dbb.scriptMapping = Cobol.groovy :: **/*.cbl dbb.scriptMapping = LinkEdit.groovy :: **/*.lnk dbb.scriptMapping = PLI.groovy :: **/*.pli dbb.scriptMapping = Easytrieve.groovy :: **/*.ezt -dbb.scriptMapping = ZunitConfig.groovy :: **/*.bzucfg +dbb.scriptMapping = TazUnitTest.groovy :: **/*.bzucfg dbb.scriptMapping = Transfer.groovy :: **/*.jcl, **/*.xml dbb.scriptMapping = zCEE3.groovy :: **/openapi.yaml dbb.scriptMapping = CRB.groovy :: **/crb/*.yaml -dbb.scriptMapping = zCEE2.groovy :: **/zosconnect_artefacts/**/*.* +dbb.scriptMapping = zCEE2.groovy :: **/zosconnect_artefacts/apis/*.*,**/zosconnect_artefacts/services/*.*,**/zosconnect_artefacts/apiRequester/*.properties,**/zosconnect_artefacts/SAR/*.properties # # dbb.scannerMapping to map files extensions to DBB dependency scanner configurations @@ -45,20 +45,20 @@ dbb.scriptMapping = zCEE2.groovy :: **/zosconnect_artefacts/**/*.* # or activation of preprocessing # isCICS - boolean flag indicating that the process for the module requires the -# CICS libraries, source code needs preprocessing +# CICS libraries, source code needs preprocessing # # flag is set by the DBB scanner if it detects EXEC CICS statements. -# +# # Override the flag for source code or linkcards that are executed in a CICS environment # to enable proprocessing and to resolve appropriate language interface module -# +# isCICS = true :: **/cobol/member.cbl # isSQL - boolean flag indicating that the process for the module requires the -# Db2 libraries, source code needs preprocessing +# Db2 libraries, source code needs preprocessing # # flag is set by the DBB scanner if it detects SQL statements. -# +# isSQL = true :: **/cobol/member.cbl # isMQ - boolean flag indicating that the process for the module requires the @@ -68,20 +68,20 @@ isSQL = true :: **/cobol/member.cbl # # flag is set by the DBB scanner if it detects MQ calls. # -# Override the flag to force adding +# Override the flag to force adding # isMQ = true :: **/cobol/member.cbl # isIMS - indicating that the process for the module requires the # IMS libraries in the link phase. Applicable for # DL/I batch programs -# IMS online programs +# IMS online programs # # flag is NOT set by the DBB scanner. # # Set the flag for source code or link cards -# to resolve appropriate IMS language interface module (DFSLI000) -# +# to resolve appropriate IMS language interface module (DFSLI000) +# isIMS = true :: **/cobol/DLIBATCH.cbl @@ -90,7 +90,7 @@ isIMS = true :: **/cobol/DLIBATCH.cbl # # > Please check for available file property overwrites within samples/application-conf/README.md < # The tables indicate which build parameters can be overwritten for each build file to overwrite the default settings. -# +# # The subsequent section provides the configuration parameters to outline # - file level property overwrites by loading individual artifact properties files # - loading language configurations that define multiple build parameters for the mapped files @@ -103,9 +103,9 @@ isIMS = true :: **/cobol/DLIBATCH.cbl # # Flag to enable the zAppBuild capability to load individual artifact properties files for source files. # Example 1: The following file path pattern looks for an individual artifact properties file for epsmlist.cbl: -# loadFileLevelProperties = true :: **/cobol/epsmlist.cbl +# loadFileLevelProperties = true :: **/cobol/epsmlist.cbl # Example 2: Enable individual artifact properties file for all COBOL files: loadFileLevelProperties=true :: **/cobol/*.cbl -# loadFileLevelProperties = true :: **/cobol/*.cbl +# loadFileLevelProperties = true :: **/cobol/*.cbl # # Language Configuration properties management @@ -113,30 +113,30 @@ isIMS = true :: **/cobol/DLIBATCH.cbl # Flag to enable the zAppBuild capability to load language configuration properties files for source files. # Example 1: Enable language configuration mapping for all COBOL files: loadLanguageConfigurationProperties=true :: **/cobol/*.cbl # loadLanguageConfigurationProperties = true :: **/cobol/*.cbl -# Example 2: The following file path pattern looks for a language configuration properties file mapping for epsnbrvl.cbl and epsmpmt.cbl +# Example 2: The following file path pattern looks for a language configuration properties file mapping for epsnbrvl.cbl and epsmpmt.cbl # in the application-conf/languageconfigurationMapping.properties file: # loadLanguageConfigurationProperties = true :: **/cobol/epsnbrvl.cbl, **/cobol/epsmpmt.cbl # # Properties mapping for assigning the language configuration to build files to specify the build configuration # This option is an alternative approach for mapping build files to the language configuration file -# +# # languageConfiguration = languageConfigProps01 :: **/cobol/epsnbrvl.cbl, **/cobol/epsmpmt.cbl # languageConfiguration = languageConfigProps02 :: **/cobol/epscmort.cbl ######### -# zUnit Integration +# TAZ Unit Test Integration ######### # # file mapping for generated zUnit Test case programs (Cobol) to use a seperate set of libraries -# cobol_testcase = true :: **/testcase/*.cbl +# cobol_testcase = true :: **/tests/*.cbl # file mapping for generated zUnit Test case programs (PL/I) to use a seperate set of libraries -# pli_testcase = true :: **/testcase/*.pli +# pli_testcase = true :: **/tests/*.pli # skip creating IDENTIFY binder control card statement for generated zunit test cases -# cobol_identifyLoad=false **/testcase/*.cbl, **/testcase/*.pli +# cobol_identifyLoad=false **/tests/*.cbl, **/tests/*.pli # mapping for overwriting the impactResolution rules in application.properties # impactResolutionRules=[${copybookRule},${linkRule}] :: **/copy/*.cpy,**/cobol/*.cbl diff --git a/test/README.md b/test/README.md index befef217b..0c98ed39a 100644 --- a/test/README.md +++ b/test/README.md @@ -163,7 +163,7 @@ OUTPUT STREAM: ** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/LinkEdit.properties ** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/PLI.properties ** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/REXX.properties -** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/ZunitConfig.properties +** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/TazUnitTest.properties ** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/Transfer.properties ** Loading property file /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/build-conf/defaultzAppBuildConf.properties ** appConf = /ZT01/var/jenkins/workspace/dbb-zappbuild-testframework-withParms/samples/MortgageApplication/application-conf diff --git a/test/applications/HelloWorld/application-conf/application.properties b/test/applications/HelloWorld/application-conf/application.properties index d20f7163a..d049932c2 100644 --- a/test/applications/HelloWorld/application-conf/application.properties +++ b/test/applications/HelloWorld/application-conf/application.properties @@ -22,7 +22,7 @@ buildOrder=BMS.groovy,MFS.groovy,Cobol.groovy,Assembler.groovy,PLI.groovy,LinkEd # # Comma seperated list of the test script processing order -testOrder=ZunitConfig.groovy +testOrder=TazUnitTest.groovy # # Flag to log output in table views instead of printing raw JSON data @@ -185,13 +185,13 @@ linkSearch = search:[:LINK]${workspace}/?path=${application}/cobol/*.cbl # zunitTestConfigSearch # searchPath to locate zunit config files # used in impact analysis -zunitTestConfigSearch = search:[:ZUNITINC]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli;${application}/testcase/*.cbl;${application}/testcase/*.pli +zunitTestConfigSearch = search:[:ZUNITINC]${workspace}/?path=${application}/cobol/*.cbl;${application}/pli/*.pli;${application}/tests/*.cbl;${application}/tests/*.pli # -# zunitPlayfileSearch +# tazRecordingFileSearch # searchPath to locate zunit playback files # used in dependency resolution -zunitPlayfileSearch = search:[SYSPLAY:]${workspace}/?path=${application}/testplayfiles/*.bzuplay +tazRecordingFileSearch = search:[SYSPLAY:]${workspace}/?path=${application}/tests/*.rec # # zunitTestcasePgmSearch diff --git a/test/applications/HelloWorld/application-conf/file.properties b/test/applications/HelloWorld/application-conf/file.properties index a2fc2cfa8..377f97cbb 100644 --- a/test/applications/HelloWorld/application-conf/file.properties +++ b/test/applications/HelloWorld/application-conf/file.properties @@ -10,7 +10,7 @@ dbb.scriptMapping = DBDgen.groovy :: **/dbd/*.asm dbb.scriptMapping = Cobol.groovy :: **/*.cbl dbb.scriptMapping = LinkEdit.groovy :: **/*.lnk dbb.scriptMapping = PLI.groovy :: **/*.pli -dbb.scriptMapping = ZunitConfig.groovy :: **/*.bzucfg +dbb.scriptMapping = TazUnitTest.groovy :: **/*.bzucfg dbb.scriptMapping = Transfer.groovy :: **/*.jcl, **/*.xml # diff --git a/test/applications/MortgageApplication/test.properties b/test/applications/MortgageApplication/test.properties index 456ae949e..9d64f60c1 100644 --- a/test/applications/MortgageApplication/test.properties +++ b/test/applications/MortgageApplication/test.properties @@ -67,9 +67,9 @@ impactBuild_changedFiles = bms/epsmort.bms,cobol/epsmlist.cbl,copybook/epsmtout. impactBuild_datasetsToCleanUp = BMS,COBOL,LINK # # Use file properties to associate expected files built to changed files -impactBuild_expectedFilesBuilt = epsmort.bms,epscmort.cbl :: bms/epsmort.bms +impactBuild_expectedFilesBuilt = epsmort.bms,epscmort.cbl,epsnbrvl.cbl :: bms/epsmort.bms impactBuild_expectedFilesBuilt = epsmlist.cbl,epsmlist.lnk :: cobol/epsmlist.cbl -impactBuild_expectedFilesBuilt = epsmlist.cbl,epscsmrt.cbl,epscmort.cbl,epsmlist.lnk :: copybook/epsmtout.cpy +impactBuild_expectedFilesBuilt = epsmlist.cbl,epscsmrt.cbl,epscmort.cbl,epsmlist.lnk,epsnbrvl.cbl :: copybook/epsmtout.cpy impactBuild_expectedFilesBuilt = epsmlist.lnk :: link/epsmlist.lnk @@ -124,7 +124,7 @@ impactBuild_preview_changedFiles = copybook/epsmtout.cpy impactBuild_preview_datasetsToCleanUp = BMS,COBOL,LINK # # Use file properties to associate expected files built to changed files -impactBuild_preview_expectedFilesBuilt = epsmlist.cbl,epscsmrt.cbl,epscmort.cbl,epsmlist.lnk :: copybook/epsmtout.cpy +impactBuild_preview_expectedFilesBuilt = epsmlist.cbl,epscsmrt.cbl,epscmort.cbl,epsmlist.lnk,epsnbrvl.cbl :: copybook/epsmtout.cpy ############################# diff --git a/test/testScripts/impactBuild_preview.groovy b/test/testScripts/impactBuild_preview.groovy index 2c0fb83cd..e41b33cb9 100644 --- a/test/testScripts/impactBuild_preview.groovy +++ b/test/testScripts/impactBuild_preview.groovy @@ -111,14 +111,14 @@ def validateImpactBuild(String changedFile, PropertyMappings filesBuiltMappings, try{ // Validate clean build - assert outputStream.contains("Build State : CLEAN") : "*! IMPACT BUILD FAILED FOR $changedFile\nOUTPUT STREAM:\n$outputStream\n" + assert outputStream.contains("Build State : CLEAN") : "*! IMPACT BUILD WITH PREVIEW FOR $changedFile\nOUTPUT STREAM:\n$outputStream\n" // Validate expected number of files built def numImpactFiles = expectedFilesBuiltList.size() - assert outputStream.contains("Total files processed : ${numImpactFiles}") : "*! IMPACT BUILD FOR $changedFile TOTAL FILES PROCESSED ARE NOT EQUAL TO ${numImpactFiles}\nOUTPUT STREAM:\n$outputStream\n" + assert outputStream.contains("Total files processed : ${numImpactFiles}") : "*! IMPACT BUILD WITH PREVIEW FOR $changedFile TOTAL FILES PROCESSED ARE NOT EQUAL TO ${numImpactFiles}\nOUTPUT STREAM:\n$outputStream\n" // Validate expected built files in output stream - assert expectedFilesBuiltList.count{ i-> outputStream.contains(i) } == expectedFilesBuiltList.size() : "*! IMPACT BUILD FOR $changedFile DOES NOT CONTAIN THE LIST OF BUILT FILES EXPECTED ${expectedFilesBuiltList}\nOUTPUT STREAM:\n$outputStream\n" + assert expectedFilesBuiltList.count{ i-> outputStream.contains(i) } == expectedFilesBuiltList.size() : "*! IMPACT BUILD WITH PREVIEW FOR $changedFile DOES NOT CONTAIN THE LIST OF BUILT FILES EXPECTED ${expectedFilesBuiltList}\nOUTPUT STREAM:\n$outputStream\n" println "**" println "** IMPACT BUILD WITH PREVIEW TEST : PASSED FOR $changedFile **" diff --git a/utilities/ADMIN.pw b/utilities/ADMIN.pw deleted file mode 100644 index b3059fc01..000000000 --- a/utilities/ADMIN.pw +++ /dev/null @@ -1,7 +0,0 @@ - - - -1.0 -1 -903xCaMFPZs= - \ No newline at end of file diff --git a/utilities/BindUtilities.groovy b/utilities/BindUtilities.groovy index c53bc39c5..bca78069f 100644 --- a/utilities/BindUtilities.groovy +++ b/utilities/BindUtilities.groovy @@ -4,109 +4,228 @@ import com.ibm.dbb.metadata.* import com.ibm.dbb.dependency.* import groovy.transform.* import groovy.cli.commons.* +import com.ibm.dbb.build.JobExec /** * This script builds a DB2 application package for SQL programs in the application. */ +@Field BuildProperties props = BuildProperties.getInstance() + bind(args) -def bindPackage(String file, String dbrmHLQ, String workDir, String confDir, String SUBSYS, String COLLID, String OWNER, String QUAL, boolean verbose) { - // define local properties - def dbrmPDS = "${dbrmHLQ}" - def clistPDS = "${dbrmHLQ}.CLIST" - def cmdscpDS = "${dbrmHLQ}.ISPFGWY.EXEC" - def member = CopyToPDS.createMemberName(file) - def logFile = new File("${workDir}/${member}_bind.log") - def srcOptions = "cyl space(1,1) lrecl(80) dsorg(PO) recfm(F,B) dsntype(library) msg(1)" +/** + * Method invoked by zAppBuild language scripts + */ +def bindPackage(String buildFile, String dbrmPDS) { + // Retrieve file overrides + bind_jobcard = props.getFileProperty('bind_jobCard', buildFile) + db2_subsys = props.getFileProperty('bind_db2Location', buildFile) + db2_collection = props.getFileProperty('bind_collectionID', buildFile) + db2_qualifier = props.getFileProperty('bind_qualifier', buildFile) + db2_package_owner = ( !props.bind_packageOwner ) ? System.getProperty("user.name") : props.getFileProperty('bind_packageOwner', buildFile) - println("*** Binding $file") + // execute Bind Package Job + def (bindRc, bindLogFile) = executeBindPackage(buildFile, bind_jobcard, dbrmPDS, props.buildOutDir, db2_subsys, db2_collection, db2_package_owner, db2_qualifier, props.SDSNLOAD, props.verbose && props.verbose.toBoolean()) + return [bindRc, bindLogFile] +} - // create BIND CLIST if necessary - def clist = new File("${workDir}/bind.clist") - if (clist.exists()) { - clist.delete() - } - - clist << """PROC 6 SUBSYS COLLID MEMBER LIB OWNER QUAL - DSN SYSTEM(&SUBSYS) - BIND PACKAGE(&COLLID) + - MEMBER(&MEMBER) + - LIBRARY('&LIB') + - OWNER(&OWNER) + - QUALIFIER(&QUAL) + - ACTION(REPLACE) + - ISOLATION(CS) - END -EXIT CODE(&LASTCC) +/** + * Method invoked by zAppBuild language scripts + */ +def bindPlan(String buildFile) { + + // Retrieve file overrides + bind_jobcard = props.getFileProperty('bind_jobCard', buildFile) + db2_subsys = props.getFileProperty('bind_db2Location', buildFile) + db2_plan = props.getFileProperty('bind_plan', buildFile) + db2_plan_pklist = props.getFileProperty('bind_plan_pklist', buildFile) + db2_qualifier = props.getFileProperty('bind_qualifier', buildFile) + // Do we need a different plan owner? + db2_plan_owner = ( !props.bind_packageOwner ) ? System.getProperty("user.name") : props.getFileProperty('bind_packageOwner', buildFile) + + // execute Bind Package Job + def (bindRc, bindLogFile) = executeBindPlan(buildFile, bind_jobcard, props.buildOutDir, db2_subsys, db2_plan, db2_plan_pklist, db2_plan_owner, db2_qualifier, props.SDSNLOAD, props.verbose && props.verbose.toBoolean()) + return [bindRc, bindLogFile] +} + +/** + * Execute Bind Package Job + */ +def executeBindPackage(String file, String jobCard, String dbrmPDS, String workDir, String db2_subsys, String db2_collection, String db2_package_owner, String db2_qualifier, String sdsn_lib, boolean verbose) { + + def dbrm_member = CopyToPDS.createMemberName(file) + + String jcl = jobCard.replace("\\n", "\n") + jcl += """\ +\n//* +//**PKGBIND +//PKGBIND EXEC PGM=IKJEFT01,DYNAMNBR=20,COND=(4,LT) +//STEPLIB DD DSN=${sdsn_lib},DISP=SHR +//SYSTSPRT DD SYSOUT=* +//SYSPRINT DD SYSOUT=* +//SYSUDUMP DD SYSOUT=* +//SYSIN DD DUMMY +//SYSTSIN DD * +DSN SYSTEM(${db2_subsys}) +BIND PACKAGE(${db2_collection}) + + MEMBER(${dbrm_member}) + + LIBRARY('${dbrmPDS}') + + OWNER(${db2_package_owner}) + + QUALIFIER(${db2_qualifier}) + + ACTION(REPLACE) + + ISOLATION(CS) +END +// """ + // bind the build file + if ( verbose ) { + println("*** Execute Bind Package Job for program $file using \n$jcl")} + + jobExec = new JobExec().text(jcl).buildFile(file) + def rc = jobExec.execute() + + if ( verbose ) { + println("*** Bind Package Job(${jobExec.getSubmittedJobId()}) for program $file completed with RC=$rc")} - // create CLIST PDS if necessary - new CreatePDS().dataset(clistPDS).options(srcOptions).create() + jobExec.saveOutput(new File("${workDir}/${dbrm_member}_bind_pkg.log")) + + return [ + rc, + "${workDir}/${dbrm_member}_bind_pkg.log" + ] + +} + +/** + * Execute Bind Plan Job + */ - // copy CLIST to PDS - if ( verbose ) - println("*** Copying ${workDir}/bind.clist to $clistPDS(BIND)") - new CopyToPDS().file(clist).dataset(clistPDS).member("BIND").execute() +def executeBindPlan(String file, String jobCard, String workDir, String db2_subsys, String db2_plan, String db2_package_list, String db2_plan_owner, String db2_qualifier, String sdsn_lib, boolean verbose) { + String jcl = jobCard.replace("\\n", "\n") + jcl += """\ +\n//* +//**PLNBIND +//PLNBIND EXEC PGM=IKJEFT01,DYNAMNBR=20,COND=(4,LT) +//STEPLIB DD DSN=${sdsn_lib},DISP=SHR +//SYSTSPRT DD SYSOUT=* +//SYSPRINT DD SYSOUT=* +//SYSUDUMP DD SYSOUT=* +//SYSIN DD DUMMY +//SYSTSIN DD * +DSN SYSTEM(${db2_subsys}) +BIND PLAN(${db2_plan}) + + PKLIST(${db2_package_list}) + + OWNER(${db2_plan_owner}) + + QUALIFIER(${db2_qualifier}) + + ACTION(REPLACE) + + ISOLATION(CS) + + RELEASE(COMMIT) + + ENCODING(EBCDIC) +END +// +""" // bind the build file - if ( verbose ) - println("*** Executing CLIST to bind program $file") - - // define TSOExec to run the bind clist - def bind = new TSOExec().file(file) - .command("exec '$clistPDS(BIND)'") - .options("'${SUBSYS} ${COLLID} $member $dbrmPDS ${OWNER} ${QUAL}'") - .logFile(logFile) - .confDir(confDir) - .keepCommandScript(true) - bind.dd(new DDStatement().name("CMDSCP").dsn(cmdscpDS).options("shr")) - - // execute the bind clist - def rc = bind.execute() - - return [rc,"${workDir}/${member}_bind.log"] + if ( verbose ) { + println("*** Execute Bind Plan Job for program $file using \n$jcl")} + + jobExec = new JobExec().text(jcl).buildFile(file) + def rc = jobExec.execute() + + if ( verbose ) { + println("*** Bind Plan Job(${jobExec.getSubmittedJobId()}) for program $file completed with RC=$rc")} + + jobExec.saveOutput(new File("${workDir}/${db2_plan}_bind_plan.log")) + + return [ + rc, + "${workDir}/${db2_plan}_bind_plan.log" + ] } + //Parse the command line and bind def bind(String[] cliArgs) { + + // Use BindUtilities.groovy as a standalone util + + // groovyz BindUtilities.groovy --file /u/dbehm/userBuild/MortgageApplication/cobol/epscmort.cbl \ + // --dbrmPDS DBEHM.UB.DBRM \ + // --workDir /u/dbehm/userBuild/work \ + // --jobCard "//BINDPKG JOB 'DBB-PKGBIND',MSGLEVEL=(1,1),MSGCLASS=R,NOTIFY=&SYSUID" \ + // --subSys DBC1 \ + // --collId MORTCL \ + // --owner DBEHM \ + // --qual MORT \ + // --maxRc 4 \ + // --dsnLibrary DBC0CFG.DB2.V12.SDSNLOAD \ + // --packageList '*.MORTGAGE.*' \ + // --plan MORTPL \ + // --bindPackage \ + // --bindPlan + def cli = new CliBuilder(usage: "BindUtilities.groovy [options]", header: '', stopAtNonOption: false) + cli.f(longOpt:'file', args:1, required:true, 'The build file name.') - cli.d(longOpt:'dbrmHLQ', args:1, required:true, 'DBRM partition data sets') + cli.d(longOpt:'dbrmPDS', args:1, required:true, 'DBRM partition data sets') cli.w(longOpt:'workDir', args:1, required:true, 'Absolute path to the working directory') - cli.c(longOpt:'confDir', args:1, required:true, 'Absolute path to runIspf.sh folder') - + cli.j(longOpt:'jobCard', args:1, required:true, 'Jobcard for Bind JCL') + cli.l(longOpt:'dsnLibrary', args:1, required:true, 'SDSN Load library') cli.s(longOpt:'subSys', args:1, required:true, 'The name of the DB2 subsystem') cli.p(longOpt:'collId', args:1, required:true, 'Specify the DB2 collection (Package)') cli.o(longOpt:'owner', args:1, required:true, 'The owner of the package') cli.q(longOpt:'qual', args:1, required:true, 'The value of the implicit qualifier') cli.m(longOpt:'maxRc', args:1, 'The maximun return value') - + + cli.bpkg(longOpt:'bindPackage', 'Flag execute bind package') + cli.bpla(longOpt:'bindPlan', 'Flag execute bind plan') + cli.pl(longOpt:'plan', args:1, 'The Db2 Plan name') + + + cli.pkl(longOpt:'packageList', args:1, 'Package List for Bind Plan') cli.v(longOpt:'verbose', 'Flag to turn on script trace') - + def opts = cli.parse(cliArgs) - + // if opt parse fail exit. if (! opts) { System.exit(1) } - + if (opts.help) { cli.usage() System.exit(0) } - + def maxRC = opts.m ? opts.m.toInteger() : 0 - def (rc, logFile) = bindPackage(opts.f, opts.d, opts.w, opts.c, opts.s, opts.p, opts.o, opts.q, opts.v) - if ( rc > maxRC ) { - String errorMsg = "*! The bind return code ($rc) for $opts.f exceeded the maximum return code allowed ($maxRC)\n** See: $logFile" - println(errorMsg) - System.exit(1) + + // perform bind package + if (opts.bpkg) { + + def (rc, logFile) = executeBindPackage(opts.f, opts.j, opts.d, opts.w, opts.s, opts.p, opts.o, opts.q, opts.l, opts.v) + if ( rc > maxRC ) { + String errorMsg = "*! The bind package return code ($rc) for $opts.f exceeded the maximum return code allowed ($maxRC)\n** See: $logFile" + println(errorMsg) + System.exit(1) + } + } + + // perform bind plan + if (opts.bpla) { + + def (rc, logFile) = executeBindPlan(opts.f, opts.j, opts.w, opts.s, opts.pl, opts.pkl, opts.o, opts.q, opts.l, opts.v) + if ( rc > maxRC ) { + String errorMsg = "*! The bind plan return code ($rc) for $opts.f exceeded the maximum return code allowed ($maxRC)\n** See: $logFile" + println(errorMsg) + System.exit(1) + } } + } diff --git a/utilities/BuildUtilities.groovy b/utilities/BuildUtilities.groovy index d64f94588..b19101d9f 100644 --- a/utilities/BuildUtilities.groovy +++ b/utilities/BuildUtilities.groovy @@ -146,12 +146,12 @@ def copySourceFiles(String buildFile, String srcPDS, String dependencyDatasetMap copiedFileCache.add(dependencyLoc) // create member name String memberName = CopyToPDS.createMemberName(dependencyPath) - // retrieve zUnit playback file extension - zunitFileExtension = (props.zunit_playbackFileExtension) ? props.zunit_playbackFileExtension : null + // retrieve TAZ recording file extension + tazRecordingFileExtension = (props.tazunittest_playbackFileExtension) ? props.tazunittest_playbackFileExtension : null // get index of last '.' in file path to extract the file extension def extIndex = dependencyLoc.lastIndexOf('.') try { - if( zunitFileExtension && !zunitFileExtension.isEmpty() && (dependencyLoc.substring(extIndex).contains(zunitFileExtension))){ + if( tazRecordingFileExtension && !tazRecordingFileExtension.isEmpty() && (dependencyLoc.substring(extIndex).contains(tazRecordingFileExtension))){ new CopyToPDS().file(new File(dependencyLoc)) .copyMode(CopyMode.BINARY) .dataset(dependencyPDS) @@ -215,10 +215,10 @@ def copySourceFiles(String buildFile, String srcPDS, String dependencyDatasetMap copiedFileCache.add(physicalDependencyLoc) // create member name String memberName = CopyToPDS.createMemberName(physicalDependency.getFile()) - //retrieve zUnitFileExtension plbck - zunitFileExtension = (props.zunit_playbackFileExtension) ? props.zunit_playbackFileExtension : null + //retrieve Taz Recording file + tazRecordingFileExtension = (props.tazunittest_playbackFileExtension) ? props.tazunittest_playbackFileExtension : null try { - if( zunitFileExtension && !zunitFileExtension.isEmpty() && ((physicalDependency.getFile().substring(physicalDependency.getFile().indexOf("."))).contains(zunitFileExtension))){ + if( tazRecordingFileExtension && !tazRecordingFileExtension.isEmpty() && ((physicalDependency.getFile().substring(physicalDependency.getFile().indexOf("."))).contains(tazRecordingFileExtension))){ new CopyToPDS().file(new File(physicalDependencyLoc)) .copyMode(CopyMode.BINARY) .dataset(dependencyPDS) @@ -1009,16 +1009,35 @@ def loadBuildProperties(String propertyFile) { } /** - * Validates if a buildFile is a zUnit generated test case program + * Validates if a buildFile is a TAZ generated test harness program * * returns true / false * */ -def isGeneratedzUnitTestCaseProgram(String buildFile) { +def isGeneratedTazTestCaseProgram(String buildFile) { if (props.getFileProperty('cobol_testcase', buildFile).equals('true') || props.getFileProperty('pli_testcase', buildFile).equals('true')) { return true } return false } +/** + * Checks if a zFS file exists + * if the file exists, returns true + * if not, returns false and updates the Build Result accordingly with an error message + * + */ + +def fileExists(String fileLoc, String errorMessage) { + File file = new File(fileLoc) + if (!file.exists()) { + String errorMsg = "*! ${errorMessage} - $fileLoc not found." + println(errorMsg) + props.error = "true" + buildUtils.updateBuildResult(errorMsg:errorMsg) + return false + } else { + return true + } +} diff --git a/utilities/ImpactUtilities.groovy b/utilities/ImpactUtilities.groovy index 23180eafa..b061886bb 100644 --- a/utilities/ImpactUtilities.groovy +++ b/utilities/ImpactUtilities.groovy @@ -64,8 +64,8 @@ def createImpactBuildList() { // if the changed file has a build script then add to build list if (ScriptMappings.getScriptName(changedFile)) { // skip adding generated test cases, when the testing is disabled - if (buildUtils.isGeneratedzUnitTestCaseProgram(changedFile) && !(props.runzTests && props.runzTests.toBoolean())) { - if (props.verbose) println "** Identified $changedFile as a generated zunit test case program. Processing zUnit tests is not enabled for this build. Skip building this program." + if (buildUtils.isGeneratedTazTestCaseProgram(changedFile) && !(props.runzTests && props.runzTests.toBoolean())) { + if (props.verbose) println "** Identified $changedFile as a generated TAZ unit test case program. Processing TAZ unit tests is not enabled for this build. Skip building this program." } else { buildSet.add(changedFile) if (props.verbose) println "** Found build script mapping for $changedFile. Adding to build list" @@ -122,7 +122,23 @@ def createImpactBuildList() { if (props.verbose) println "** Impact analysis for $changedFile has been skipped due to configuration." } } - + + Set buildLinkSet = new HashSet() + buildSet.each { buildFile -> + String addSubmodulesToBuildList = props.getFileProperty('addSubmodulesToBuildList', buildFile) + + //include statically called sub programs when the main program changes + + if (addSubmodulesToBuildList != null && addSubmodulesToBuildList.toBoolean()) { + // Call addLinkDependencies to append link dependencies to buildSet + if (props.verbose) println "** Perform analysis to add statically called sub modules to build list for ${buildFile}." + buildLinkSet = addLinkDependencies(buildFile) + } + } + if (buildLinkSet !=null) { + buildSet.addAll(buildLinkSet) + } + // Perform impact analysis for property changes if (props.impactBuildOnBuildPropertyChanges && props.impactBuildOnBuildPropertyChanges.toBoolean()){ if (props.verbose) println "*** Perform impacted analysis for property changes." @@ -169,7 +185,7 @@ def createImpactBuildList() { } } - + return [buildSet, changedFiles, deletedFiles, renamedFiles, changedBuildProperties] } @@ -786,7 +802,7 @@ def boolean shouldCalculateImpacts(String changedFile){ if (onskipImpactCalculationList) return false // return false if the changed file is a generated test case program but testing is disabled - if (buildUtils.isGeneratedzUnitTestCaseProgram(changedFile) && !(props.runzTests && props.runzTests.toBoolean())) { + if (buildUtils.isGeneratedTazTestCaseProgram(changedFile) && !(props.runzTests && props.runzTests.toBoolean())) { return false } @@ -867,4 +883,49 @@ def sortFileList(list) { */ def isMappedAsZUnitConfigFile(String file) { return (dependencyScannerUtils.getScanner(file).getClass() == com.ibm.dbb.dependency.ZUnitConfigScanner) +} + +/* + * addLinkDependencies - + * method to identify all statically called sub module programs when the main program changes + * + * @return list of statically called sub modules + * + */ +def addLinkDependencies(buildFile) { + Set buildLinkSet = new HashSet() + MetadataStore metadataStore = MetadataStoreFactory.getMetadataStore() + def logicalFile = buildUtils.relativizePath(buildFile) + def logicalFiles = metadataStore.getCollection(props.applicationOutputsCollectionName).getLogicalFile(logicalFile) + if (logicalFiles) { + // Check if any logical files are found + // List all link dependencies for every main program that changes from the output collection. + // This will return only the program name and not the absolute path + logicalFiles.each { logicalFileRecord -> + def dependencies = logicalFileRecord.getLogicalDependencies() + + dependencies.each { logicalDep -> + if (logicalDep.getCategory() == "LINK") { + def linkDepName = logicalDep.getLname() + def linkDepLogicalFile = metadataStore.getCollection(props.applicationCollectionName).getLogicalFiles(linkDepName) + + // Get the logical path for all the link dependencies returned + linkDepLogicalFile.each { filePath -> + // Link Dependency Files to be added + def linkDepFile = filePath.getFile() + + if (linkDepFile != logicalFile) { + if (ScriptMappings.getScriptName(linkDepFile)) { + buildLinkSet.add(linkDepFile) + if (props.verbose) println "** $linkDepFile has a link dependency to $logicalFile. Adding to build list" + } + } + } + } + } + } + } + + + return buildLinkSet } \ No newline at end of file diff --git a/utilities/README.md b/utilities/README.md index a6c045229..ac3aa5a61 100644 --- a/utilities/README.md +++ b/utilities/README.md @@ -61,6 +61,6 @@ groovyz /u/ibmuser/zAppBuild/utilities/DatasetValidationUtilities.groovy -d /var ** The dataset DEBUG.V14R1.SEQAMOD referenced for property SEQAMOD was found. ** The dataset DBC0CFG.SDSNEXIT referenced for property SDSNEXIT was found. ** The dataset IMS.V15R1.SDFSRESL referenced for property SDFSRESL was found. -** The dataset RATCFG.ZUNIT.SBZUSAMP referenced for property SBZUSAMP was found. +** The dataset RATCFG.ZUNIT.SEQASAMP referenced for property SEQASAMP was found. ** The dataset CEE.SCEELKED referenced for property SCEELKED was found. ``` diff --git a/version.properties b/version.properties index b5116936a..3ac7fcaab 100644 --- a/version.properties +++ b/version.properties @@ -2,7 +2,7 @@ # delivered via the public github repository # https://github.com/IBM/dbb-zappbuild. # Helps to understand from which version a copy was taken. -zappbuild_baseVersion=3.9.0 +zappbuild_baseVersion=3.10.0 # use this property to add you own version suffix to indicate # contributed enhancements and your own revisions