Copybooks in repository are not moving to PDS, Compilation is using syslib copybooks in cobol.properties concatenation

[APSDevp]

COBOL copybooks are taken from syslib concatenation in cobol.properties and not from COPYBOOK folder in the repository. Copybooks are moved to USS but not copied to PDS in our case we modified to User.copybook.

[dennis-behm]

@APSDevp, this does not look like a zAppBuild issue.

Can you please check, if the SearchPathDependencyResolver is resolving the local files? Also check your SearchPath configuration which is used by DBB to locate the copybooks.

In the verbose output, you should find something like this:

*** Resolution rules for MortgageApplication/cobol/epsnbrvl.cbl:
search:/u/dbehm/userBuild/?path=MortgageApplication/copybook/*.cpy
*** Physical dependencies for MortgageApplication/cobol/epsnbrvl.cbl:
{
  "lname": "EPSNBRPM",
  "library": "SYSLIB",
  "category": "COPY",
  "sourceDir": "/u/dbehm/userBuild/",
  "file": "MortgageApplication/copybook/epsnbrpm.cpy",
  "resolved": true,
  "excluded": false
}

[dennis-behm]

Converting this into a discussion

[APSDevp]

@dennis-behm

Thanks for your quick response! I have tried checking with verbose mode and my findings are

 1) Script is not using the SearchPathDependencyResolver it is using the deprecated resolver with the search path of sourceDir and Directory where the copybooks are present and it has used the default scanner, however, couldn't able to see the "file" : parameter where copybooks are stored as example you have shared 

*** Physical dependencies for {app'name_/cobol/{prog_name}.cbl
{excluded:false,lnanme:"{cpybookName},"library": "SYSLIB", "category": "COPY","resolved": false,} <-- likewise am getting.

 2) In cobol.groovy at present in this repository below piece of code is there

       // configure dependency resolution and create logical file	
String dependencySearch = props.getFileProperty('cobol_dependencySearch', buildFile)
SearchPathDependencyResolver dependencyResolver = new SearchPathDependencyResolver(dependencySearch)

   cobol.groovy which am using has different set of codes which uses the resolverUtils and  buildutils( deprecated one which my scripts invokes using)

 Could you please help me on this to resolve it? 

[dennis-behm]

@APSDevp, let's take a step back. The file attribute is only available in the output, when it is resolved! Otherwise it is an unresolved dependency.

There are two different APIs - the SearchPathDependencyResolver, which is available since DBB 1.1.2. Starting with DBB 2.0, this is the only available API.

The DependencyResolver is only available on DBB 1.x, and is marked as deprecated, and is replaced by SearchPathDependencyResolver.

The old DependencyResolver API, requires to configure the pseudo JSON -

dbb-zappbuild/samples/application-conf/application.properties

Lines 150 to 158 in 54abed2

	# Rule to locate Cobol copy books. This rule defaults to the local copybook folder
	# in the main application folder.
	# used in dependency resolution and impact analysis
	# ** deprecated **
	copybookRule = {"library": "SYSLIB", \
	"searchPath": [ \
	{"sourceDir": "${workspace}", "directory": "${application}/copybook"} \
	] \
	}

The new SearchPathDependencyResolver API is using the new schema for the search path -

dbb-zappbuild/samples/application-conf/application.properties

Lines 160 to 182 in 54abed2

	#
	# copybookSearch
	# searchPath to locate Cobol copybooks
	# used in dependency resolution and impact analysis
	#
	# Please be as specific as possible when configuring the searchPath.
	# Alternate configurations:
	#
	# dependency resolution from multiple repositories / multiple root folders:
	# copybookSearch = search:${workspace}/?path=**/copybook/*.cpy
	#
	# dependency resolution across all directories in build workspace, but filtering on the file extension cpy:
	# copybookSearch = search:${workspace}/?path=**/*.cpy
	# dependency resolution across all directories in build workspace, but filtering on the file extension cpy and cobcpy:
	# copybookSearch = search:${workspace}/?path=**/*.cpy;**/*.cobcpy
	#
	# dependency resolution in the application directory and a shared common copybook location:
	# copybookSearch = search:${workspace}/?path=${application}/copybook/*.cpy;/u/build/common/copybooks/*.cpy
	#
	# More samples can be found along with the syntax for the search path configurations at:
	# https://www.ibm.com/docs/en/dbb/1.1.0?topic=scripts-how-manage-build-dependencies#6-resolving-logical-build-dependencies-to-local-physical-files
	#
	copybookSearch = search:${workspace}/?path=${application}/copybook/*.cpy

Is your searchPathConfiguration correctly set up? And do you see the second line in the verbose output?

*** Resolution rules for MortgageApplication/cobol/epsnbrvl.cbl:
search:/u/dbehm/userBuild/?path=MortgageApplication/copybook/*.cpy

Can you share the section of your verbose output? If you cannot share it here, please open an official support ticket with IBM.

[APSDevp]

hi @dennis-behm

Yes able to see similar to this
*** Resolution rules for MortgageApplication/cobol/epsnbrvl.cbl:
search:/u/dbehm/userBuild/?path=MortgageApplication/copybook/*.cpy --> under this path there was a copybook folder with copybooks but still not copied to PDS

Next line is *** physical dependencies {application/COBOL/{progname}

{excluded:false,lnanme:"{cpybookName},"library": "SYSLIB", "category": "COPY","resolved": false,}

whether am missing any Properties?

[dennis-behm]

Hi,

With the provided information I can only guess which makes it difficult to help - without any log, or seeing your concrete configuration and checking your system. I would recommend that you open an IBM support ticket.

One last guess is that the filter is not formed correctly for the dependency resolution.

Please run an ls like below with your filter on USS:

DBEHM:/u/dbehm/userBuild: >ls -lisa MortgageApplication/**/*.cpy
17247   16 -rw-------   1 BPXROOT  TIVUSR       577 Jan 10  2023 MortgageApplication/copybook/epsmortf.cpy
17250   16 -rw-------   1 BPXROOT  TIVUSR       104 Mar 30 13:51 MortgageApplication/copybook/epsmtcom.cpy
17251   16 -rw-------   1 BPXROOT  TIVUSR       333 Mar 30 13:51 MortgageApplication/copybook/epsmtinp.cpy
 3639   16 -rw-------   1 BPXROOT  TIVUSR       399 Mar 30 13:51 MortgageApplication/copybook/epsmtout.cpy
17252   16 -rw-------   1 BPXROOT  TIVUSR       386 Mar 30 13:51 MortgageApplication/copybook/epsnbrpm.cpy
17253   16 -rw-------   1 BPXROOT  TIVUSR       534 Mar 30 13:51 MortgageApplication/copybook/epspdata.cpy
15535    2 -rw-------   1 BPXROOT  TIVUSR        32 May 10 15:27 MortgageApplication/copybook/samp.cpy

Do you use any ** patterns in the filter?

[APSDevp]

@dennis-behm

When I tried to like this "ls -lisa MortgageApplication/**/*.cpy" - able to see all the copybooks.

But when I tried the same with "ls -lisa MortgageApplication/copybook/*.cpy " it showed an invalid directory.

As you have guessed - the filter is not formed correctly.

Do I need to change -> copybookSearch = search:${workspace}/?path=${application}/copybook/.cpy to copybookSearch = search:${workspace}/?path=${application}/**/.cpy will it work ?

[dennis-behm]

@APSDevp , you need provide the correct path that your application is using which can be copybook but also any other name.

Don't know if this is a typo in the msg ?path=${application}/copybook/.cpy but it misses the * before .cpy

Also MortgageApplication/copybook/*.cpy works fine:

DBEHM:/u/dbehm/userBuild: >ls -lisa MortgageApplication/copybook/*.cpy
17247   16 -rw-------   1 BPXROOT  TIVUSR       577 Jan 10  2023 MortgageApplication/copybook/epsmortf.cpy
17250   16 -rw-------   1 BPXROOT  TIVUSR       104 Mar 30 13:51 MortgageApplication/copybook/epsmtcom.cpy
17251   16 -rw-------   1 BPXROOT  TIVUSR       333 Mar 30 13:51 MortgageApplication/copybook/epsmtinp.cpy
 3639   16 -rw-------   1 BPXROOT  TIVUSR       399 Mar 30 13:51 MortgageApplication/copybook/epsmtout.cpy
17252   16 -rw-------   1 BPXROOT  TIVUSR       386 Mar 30 13:51 MortgageApplication/copybook/epsnbrpm.cpy
17253   16 -rw-------   1 BPXROOT  TIVUSR       534 Mar 30 13:51 MortgageApplication/copybook/epspdata.cpy
15535    2 -rw-------   1 BPXROOT  TIVUSR        32 May 10 15:27 MortgageApplication/copybook/samp.cpy

[APSDevp]

@dennis-behm

The issue here is the directory structure is "COPYBOOK" but the script searching for "copybook", after changing it - now able to see the repository copybook in the PDS, thanks for your support - Have learned a lot in this journey.