Better builds + 9.0.0.3 + IHS shutdown fix #22

trajano · 2017-05-23T15:00:25Z

Addresses #20

The code is written out, but mostly commented out to use static routing until WASdev/ci.docker.ibm-http-server#22 is merged

davidcurrie · 2017-05-24T20:48:09Z

Many thanks for the PR. Firstly, we'd need you to signoff on the commits to indicate acceptance of the DCO. I'd happily take the v9.0.0.3 commit but I'm going to need help understanding the changes that you have made to the build process. For example, why does ilan/Dockerfile now contain a shebang? And the approach that you have taken seems to layer in the entire .tgz file before unpacking it which is going to create excessively large images. What are you trying to achieve with your better builds?

trajano · 2017-05-24T20:56:24Z

So how do I do this acceptance of the DCO?

trajano · 2017-05-24T21:03:57Z

If you can get the 9.0.0.3 deployed soon that would be awesome because it is blocking my builds for WebSphere Liberty dynamicRouting.

trajano · 2017-05-24T21:04:15Z

I added more details about the changes on the commit message.

trajano · 2017-05-24T20:57:42Z

ilan/build_all

@@ -25,4 +25,4 @@ while read line; do
  if [[ $line == \#* ]]; then continue; fi
  version=$(cut -d, -f1 <<< $line)
 ./build $version $1 $2
-done < versions.csv
+done < im/versions.csv


this is not right will fix this first

trajano · 2017-05-24T21:05:02Z

ilan/install_ihs.sh

@@ -79,4 +76,3 @@ function install_version() {


 install_version
-tar -zcf /host/ihs${VERSION}.tar.gz /opt/IBM/HTTPServer /opt/IBM/WebSphere/Plugins /opt/IBM/WebSphere/Toolbox


This was moved to Dockerfile

@davidcurrie The installation-manager image is transient anyway it does not get pushed anywhere. But I did make most of these changes so I can do a build on Windows #20

trajano · 2017-05-24T21:05:57Z

ilan/im/Dockerfile

@@ -1,31 +0,0 @@
-#!/bin/bash


A new Dockerfile gets generated per version.

trajano · 2017-05-24T21:08:09Z

ilan/build

@@ -21,15 +21,13 @@ if [ $# != 3 ]; then
  exit 1
 fi

-docker build -t installation-manager im || exit $?
-docker run --rm -v $(pwd):/host installation-manager /host/install_ihs $1 $2 $3 || exit $?
+docker build -t installation-manager . --build-arg VERSION=$1 --build-arg IBM_ID=$2  --build-arg IBM_PASSWORD=$3|| exit $?


install_ihs.sh is called within docker during image build. The images are "cached up to the point of line 47. so there's no need to concern about installing Installation Manager again.

trajano · 2017-05-24T21:12:53Z

ilan/Dockerfile

-# limitations under the License.                                           #
-#                                                                          #
-############################################################################
+#!/bin/bash


This is weird I'll remove this

Moved the logic of building the images into docker. Removed using any absolute paths on the build system (i.e. $(pwd) which does not work correctly on Windows) Added .gitattributes to ensure line endings for files being used in the Docker build containers are correct. Please compare with whitespace checks turned off. In order to support builds within the container bsdtar needs to be used rather than the regular tar due to docker/hub-feedback#727 The tar server was also removed in favor of copying from the container to the "im" folder and imported back using Dockerfile for the final image. The step where the files are tarred in install_ihs was removed and put into docker. This makes it easier to debug issues with tar

davidcurrie · 2017-05-24T21:14:19Z

Thanks for the updates. I'll see what we can do about getting a 9.0.0.3 image up tomorrow. I can't make any promises on reviewing the other part of the PR until next week though (heading in to a long weekend here in the UK). You need to specify the -s option on git commit to sign-off on the commits and indicate acceptance of the DCO.

No need for traps and shutdown hooks, simply run IHS in foreground and it will handle the shutdowns properly. Fixes WASdev#23 Signed-off-by: Archimedes Trajano <[email protected]>

trajano · 2017-05-24T21:16:56Z

NP this is a big change anyway. But as long as the 9.0.0.3 gets built tomorrow I'm okay. This was more to help others who needed to get dynamicRouting to work and needed to build on a Windows machine.

trajano · 2017-05-24T21:18:55Z

@davidcurrie the installation-manager docker image is transient (i.e. it does not get pushed anywhere) so even if it had the tgz file it will not be relevant later. Unless you realize that there was a problem in the second part of the build process which is to assemble the image.

However, now that I think about it I should change the order of one of the lines.

trajano · 2017-05-24T21:22:35Z

OK that last change will allow you to fix the Dockerfile.template image to do any other fixes after the tgz is created. So if you make changes to ihsstart which I did you don't have to wait as long because the tgz is still there.

davidcurrie · 2017-05-25T14:49:04Z

ilan/Dockerfile.template

+RUN export tar='bsdtar'
+
+COPY ihsstart.sh /work/
+COPY im/ihs@[email protected] /ihs.tar.gz


This step means that you end up with the .tar.gz in a layer in the final image even though it is hidden by the layer above. That's why the current approach serves up the files so that they can be pulled in, unpacked and then deleted all as part of the same layer.

Hmm you're right I should have an rm after that step to get rid of the ihs.tar.gz file.

Actually that's because I don't use --squash on the docker build stage https://docs.docker.com/engine/reference/commandline/build/#squash-an-images-layers---squash-experimental-only it's still experimental but the end users should only get the final image rather than the whole history no?

trajano · 2017-05-25T15:03:30Z

ilan/build


 # Build image from hosted tar file
 echo "Building image"
-docker build -t ibm-http-server:$1 --build-arg TAR_URL=$tar_url . || exit $?
-docker rm -f tar_server
+docker build -t ibm-http-server:$1 -f im/DockerFile.$1 . || exit $?


I could add --squash here to get rid of the intermediate images but the feature is still experimental in Docker 1.13 so I opted not to for now. The end result for the end users will not have the ihs.tar.gz file regardless.

Do it after the tar is complete. That way if there are any problems in the Dockerfile.template it will not require a rebuild Signed-off-by: Archimedes Trajano <[email protected]>

Added v9.0.0.3

194ea09

fixes WASdev#19

trajano added a commit to trajano/jee that referenced this pull request May 23, 2017

Provided support to get ready for dynamicRouting

0d76683

The code is written out, but mostly commented out to use static routing until WASdev/ci.docker.ibm-http-server#22 is merged

trajano force-pushed the better-builds branch from 4e42c45 to 9b5555f Compare May 24, 2017 21:03

trajano commented May 24, 2017

View reviewed changes

trajano changed the title ~~Better builds + 9.0.0.3~~ Better builds + 9.0.0.3 + IHS shutdown fix May 24, 2017

trajano commented May 24, 2017

View reviewed changes

trajano force-pushed the better-builds branch from 9e05a98 to 8c210a2 Compare May 24, 2017 21:13

Simplified ihsstart.sh

e946112

No need for traps and shutdown hooks, simply run IHS in foreground and it will handle the shutdowns properly. Fixes WASdev#23 Signed-off-by: Archimedes Trajano <[email protected]>

trajano force-pushed the better-builds branch from 8c210a2 to e946112 Compare May 24, 2017 21:15

davidcurrie reviewed May 25, 2017

View reviewed changes

trajano commented May 25, 2017

View reviewed changes

trajano mentioned this pull request May 26, 2017

The provider for keystore type 'IBMCMSKS' is not available. #24

Closed

Moved the order of the Dockerfile.template

83a8bca

Do it after the tar is complete. That way if there are any problems in the Dockerfile.template it will not require a rebuild Signed-off-by: Archimedes Trajano <[email protected]>

trajano force-pushed the better-builds branch from 55ea375 to 83a8bca Compare June 6, 2017 15:34

		@@ -79,4 +76,3 @@ function install_version() {


		install_version
		tar -zcf /host/ihs${VERSION}.tar.gz /opt/IBM/HTTPServer /opt/IBM/WebSphere/Plugins /opt/IBM/WebSphere/Toolbox

Better builds + 9.0.0.3 + IHS shutdown fix #22

Are you sure you want to change the base?

Better builds + 9.0.0.3 + IHS shutdown fix #22

Uh oh!

Conversation

trajano commented May 23, 2017

Uh oh!

davidcurrie commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davidcurrie commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

trajano commented May 24, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!