Strip mongodb server from Clarus in Docker

[hargathor]

In order to have a more "Docker friendly" type of container we will need to strip the CLARUS container from the mongodb database. One openjdk:8 and an external mongodb (which can be a other container).

Does this impact the code or the CLARUS proxy is already able to address an external database ?
The resolution of this issue have a very high impact on the upcomming performance testing.

[tchevallier]

Both options are supported by the CLARUS proxy (embedded mongodb and external mongodb), so yes it can work with an external mongodb.

[hargathor]

Nice ! And how do you activate/configure it ? You just have to set the environment variable EMBEDDED_MONGO_DB to the external mongodb endpoint ?

[tchevallier]

• for running a proxy with the embedded mongodb (i.e. in development mode), one need to add this option to the command line when running it -DEMBEDDED_MONGO_DB=localhost:27017
• however it is preferrable in the production mode to connect to an external mongodb instead.
• In any way the mongoDB client is set up in the data protection modules. At present, it is only the case for simple encryption, and I am not sure it is possible to configure it, i.e. change the host and port of the mongoDB instance to connect to (by default localhost:27017). We should probably propose an evolution to make it configurable.

[hargathor]

I've updated the Docker container in order to link it to a external mongodb and I end up with this stacktrace

Download PRODUCTION:Linux:B64 START
13:27:55.912 ERROR - prepare executable
java.io.IOException: Could not open inputStream for https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.1.tgz
        at de.flapdoodle.embed.process.store.Downloader.downloadInputStream(Downloader.java:131) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
        at de.flapdoodle.embed.process.store.Downloader.download(Downloader.java:69) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
        at de.flapdoodle.embed.process.store.ArtifactStore.checkDistribution(ArtifactStore.java:66) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
        at de.flapdoodle.embed.process.store.ExtractedArtifactStore.checkDistribution(ExtractedArtifactStore.java:60) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
        at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:56) [de.flapdoodle.embed.process-2.0.1.jar:na]
        at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49) [de.flapdoodle.embed.process-2.0.1.jar:na]
        at eu.clarussecure.proxy.protection.mongodb.EmbeddedMongoDB.start(EmbeddedMongoDB.java:77) [proxy-main-1.0.1.jar:1.0.1]
        at eu.clarussecure.proxy.Proxy.main(Proxy.java:209) [proxy-main-1.0.1.jar:1.0.1]
Caused by: javax.net.ssl.SSLKeyException: RSA premaster secret error
        at sun.security.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:87) ~[na:1.8.0_131]
        at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:912) ~[na:1.8.0_131]
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:348) ~[na:1.8.0_131]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) ~[na:1.8.0_131]
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961) ~[na:1.8.0_131]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) ~[na:1.8.0_131]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) ~[na:1.8.0_131]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) ~[na:1.8.0_131]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) ~[na:1.8.0_131]
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) ~[na:1.8.0_131]
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) ~[na:1.8.0_131]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546) ~[na:1.8.0_131]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474) ~[na:1.8.0_131]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.8.0_131]
        at de.flapdoodle.embed.process.store.Downloader.downloadInputStream(Downloader.java:127) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
        ... 7 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: SunTls12RsaPremasterSecret KeyGenerator not available
        at javax.crypto.KeyGenerator.<init>(KeyGenerator.java:169) ~[na:1.8.0_131]
        at javax.crypto.KeyGenerator.getInstance(KeyGenerator.java:223) ~[na:1.8.0_131]
        at sun.security.ssl.JsseJce.getKeyGenerator(JsseJce.java:251) ~[na:1.8.0_131]
        at sun.security.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:78) ~[na:1.8.0_131]
        ... 21 common frames omitted
Exception in thread "main" de.flapdoodle.embed.process.exceptions.DistributionException: java.io.IOException: Could not open inputStream for https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.1.tgz
        at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:69)
        at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
        at eu.clarussecure.proxy.protection.mongodb.EmbeddedMongoDB.start(EmbeddedMongoDB.java:77)
        at eu.clarussecure.proxy.Proxy.main(Proxy.java:209)
Caused by: java.io.IOException: Could not open inputStream for https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.1.tgz
        at de.flapdoodle.embed.process.store.Downloader.downloadInputStream(Downloader.java:131)
        at de.flapdoodle.embed.process.store.Downloader.download(Downloader.java:69)
        at de.flapdoodle.embed.process.store.ArtifactStore.checkDistribution(ArtifactStore.java:66)
        at de.flapdoodle.embed.process.store.ExtractedArtifactStore.checkDistribution(ExtractedArtifactStore.java:60)
        at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:56)
        ... 3 more
Caused by: javax.net.ssl.SSLKeyException: RSA premaster secret error
        at sun.security.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:87)
        at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:912)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:348)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
        at de.flapdoodle.embed.process.store.Downloader.downloadInputStream(Downloader.java:127)
        ... 7 more
Caused by: java.security.NoSuchAlgorithmException: SunTls12RsaPremasterSecret KeyGenerator not available
        at javax.crypto.KeyGenerator.<init>(KeyGenerator.java:169)
        at javax.crypto.KeyGenerator.getInstance(KeyGenerator.java:223)
        at sun.security.ssl.JsseJce.getKeyGenerator(JsseJce.java:251)
        at sun.security.ssl.RSAClientKeyExchange.<init>(RSAClientKeyExchange.java:78)
        ... 21 more

Any idea ? It seems to failed on a key generation of some sort.

[dieriver]

I know what it's happening! I told Thierry about this "bug", but it's only about a misconfiguration when starting the proxy. I will contact you through email.

[hargathor]

Ok. I've created the branch issues/12 so you can re-create it on your side if you need it. Here is the commands I've ran

git clone https://github.com/clarus-proxy/proxy.git
cd proxy
git checkout issues/12
cd docker
docker build -t clarus/proxy:1.1.0-SNAPSHOT .
docker run -d -p 27017:27017 --name mongodb mongo
docker run --rm -P -e SECURITY_POLICY="/etc/clarus/security-policy/boreholes_3857_simple_encryption.xml" -v /home/ubuntu/CLARUS/proxy/main/src/test/resources/:/etc/clarus/security-policy/ -e EMBEDDED_MONGO_DB="mongodb:27017" --link mongodb clarus/proxy:1.1.0-SNAPSHOT

[tchevallier]

What is the use of EMBEDDED_MONGO_DB option in the command line ? if you want to use an external mongodb, then it is not necessary.
Actually the de.flapdoodle.embed.process is the embedded mongodb we use for unit tests (and unit tests only)
see https://github.com/flapdoodle-oss/de.flapdoodle.embed.process