Updates log4j configurations to use the org.opensearch.dataprepper pa…

…ckage. Updates documentation to this package as well to help clarify expected outcomes for users. Contributes toward opensearch-project#344. (opensearch-project#1841)

Signed-off-by: David Venable <[email protected]>

data-prepper-core/src/main/java/org/opensearch/dataprepper/plugin/DefaultPluginFactory.java

@@ -47,7 +47,7 @@ public class DefaultPluginFactory implements PluginFactory {
 
         if(pluginProviders.isEmpty()) {
             throw new RuntimeException("Data Prepper requires at least one PluginProvider. " +
-                    "Your Data Prepper configuration may be missing the com.amazon.dataprepper.plugin.PluginProvider file.");
+                    "Your Data Prepper configuration may be missing the org.opensearch.dataprepper.plugin.PluginProvider file.");
         }
     }
 

data-prepper-core/src/test/java/org/opensearch/dataprepper/plugins/TestPlugin.java

@@ -12,7 +12,7 @@
 
 /**
  * Used for integration testing the plugin framework.
- * TODO: Move this into the com.amazon.dataprepper.plugin package once alternate packages are supported per #379.
+ * TODO: Move this into the org.opensearch.dataprepper.plugin package once alternate packages are supported per #379.
  */
 @DataPrepperPlugin(name = "test_plugin", pluginType = TestPluggableInterface.class, pluginConfigurationType = TestPluginConfiguration.class)
 public class TestPlugin implements TestPluggableInterface {

docs/logstash_migration_guide.md

@@ -37,5 +37,5 @@ You can find the converted `logstash.yaml` in the same directory where you place
 The following output in your terminal indicates Data Prepper is running correctly:
 
 ```
-INFO  com.amazon.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
+INFO  org.opensearch.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
 ```

docs/plugin_development.md

@@ -8,7 +8,7 @@ are created as Data Prepper plugins.
 Plugins are created as Java classes. They must conform to the following.
 
 * The class must be annotated with [`@DataPrepperPlugin`](../data-prepper-api/src/main/java/com/amazon/dataprepper/model/annotations/DataPrepperPlugin.java)
-* The class must be in the `com.amazon.dataprepper.plugins` package
+* The class must be in the `org.opensearch.dataprepper.plugins` package
 * The class must implement the required interface
 * The class must have a valid constructor (see below)
 

docs/simple_pipelines.md

@@ -51,18 +51,18 @@ docker run --name data-prepper -p 4900:4900 -v ${PWD}/pipelines.yaml:/usr/share/
 You should see log output and after a few seconds some UUIDs in the output. It should look something like the following:
 
 ```
-2021-09-30T20:19:44,147 [main] INFO  com.amazon.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
-2021-09-30T20:19:44,681 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:45,183 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:45,687 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:46,191 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:46,694 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:47,200 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:47,704 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:48,207 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:48,677 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:49,179 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-09-30T20:19:49,181 [simple-test-pipeline-processor-worker-1-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  simple-test-pipeline Worker: Processing 6 records from buffer
+2021-09-30T20:19:44,147 [main] INFO  org.opensearch.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
+2021-09-30T20:19:44,681 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:45,183 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:45,687 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:46,191 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:46,694 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:47,200 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:47,704 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:48,207 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:48,677 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:49,179 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-09-30T20:19:49,181 [simple-test-pipeline-processor-worker-1-thread-1] INFO  org.opensearch.dataprepper.pipeline.ProcessWorker -  simple-test-pipeline Worker: Processing 6 records from buffer
 07dc0d37-da2c-447e-a8df-64792095fb72
 5ac9b10a-1d21-4306-851a-6fb12f797010
 99040c79-e97b-4f1d-a70b-409286f2a671
@@ -126,8 +126,8 @@ docker run --name data-prepper -p 4900:4900 -v ${PWD}/pipelines.yaml:/usr/share/
 You will see output like the following.
 
 ```
-2021-10-01T09:58:19,907 [random-source-pool-0] INFO  com.amazon.dataprepper.plugins.source.RandomStringSource - Writing to buffer
-2021-10-01T09:58:19,908 [simple-sample-pipeline-processor-worker-1-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  simple-sample-pipeline Worker: Processing 6 records from buffer
+2021-10-01T09:58:19,907 [random-source-pool-0] INFO  org.opensearch.dataprepper.plugins.source.RandomStringSource - Writing to buffer
+2021-10-01T09:58:19,908 [simple-sample-pipeline-processor-worker-1-thread-1] INFO  org.opensearch.dataprepper.pipeline.ProcessWorker -  simple-sample-pipeline Worker: Processing 6 records from buffer
 B77D726E-2B92-458B-8C14-E5F8C403F1B0
 6B8FCC48-ED40-462C-8C43-AB218C1DA478
 3D0054FB-5D93-444F-A1AC-51827BD4FF37

examples/log-ingestion/README.md

@@ -61,7 +61,7 @@ docker run --name data-prepper -v /full/path/to/log_pipeline.yaml:/usr/share/dat
 If Data Prepper is running correctly, you should see something similar to the following line as the latest output in your terminal.
 
 ```
-INFO  com.amazon.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
+INFO  org.opensearch.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
 ```
 
 ### Apache Log Generator
@@ -122,7 +122,7 @@ fluent-bit  | [2021/10/30 17:16:39] [ info] [output:http:http.0] host.docker.int
 The following Data Prepper output indicates that Data Prepper is successfully processing logs from FluentBit
 
 ```
-2021-10-30T12:17:17,474 [log-pipeline-prepper-worker-1-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  log-pipeline Worker: Processing 2 records from buffer
+2021-10-30T12:17:17,474 [log-pipeline-prepper-worker-1-thread-1] INFO  org.opensearch.dataprepper.pipeline.ProcessWorker -  log-pipeline Worker: Processing 2 records from buffer
 ```
 
 Finally, head into OpenSearch Dashboards ([http://localhost:5601](http://localhost:5601)) to view your processed logs.

release/smoke-tests/README.md

@@ -70,8 +70,8 @@ docker-compose logs -f data-prepper
 The Data Prepper container should show events being processed:
 ```
 ...
-data-prepper_1         | 2021-12-03T19:57:30,488 [service-map-pipeline-prepper-worker-3-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  service-map-pipeline Worker: No records received from buffer
-data-prepper_1         | 2021-12-03T19:57:31,925 [grok-pipeline-prepper-worker-7-thread-1] INFO  com.amazon.dataprepper.pipeline.ProcessWorker -  grok-pipeline Worker: Processing 8 records from buffer
+data-prepper_1         | 2021-12-03T19:57:30,488 [service-map-pipeline-prepper-worker-3-thread-1] INFO  org.opensearch.dataprepper.pipeline.ProcessWorker -  service-map-pipeline Worker: No records received from buffer
+data-prepper_1         | 2021-12-03T19:57:31,925 [grok-pipeline-prepper-worker-7-thread-1] INFO  org.opensearch.dataprepper.pipeline.ProcessWorker -  grok-pipeline Worker: Processing 8 records from buffer
 ...
 ```
 ### 3. Confirm Open Search is running

shared-config/log4j.properties

@@ -20,6 +20,6 @@ log4j.appender.file.layout=org.apache.log4j.PatternLayout
 log4j.appender.file.layout.ConversionPattern=%d{ISO8601} [%t] %-5p %40C - %m%n
 
 #package specific levels
-log4j.logger.com.amazon.dataprepper.pipeline=INFO
-log4j.logger.com.amazon.dataprepper.parser=INFO
-log4j.logger.com.amazon.dataprepper.plugins=INFO
+log4j.logger.org.opensearch.dataprepper.pipeline=INFO
+log4j.logger.org.opensearch.dataprepper.parser=INFO
+log4j.logger.org.opensearch.dataprepper.plugins=INFO

shared-config/log4j2.properties

@@ -17,16 +17,16 @@ appender.console.layout.pattern = %d{ISO8601} [%t] %-5p %40C - %m%n
 rootLogger.level = warn
 rootLogger.appenderRef.stdout.ref = STDOUT
 
-logger.core.name = com.amazon.dataprepper
+logger.core.name = org.opensearch.dataprepper
 logger.core.level = info
 
-logger.pipeline.name = com.amazon.dataprepper.pipeline
+logger.pipeline.name = org.opensearch.dataprepper.pipeline
 logger.pipeline.level = info
 
-logger.parser.name = com.amazon.dataprepper.parser
+logger.parser.name = org.opensearch.dataprepper.parser
 logger.parser.level = info
 
-logger.plugins.name = com.amazon.dataprepper.plugins
+logger.plugins.name = org.opensearch.dataprepper.plugins
 logger.plugins.level = info
 
 logger.springframework.name = org.springframework