You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
As a Data Prepper user, I'd like to use custom index type in opensearch sink to set up index template when using vpc collection as sink.
2024-03-07T21:04:55,273 [log-pipeline-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager - Index template vpc-aoss-test-000003-index-template does not exist and should be created
2024-03-07T21:04:55,285 [log-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink with a retryable exception.
org.opensearch.client.opensearch._types.OpenSearchException: Request failed: [http_exception] server returned 401
at org.opensearch.client.transport.aws.AwsSdk2Transport.parseResponse(AwsSdk2Transport.java:494) ~[opensearch-java-2.8.1.jar:?]
at org.opensearch.client.transport.aws.AwsSdk2Transport.executeSync(AwsSdk2Transport.java:392) ~[opensearch-java-2.8.1.jar:?]
at org.opensearch.client.transport.aws.AwsSdk2Transport.performRequest(AwsSdk2Transport.java:192) ~[opensearch-java-2.8.1.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.index.ComposableTemplateAPIWrapper.putTemplate(ComposableTemplateAPIWrapper.java:34) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.index.ComposableIndexTemplateStrategy.createTemplate(ComposableIndexTemplateStrategy.java:47) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:258) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:234) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.setupIndex(AbstractIndexManager.java:224) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitializeInternal(OpenSearchSink.java:231) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitialize(OpenSearchSink.java:193) ~[opensearch-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.model.sink.AbstractSink.initialize(AbstractSink.java:52) ~[data-prepper-api-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.pipeline.Pipeline.isReady(Pipeline.java:200) ~[data-prepper-core-2.7.0-SNAPSHOT.jar:?]
at org.opensearch.dataprepper.pipeline.Pipeline.lambda$execute$2(Pipeline.java:252) ~[data-prepper-core-2.7.0-SNAPSHOT.jar:?]
Describe the solution you'd like
Support custom index type for private opensearch serverless collection as sink. This can probably be resolved by just moving network policy update code ahead of setting up index.
Describe alternatives you've considered (Optional)
N/A
Additional context
N/A
The text was updated successfully, but these errors were encountered:
kkondaka
changed the title
Support custom index type for private opensearch serverless collection as sink
[BUG] Support custom index type for private opensearch serverless collection as sink
Feb 27, 2024
Is your feature request related to a problem? Please describe.
As a Data Prepper user, I'd like to use custom index type in opensearch sink to set up index template when using vpc collection as sink.
Current issue with this setup is when opensearch sink initializes, it tries to set up index and index template before creating or updating serverless network policy, which results in 401 errors.
Trackback looks like this:
Describe the solution you'd like
Support custom index type for private opensearch serverless collection as sink. This can probably be resolved by just moving network policy update code ahead of setting up index.
Describe alternatives you've considered (Optional)
N/A
Additional context
N/A
The text was updated successfully, but these errors were encountered: