AWS ElasticSearch endpoint

[physicalattraction]

I have the following configuration in my Django:

ELASTICSEARCH_ENDPOINT = 'https://search-fashionexchange-{uuid}.eu-central-1.es.amazonaws.com'.format(
    uuid=settings.GET_SECRET('ELASTICSEARCH_ENDPOINT_UUID'))
LOGGING = {
    ...
    'handlers': {
        'elasticsearch': {
            'level': 'INFO',
            'class': 'cmreslogging.handlers.CMRESHandler',
            'hosts': [{'host': ELASTICSEARCH_ENDPOINT, 'port': 80}],
            'es_index_name': 'fashionexchange',
            'es_additional_fields': {'app': settings.SERVICE_NAME, 'environment': settings.ENVIRONMENT},
            'auth_type': CMRESHandler.AuthType.NO_AUTH,
            'use_ssl': False,
        },
    }
}

When logging something, I get the following error:

Traceback (most recent call last):
  File "/webapps/MessageCenter/lib/python3.5/site-packages/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/webapps/MessageCenter/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/webapps/MessageCenter/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 271, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='https', port=80): Max retries exceeded with url: //search-fashionexchange-7jsibto6vo6xwupirk4sboisqq.eu-central-1.es.amazonaws.com:80/_bulk (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f514c08ed68>: Failed to establish a new connection: [Errno -2] Name or service not known',))

It seems like it cannot recognize the endpoint when I specify the protocol https. If I don't specify the protocol, it automatically prepends http in front of it.

[physicalattraction]

I guess I just have to use 'use_ssl': True and not specify the protocol here.

Another point I found: in the README, the AWS signing is already explained, but also in the README I read the auth options are: NO_AUTH, BASIC_AUTH, KERBEROS_AUTH. When I install the library through pip, I also don't see AWS_SIGNED_AUTH yet. Is there a roadmap when this will be available (1.0.1)?

[physicalattraction]

When installing it from pip from scratch, I do get the latest version. I guess pip just thought it already had version 1.0.0 and didn't update it anymore.

[cmanaha]

Yep, the https vs use_ssl seems to be misleading. Something to add to the next version, I guess. I'll leave this issue open and take care of it as son as I get some time.

As for this handler there is no formal roadmap at the moment. I took a few collaborations on the project and added features as people were sending over their PR's.

I'm not entirely sure there enough scope for a formal roadmap. In fact, I had a catch up a few weeks ago with Rasmus; Seems APM agents and modules are heading in a similar direction. I'd be quite happy if eventually all this ends up in an open source module (APM python module) backed by elastic (curating and maintaining OS projects takes time!)... In the meanwhile I'll keep taking contributions and maintaining this package.