Option to Disable AWS IMDS Priority

[ljstrnadiii]

Feature description

There are cases where the instance metadata service should not be prioritized over other authentication methods for EC2 instances and may fail with permission issues.

Additional context

Running the command below fails because the instance metadata service take priority over AWS_CONFIG_FILE and AWS_PROFILE and fails because it does not have permission. We can not grant the SA attached to the node permissions to this bucket. We instead use other mechanism that would normally have higher priority.

AWS_CONFIG_FILE=/.aws/config.yaml AWS_PROFILE=default CPL_CURL_VERBOSE=YES ogrinfo/vsis3/bucket/small_aoi.geojson

I have also tried to set CPL_AWS_AUTODETECT_EC2=NO but that did not work.

Perhaps I am missing something and disabling this authentication order is already possible. Note that when using boto3 I am able to access this data since IMDS is lower in priority that these AWS env vars.

[ljstrnadiii]

@rouault do you happen to know any way to bypass this issue. I am no C++ developer, but I see that setting CPL_AWS_AUTODETECT_EC2=NO would have no effect.

#ifdef __linux
    // Optimization on Linux to avoid the network request
    // See
    // http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html
    // Skip if either:
    // - CPL_AWS_AUTODETECT_EC2=NO
    // - CPL_AWS_CHECK_HYPERVISOR_UUID=NO (deprecated)

    if (!CPLTestBool(CPLGetConfigOption("CPL_AWS_AUTODETECT_EC2", "YES")))
    {
        return EC2InstanceCertainty::MAYBE;
    }
    else
    {
        const char *opt =
            CPLGetConfigOption("CPL_AWS_CHECK_HYPERVISOR_UUID", "");
        if (opt[0])
        {
            CPLDebug(AWS_DEBUG_KEY,
                     "CPL_AWS_CHECK_HYPERVISOR_UUID is deprecated. Use "
                     "CPL_AWS_AUTODETECT_EC2 instead");
            if (!CPLTestBool(opt))
            {
                return EC2InstanceCertainty::MAYBE;
            }
        }
    }

[ljstrnadiii]

This might be a false feature request. I believe I misunderstood the order of operations and our approach uses ~/.aws/config with a credential_process which does not seem supported at the moment. That fails, and ultimately hits IMDS. So, I may close and reopen with a request to support credential_process.

[ljstrnadiii]

#13239