REST vs HTTP Api

[simplexx]

Hey there, thanks a lot for this great lib! I noticed that it uses the REST api. Would it theoretically be possibe to use the HTTP Api, which is only 1/3 the price?

[Ge0rg3]

Hey, I've actually never heard of this - sounds very interesting! Will look into it more over the next month, and hopefully get some implementation done if it seems possible. 👍

A quick Google search shows that it seems to come down the functionality -- HTTP APIs currently don't offer all of the same functionality as REST, so we will need to see whether all of the API Gateway-related calls are available.

[simplexx]

I've been playing around with this a bit yesterday, created an API in my account and made the API with the target site as integration: https://target-site.com/{proxy}. Then I also created a route: /{proxy+}

After this, my api URL redirected to target site, with the specified path, for example:

https://xxxxxx.execute-api.eu-west-1.amazonaws.com/testpath

redirected to

https://target-site.com/testpath

I also cloned this project and changed some code so instead of the json api, a http api is used. not that many changes required for this. However, was not able to make it fully working yet, and not sure I will, as I d not have any much experience with python and none with using amazon services programmatically.

The new http api uses the apigatewayv2.

I think most of the code can be reused, but the methods are different (names and return values).

I did not check yet if x forwarded for header can be faked.

[simplexx]

I will play around some more and see if I can find out more things. would certainly be awesome if this new api could be used, saving 2/3 of the price is a big deal imo!

[Ge0rg3]

Sounds very promising, thanks for looking into these 😄 Please could you move your changes over to a fork so I could take a look at the changes you made?

[simplexx]

I will, please give me a few days to play with it some more, and then I'll push what I have.

[simplexx]

I opened a pull request with a version that is using the http api. needs more testing.

[simplexx]

Currently running the script in production to save costs, and it works fine. However, some things that need testing:
-Just different sites
-Multiple regions (I use just one per instance)
-forwarded header
-?

[simplexx]

One of my instances just got blacklisted, so I checked the headers and it does not work as it should:
"HTTP_X_MY_X_FORWARDED_FOR":"85.64.xxx.xxx"
"HTTP_FORWARDED":"by=35.179.xxx.xxx;for=46.127.xxx.xxx;host=xxxxxtd42.execute-api.eu-west-2.amazonaws.com;proto=https",
"HTTP_VIA":"HTTP/1.1 AmazonAPIGateway"

The true IP is leaked unfortunately.

[simplexx]

Looks like we can not spoof the HTTP_FORWARDED header, because it is restricted, which makes the whole http api unusable.

[herissondev]

Hi @simplexx did you find a work around for this ? I would love to reduce my cost of 2/3 ^^

[simplexx]

Hi @aime-risson It's not possible unfortunately :(

[Ge0rg3]

Ahh too bad, thanks for looking onto this.

[herissondev]

What I have done is using lambdas as proxys. This ended up being much cheaper and much more efficient as lambdas change ips on every run too.