From 4b9885dd4a4992a4dfe86a103c51d7d8448a7d42 Mon Sep 17 00:00:00 2001 From: Harold Hunt Date: Sun, 4 Feb 2024 17:29:26 -0500 Subject: [PATCH 1/3] Issue 108 - Specify LAMBDA_DISPATCH_RUNTIME --- extension/src/main.rs | 1 + fargate.template.yaml | 8 +++++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/extension/src/main.rs b/extension/src/main.rs index 0abcbaae..7671e1a4 100644 --- a/extension/src/main.rs +++ b/extension/src/main.rs @@ -78,6 +78,7 @@ fn main() -> anyhow::Result<()> { .unwrap(); runtime.block_on(async_main())?; } + // Default or `current_thread` runtime _ => { log::info!("Using current_thread runtime"); let runtime = tokio::runtime::Builder::new_current_thread() diff --git a/fargate.template.yaml b/fargate.template.yaml index 642fac5a..75aa9aac 100644 --- a/fargate.template.yaml +++ b/fargate.template.yaml @@ -534,7 +534,7 @@ Resources: Role: !GetAtt LambdaLBExecutionRole.Arn Code: ImageUri: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/lambda-dispatch-demo-app:latest - MemorySize: 256 + MemorySize: 192 Timeout: 60 PackageType: "Image" Architectures: @@ -544,7 +544,9 @@ Resources: SubnetIds: [!Ref PrivateSubnet1, !Ref PrivateSubnet2] Environment: Variables: - DOTNET_ThreadPool_UnfairSemaphoreSpinLimit: "6" + # No longer needed as we've moved to the Rust runtime + # DOTNET_ThreadPool_UnfairSemaphoreSpinLimit: "6" + LAMBDA_DISPATCH_RUNTIME: "current_thread" DirectLambda: Type: AWS::Lambda::Function @@ -553,7 +555,7 @@ Resources: Role: !GetAtt LambdaLBExecutionRole.Arn Code: ImageUri: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/lambda-dispatch-directlambda:latest - MemorySize: 256 + MemorySize: 128 Timeout: 60 PackageType: "Image" Architectures: From 21a302e0fb07e6574a87a540c5b3d0dd6d6f8b94 Mon Sep 17 00:00:00 2001 From: Harold Hunt Date: Sun, 4 Feb 2024 17:45:44 -0500 Subject: [PATCH 2/3] Issue 108 - Document settings --- README.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/README.md b/README.md index 8c3028cb..cc0c18d4 100644 --- a/README.md +++ b/README.md @@ -66,6 +66,31 @@ DotNet on Linux does suffer from a problem that is causing higher-than-necessary As of 2024-01-01, [fargate.template.yaml](fargate.template.yaml) contains an example deploy, [DockerfileLambda](DockerfileLambda) shows how to package the runtime with a Node.js lambda, and [DockerfileRouter](DockerfileRouter) packages up the router. +## Configuration - Router + +The router is configured with environment variables. + +| Name | Description | Default | +| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | +| `LAMBDA_DISPATCH_MaxWorkerThreads` | The maximum number of worker threads to use for processing requests. For best efficiency, set this to `1` and scale up router instances at ~50-70% CPU usage of 1 core. | default DotNet handling | +| `LAMBDA_DISPATCH_ChannelCount` | The number of channels that the Lambda extension should create back to the router | 20 | +| `LAMBDA_DISPATCH_MaxConcurrentCount` | The maximum number of concurrent requests that the Lambda extension should allow to be processed | 10 | +| `LAMBDA_DISPATCH_AllowInsecureControlChannel` | Opens a non-TLS HTTP2 port | false | +| `LAMBDA_DISPATCH_PreferredControlChannelScheme` | The scheme to use for the control channel
- `http` - Use HTTP
- `https` - Use HTTPS | `https` | +| `LAMBDA_DISPATCH_IncomingRequestHTTPPort` | The port to listen for incoming requests. This is the port contacted by the ALB. | 5001 | +| `LAMBDA_DISPATCH_IncomingRequestHTTPSPort` | The port to listen for incoming requests. This is the port contacted by the ALB. | 5002 | +| | +| `LAMBDA_DISPATCH_ControlChannelInsecureHTTP2Port` | The non-TLS port to listen for incoming control channel requests. This is the port contacted by the Lambda extension. | 5003 | +| `LAMBDA_DISPATCH_ControlChannelHTTP2Port` | The TLS port to listen for incoming control channel requests. This is the port contacted by the Lambda extension. | 5004 | + +## Configuration - Lambda Extension + +The extension is configured with environment variables. + +| Name | Description | Default | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------- | +| LAMBDA_DISPATCH_RUNTIME | The runtime to use for the Lambda dispatch
- `current_thread` - Configures Tokio to use only the current thread for async tasks
- `multi_thread` - Configures Tokio to start the multi thread runtime, with a default of 2 threads unless the thread count is specified by `TOKIO_WORKER_THREADS`
- `default_multi_thread` - Configures Tokio to start the multi thread runtime with the default behavior of creating as many threads as there are CPU cores | `current_thread` | + ## Development See [DEVELOPMENT.md](DEVELOPMENT.md) for details on how to build and run the project locally. From 86eaf0e3b9b9faee949dc050c2f8bf209b0bb1ec Mon Sep 17 00:00:00 2001 From: Harold Hunt Date: Sun, 4 Feb 2024 18:02:35 -0500 Subject: [PATCH 3/3] Add Docker image links --- DockerfileLambdaDemoApp | 1 + README.md | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/DockerfileLambdaDemoApp b/DockerfileLambdaDemoApp index 44692b3a..cdffe761 100644 --- a/DockerfileLambdaDemoApp +++ b/DockerfileLambdaDemoApp @@ -34,6 +34,7 @@ COPY --from=extension /lambda-dispatch /opt/extensions/lambda-dispatch # Copy the node.js app COPY --from=node-build-env /app/dist/ dist/ # This is only used for testing with docker-compose +# It is not necessary to copy this into an app deployed on Lambda COPY extension/startup.sh ./ # Set the entrypoint diff --git a/README.md b/README.md index cc0c18d4..e9728d69 100644 --- a/README.md +++ b/README.md @@ -58,15 +58,29 @@ Feedback is welcome and encouraged. Please open an issue for any questions, comm ## Project Implementation -The project was built with DotNet 8 and C# and generates a single binary, similar to Go/Rust, that can be used within a Lambda to connect to the AWS Lambda Runtime API to receive invocations. The structure is similar to the [AWS Lambda Web Adapter](https://github.com/awslabs/aws-lambda-web-adapter) in that the adapter starts the contained application via a `bootstrap.sh` shell script then connects to that application on port 3000, and checks a `/health` route (which can perform cold start logic). The adapter then connects to the Router over HTTP2 to pickup requests. +The project was initially built in DotNet 8 with C# for both the Router and the Lambda Extension, but the Lambda Extension was later rewritten in Rust using Hyper and the Tokio async runtime to resolve a high CPU usage issue. However, the high CPU usage issue was not resolved directly by the Rust rewrite; instead both the router and extension needed to be restricted to use only 1 worker thread to avoid the high CPU usage; the problem of too much CPU usage was common to the DotNet Router, DotNet Extension, and the Rust Extension. -DotNet on Linux does suffer from a problem that is causing higher-than-necessary CPU usage due to the way spin locks are being used by the thread pool to poll for new work. [High CPU Problem on DotNet on Linux](https://github.com/dotnet/runtime/issues/72153#issuecomment-1216363757) There is a workaround (setting the `DOTNET_ThreadPool_UnfairSemaphoreSpinLimit` env var to 0-6), but the CPU usage is still 2-3x higher than it should be. If this is not resolved in DotNet and if the project needs to exist for a long time, then the project may need to be rewritten in Go or Rust. +The structure of the extension is similar to the [AWS Lambda Web Adapter](https://github.com/awslabs/aws-lambda-web-adapter) in that the extension connects to that application on port 3000, and waits for aa `/health` route (which can perform cold start logic) to return a 200 before connecting back to the Router over HTTP2 to pickup requests. ## Installation / Setup -As of 2024-01-01, [fargate.template.yaml](fargate.template.yaml) contains an example deploy, [DockerfileLambda](DockerfileLambda) shows how to package the runtime with a Node.js lambda, and [DockerfileRouter](DockerfileRouter) packages up the router. +As of 2024-02-04, [fargate.template.yaml](fargate.template.yaml) contains an example deploy, [DockerfileLambdaDemoApp](DockerfileLambdaDemoApp) shows how to package the runtime with a Node.js lambda, and [DockerfileRouter](DockerfileRouter) packages up the router. -## Configuration - Router +### Docker Images + +The docker images are published to the AWS ECR Public Gallery: + +- [Lambda Dispatch Router](https://gallery.ecr.aws/pwrdrvr/lambda-dispatch-router) + - Latest: `public.ecr.aws/pwrdrvr/lambda-dispatch-router:main` + - Available for both ARM64 and AMD64 +- [Lambda Dispatch Extension](https://gallery.ecr.aws/pwrdrvr/lambda-dispatch-extension) + - Latest: `public.ecr.aws/pwrdrvr/lambda-dispatch-extension:main` + - Available for both ARM64 and AMD64 +- [Lambda Dispatch Demo App](https://gallery.ecr.aws/pwrdrvr/lambda-dispatch-demo-app) + - Latest: `public.ecr.aws/pwrdrvr/lambda-dispatch-demo-app:main` + - Available for both ARM64 and AMD64 + +### Configuration - Router The router is configured with environment variables. @@ -83,7 +97,7 @@ The router is configured with environment variables. | `LAMBDA_DISPATCH_ControlChannelInsecureHTTP2Port` | The non-TLS port to listen for incoming control channel requests. This is the port contacted by the Lambda extension. | 5003 | | `LAMBDA_DISPATCH_ControlChannelHTTP2Port` | The TLS port to listen for incoming control channel requests. This is the port contacted by the Lambda extension. | 5004 | -## Configuration - Lambda Extension +### Configuration - Lambda Extension The extension is configured with environment variables.