diff --git a/content/blog/ai-slack-bot-to-chat-using-embedchain-and-pulumi-on-aws/index.md b/content/blog/ai-slack-bot-to-chat-using-embedchain-and-pulumi-on-aws/index.md index 44b62eee623f..376816f72a4b 100644 --- a/content/blog/ai-slack-bot-to-chat-using-embedchain-and-pulumi-on-aws/index.md +++ b/content/blog/ai-slack-bot-to-chat-using-embedchain-and-pulumi-on-aws/index.md @@ -1,6 +1,5 @@ --- -title: "Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS" -allow_long_title: true +title: "Build an AI Slack Bot on AWS Using Embedchain & Pulumi" authors: - tyler-mulligan tags: @@ -10,8 +9,9 @@ tags: - slack - chatbot - ai -meta_desc: "Create an AI Slack bot to chat with your data using Embedchain, Pulumi on AWS" +meta_desc: "Learn how to build an AI-powered Slack bot with Embedchain & Pulumi on AWS." date: 2024-03-18T17:21:02+01:00 +updated: 2025-03-20 meta_image: meta.png --- @@ -26,17 +26,17 @@ on AWS using [Pulumi](https://www.pulumi.com), a modern infrastructure as code ( Large Language Models (LLMs) are sophisticated AI models trained on vast amounts of text data, enabling them to generate human-like text responses. Retrieval-Augmented Generated (RAG) is a cutting-edge approach to chatbot development that combines the capabilities of LLMs with information retrieval techniques. RAG chatbots can generate responses based on both the input query and relevant information retrieved from a knowledge base, resulting in more contextually relevant and informative interactions. - + -## Embedchain +## What's an Embedchain? -Embedchain is an RAG framework powered by LangChain. It simplifies the process of creating RAG applications by providing high-level abstractions that reduce the complexity of AI technologies. With Embedchain, developers can focus on defining conversational flows and configuring the bot's behavior using YAML configuration files, rather than dealing with low-level implementation details. +Embedchain is a RAG framework powered by LangChain. It simplifies the process of creating RAG applications by providing high-level abstractions that reduce the complexity of AI technologies. With Embedchain, developers can focus on defining conversational flows and configuring the bot's behavior using YAML configuration files, rather than dealing with low-level implementation details. Embedchain supports configuration for various [data sources](https://docs.embedchain.ai/components/data-sources/overview), [LLMs](https://docs.embedchain.ai/components/llms), [vector databases](https://docs.embedchain.ai/components/vector-databases), [embedding models](https://docs.embedchain.ai/components/embedding-models), and [evaluation](https://docs.embedchain.ai/components/evaluation). ## Creating a Slack bot - + Start by cloning the project: @@ -159,7 +159,7 @@ The configuration module in this application normalizes retrieval of secrets fro Once our bot is ready, we'll deploy it on AWS using Pulumi. Pulumi allows us to define our cloud infrastructure using familiar programming languages like TypeScript or Python, making it easy to manage and scale our deployment. - + It's important to note that in this selected architecture, we're working within a few limitations. Lambda's only writable directory is `/tmp` and it's limited to 500MB, and we need to use a Docker container to overcome the 250MB filesize limitation of our application, which we'll be bundling our data with. @@ -203,6 +203,9 @@ make deploy DEPLOY_ENVIRONMENT=dev At this point, you should be able to use `/arti` to interact with your data via Slack. +You might also like: +{{< related-posts >}} + ## Conclusion In this blog post, we've explored the process of building an AI-powered Slack bot using Embedchain and deploying it on AWS with Pulumi. By harnessing the capabilities of LLMs and RAG chatbots, we can create intelligent conversational agents that enhance user interactions and provide valuable insights. With Embedchain's high-level abstractions and Pulumi's infrastructure as code platform, developers can streamline the development and deployment of AI-powered applications, unlocking new possibilities for innovation in the digital space. diff --git a/content/blog/deploy-ai-ml-vercel-app/index.md b/content/blog/deploy-ai-ml-vercel-app/index.md index afdc7e1c17e8..2bf7c1060b72 100644 --- a/content/blog/deploy-ai-ml-vercel-app/index.md +++ b/content/blog/deploy-ai-ml-vercel-app/index.md @@ -1,8 +1,9 @@ --- date: 2023-09-13 -title: "Deploying Your AI/ML Chatbot Frontend To Vercel Using Pulumi" +updated: 2025-03-20 +title: "Deploy an AI/ML Chatbot Frontend on Vercel with Pulumi" allow_long_title: true -meta_desc: "Learn how to effortlessly deploy your AI/ML chatbot frontend to Vercel using Pulumi and Python to take your AI DevOps skills to the next level" +meta_desc: "Deploy your AI chatbot frontend on Vercel using Pulumi and Python. Follow this step-by-step guide to streamline your AI DevOps workflow." meta_image: meta.png authors: - adora-nwodo @@ -16,7 +17,7 @@ tags: --- -The process of taking an idea and turning it into reality has been nothing short of extraordinary since we started innovating with Artificial Intelligence. With this technology, machines learn about and communicate with people, while also helping us in ways we never could have imagined only a few years ago. If you've been following along, you might recall our [previous article](https://www.pulumi.com/blog/mlops-the-ai-challenge-is-cloud-not-code/) where we used Python and Pulumi to a chatbot API (named katwalk) to the cloud. +The process of taking an idea and turning it into reality has been nothing short of extraordinary since we started innovating with Artificial Intelligence. With this technology, machines learn about and communicate with people, while also helping us in ways we never could have imagined only a few years ago. If you've been following along, you might recall that [the real AI challenge is cloud, not code](https://www.pulumi.com/blog/mlops-the-ai-challenge-is-cloud-not-code/) where we used Python and Pulumi to a chatbot API (named katwalk). But our AI adventure hasn't ended there; it's only just begun. @@ -25,7 +26,7 @@ But our AI adventure hasn't ended there; it's only just begun. Now, imagine this: A Vue.js chatbot frontend, seamlessly integrating human interaction and the AI chatbot API. This blog post is part of the series on AI Infrastructure as Python. We'd delve into the creation of a captivating Vue.js chatbot frontend, and with Pulumi, we'll deploy it to Vercel. > Figure 1. Architecture diagram - + The complexity of building end-to-end AI/ML applications rests not only in the algorithms and models, but also in the several layers of infrastructure required to bring them to life. Consider this: You have a Vue.js chatbot frontend, a chatbot API powered by LLaMA models, a Vercel cloud hosting environment for the frontend, and a Runpod environment for the API. But here's the catch: you have to weave all of these pieces together seamlessly. @@ -38,7 +39,7 @@ Before we get into the details of using Pulumi to deploy a Vue.js frontend app t - [Katwalk Frontend](https://github.com/pulumiverse/katwalk/tree/main/src/app/katwalk-frontend) is a demo Vue.js application that integrates with the [Katwalk Server](https://github.com/pulumiverse/katwalk) to produce a function web interface for you to communicate with your chatbot. - [Vercel](https://vercel.com/) is a modern web development cloud platform and hosting provider. It specialises in making web application deployment simple and efficient. -## Integrate Your Chatbot Service With a Frontend And Deploy It! +## How to Integrate Your Chatbot Frontend and Deploy to Vercel If you are ready to integrate your chatbot and deploy it to Vercel along with us, then let's double check a few requirements before we continue. While Pulumi supports many languages, we chose Python for this project, as Python is a familiar language in the AI industry and this is part of the *#PulumiMLOps* challenge. ### Requirements @@ -115,7 +116,7 @@ pulumi up Once your web app is running, you can visit the website at `https://{vcl_project_name}.vercel.app`, send prompts and see the response in real-time! > Figure 2. A prompt and response screenshot from our Katwalk Frontend - + ### 6. Tear down resources @@ -125,7 +126,7 @@ Now that you've tested out this code and you can see the chatbot in action, reme pulumi destroy ``` -## Explaining the Pulumi code +## Understanding the Pulumi Code for Vercel Deployment If you'd like to learn about what the Pulumi code is doing, then stick around. In this section, we'll dive into the Python Pulumi code which is responsible for deploying the Vue.js project to Vercel using Pulumi. Let's break down the code step by step to understand how it works. @@ -210,4 +211,4 @@ If you've joined us on this incredible journey to connect the dots between the A Our journey doesn't end here; it's merely a waypoint on the ever-evolving road of innovation. We invite you to share your thoughts, ideas, and suggestions for future installments of the Pulumi Python + MLOps series. Together, we'll continue pushing the boundaries of what's possible in the world of AI development. -Join us in the [Pulumi Community Slack](https://slack.pulumi.com) or in the [Pulumiverse Katwalk GitHub repository](https://github.com/pulumiverse/katwalk/) to decide what #aiChallenge we tackle next! \ No newline at end of file +Join us in the [Pulumi Community Slack](https://slack.pulumi.com) or in the [Pulumiverse Katwalk GitHub repository](https://github.com/pulumiverse/katwalk/) to decide what #aiChallenge we tackle next! diff --git a/content/blog/environments-secrets-configurations-management/index.md b/content/blog/environments-secrets-configurations-management/index.md index 5cbbaef80d0e..dfac43d3d77e 100644 --- a/content/blog/environments-secrets-configurations-management/index.md +++ b/content/blog/environments-secrets-configurations-management/index.md @@ -1,21 +1,16 @@ --- title: "Introducing Pulumi ESC: Easy and Secure Environments, Secrets and Configuration" allow_long_title: true - date: 2023-10-10T04:00:00-07:00 - -meta_desc: "Introducing Pulumi ESC: Hierarchical Environments, Secrets Management and Configuration-as-Code for the Cloud" - +updated: 2025-03-20 +meta_desc: "Manage cloud secrets, configuration, and environments effortlessly with Pulumi ESC. Try the new CLI for secure, automated workflows." meta_image: meta.png - authors: - luke-hoban - tags: - esc - secrets - platform-engineering - --- Today, we’re excited to introduce [**Pulumi ESC**](/product/esc/), a new product from Pulumi that manages and tames secrets and configuration complexity across all of your cloud infrastructure and application environments. Pulumi ESC introduces a new category of configuration-as-code product, motivated by our experience working with hundreds of Pulumi IaC customers to address their needs in managing secrets and configuration at scale within their Pulumi infrastructure and across other cloud applications and infrastructure projects. @@ -28,7 +23,7 @@ Pulumi ESC enables teams to aggregate secrets and configuration from many source For example, the Pulumi ESC CLI (`esc`) makes it possible to give your developers immediate, just-in-time authenticated and short-lived access to cloud credentials across any cloud provider with just a single command: `esc run aws-staging -- aws s3 ls`. - + Here are just a few of the exciting use cases we've seen for Pulumi ESC: @@ -40,12 +35,11 @@ Here are just a few of the exciting use cases we've seen for Pulumi ESC: Pulumi ESC is available today in preview via the new `esc` CLI, as part of Pulumi Cloud, via the Pulumi Cloud REST API, and has direct integration with Pulumi IaC stack configuration and new `pulumi env` commands. It also supports dynamically pulling secrets and configuration from other sources of truth, including AWS OIDC, AWS Secrets Manager, Azure OIDC, Azure KeyVault, Google Cloud OIDC, Google Secrets Manager, HashiCorp Vault, and Pulumi IaC Stack References - with many more sources like 1Password coming soon. -[Get Started](/docs/pulumi-cloud/esc/) with Pulumi ESC today! - + -## A Taste of Pulumi ESC +## Pulumi ESC: How It Works & Key Features Pulumi ESC allows you to define _environments_, each of which specifies a collection of _secrets_ and _configuration_ values. Environments are specified as _code_, and each environment can import one or more other environments (for composition and inheritance), and can specify its own configuration values directly, or be pulled from another secrets provider. @@ -67,7 +61,7 @@ values: aws:region: us-west-2 ``` -## Why Pulumi ESC? +## Why Use Pulumi ESC for Cloud Secrets & Configuration? Working with Pulumi customers, we’ve seen the incredible benefits they get from managing IaC complexity in their code using Pulumi, and yet they frequently end up with a lot of complexity remaining at the boundary to their IaC code - at the configuration and secrets they need to get into Pulumi IaC. Users frequently show us configuration files that are hundreds of lines long, with tons of copy/paste both between Pulumi configuration and between other sources of truth and Pulumi configuration, and with more long lived static secrets being copied around than they would like - creating maintenance burden, security concerns, frequent drift, and operational risk. @@ -77,7 +71,7 @@ As we looked at solving these problems for Pulumi we quickly realized that none * __Duplication and Copy/Paste__: Secrets are duplicated in many places. Frequently coupled to application/system-specific configuration stores. * __Too Many Long-lived Static Secrets__: Long lived static credentials are over-used, exposing companies to significant security risk. Rotation is operationally challenging. Not all systems support direct integration with OIDC and other dynamic secrets provisioning systems. -While customers frequently had good solutions for pure secrets management (AWS Secrets Manager, HashiCorp Vault, etc.), none of these solutions extended into managing the complexities of actually getting these secrets as well as non-secret configuration into the collection of logical environments that define how they will be consumed in infrastructure and applications. Customers struggled with too many walled gardens for secrets management and configuration, and lack of any consistent set of tools and workflows to help manage the complexity of how secrets and configuration flow between these systems. +While customers frequently had good solutions for pure [secrets management](/what-is/what-is-secrets-management/) (AWS Secrets Manager, HashiCorp Vault, etc.), none of these solutions extended into managing the complexities of actually getting these secrets as well as non-secret configuration into the collection of logical environments that define how they will be consumed in infrastructure and applications. Customers struggled with too many walled gardens for secrets management and configuration, and lack of any consistent set of tools and workflows to help manage the complexity of how secrets and configuration flow between these systems. Pulumi ESC was born to address these problems and needs head on. It does so through a few core design principles: @@ -91,10 +85,12 @@ Pulumi ESC was born to address these problems and needs head on. It does so thr Pulumi ESC enables a new kind of solution for configuration management of modern cloud infrastructure. -## A Tour of Pulumi ESC +## Pulumi ESC Overview We can start by creating a few simple environments using Pulumi ESC. This can be done in the Pulumi Cloud console, using the new `esc` CLI, or using the new `pulumi env` sub-commands in the Pulumi CLI. Let’s take a look at these in action. +{{< related-posts >}} + ### Creating and Managing Environments, Secrets and Configuration First, in the Pulumi Cloud console, we can go to the Environments tab and create a new environment called `aws-staging`. @@ -235,7 +231,7 @@ environment: - shopping-service-staging ``` -If I run `pulumi preview` now on my Pulumi IaC project (in this case, a project which just outputs the configuration values), we will see that configuration values for `aws:region`, `numInstances`, and `dbPassword` are all available. I didn’t have to copy them into this stack, they were automatically pulled out of the environment in Pulumi ESC. And note that the secretness of the `dbPassword` configuration value flowed through into Pulumi IaC’s config system to ensure this value is encrypted in any place it is used within our infrastructure as code. +If I run `pulumi preview` now on my Pulumi IaC project (in this case, a project which just outputs the configuration values), we will see that configuration values for `aws:region`, `numInstances`, and `dbPassword` are all available. I didn’t have to copy them into this stack, they were automatically pulled out of the environment in Pulumi ESC. And note that the secretness of the `dbPassword` configuration value flowed through into Pulumi IaC’s config system to ensure this value is encrypted in any place it is used within our [infrastructure as code](/what-is/what-is-infrastructure-as-code/). ``` Previewing update (pulumi/staging) diff --git a/content/blog/pulumi-ai/index.md b/content/blog/pulumi-ai/index.md index 13bad02606da..ea7d3bb06adf 100644 --- a/content/blog/pulumi-ai/index.md +++ b/content/blog/pulumi-ai/index.md @@ -2,8 +2,9 @@ title: "Pulumi AI: The Fastest Way to Discover, Learn, and Build Infrastructure as Code" allow_long_title: true authors: ["luke-hoban"] -meta_desc: "Explore Pulumi AI, a powerful AI Assistant that simplifies cloud infrastructure creation using Large Language Models and GPT, saving developers time and effort" +meta_desc: "Discover Pulumi AI—your AI-powered assistant for building cloud infrastructure faster and smarter. Reduce complexity and deploy in minutes." date: "2023-04-27T10:00:00-08:00" +updated: 2025-03-20 meta_image: "meta.png" tags: - features @@ -12,7 +13,7 @@ tags: --- {{% notes type="info" %}} -There are new intelligent cloud management capabilities available in **Pulumi Copilot**. [Learn More](/blog/pulumi-copilot) +There are new intelligent cloud management capabilities available in [**Pulumi Copilot**](/blog/pulumi-copilot). {{% /notes %}} We recently released [Pulumi AI](/ai/), a purpose-built AI Assistant that can create cloud infrastructure using Pulumi. It builds on the power of Large Language Models (LLMs) and GPT to dramatically reduce the time it takes to discover, learn and use new cloud infrastructure APIs. diff --git a/content/blog/redshift-etl-with-pulumi-and-aws-glue/index.md b/content/blog/redshift-etl-with-pulumi-and-aws-glue/index.md index 82579b093f49..0d1603114fe6 100644 --- a/content/blog/redshift-etl-with-pulumi-and-aws-glue/index.md +++ b/content/blog/redshift-etl-with-pulumi-and-aws-glue/index.md @@ -1,7 +1,8 @@ --- -title: Building an ETL pipeline with Amazon Redshift and AWS Glue +title: How To Build An ETL Pipeline With Amazon Redshift & AWS Glue date: 2022-12-23 -meta_desc: Learn how to combine AWS Glue and Amazon Redshift to build a fully-automated ETL pipeline with Pulumi. +updated: 2025-03-20 +meta_desc: Build an automated ETL pipeline using AWS Glue & Redshift. Learn step-by-step with Pulumi to avoid duplicate data & streamline ingestion. meta_image: meta.png authors: - christian-nunciato @@ -27,7 +28,7 @@ There's a [lot more](https://aws.amazon.com/glue/features/) you can do with AWS But first ... -## Picking up where we left off +## Set Up Redshift & Glue Integration If you haven't already, I'd encourage you to [read through the previous post](/blog/building-a-data-warehouse-on-aws-with-redshift-and-pulumi/) to get up to speed on what we're building and why. In that post, the situation was such that some hypothetical application was generating "events" --- little bits of JSON, essentially --- and writing them periodically to a text file in S3, and as data scientists, we needed some way to gather up all of these events and load them into a Redshift cluster in order to analyze them later. @@ -282,7 +283,7 @@ When the deployment completes, you should see that eight new resources were crea Now it's time to start adding the Glue components to --- well, glue it all together. -## Extending the program +## Automating ETL Jobs in AWS Glue At a high-level, we'll need three components to complete our ETL pipeline-to-be: @@ -698,7 +699,7 @@ Duration: 7s You should see that the Glue components were created. Now let's load some sample data. -## Upload some data +## Load Sample Data into AWS Glue As before, we'll simulate this part, since we don't have an actual application generating real data for us. Run the following command to write a few JSON records to a file called `events-1.txt`: @@ -749,7 +750,7 @@ and sit back while the pipeline ingests this new data, blissfully ignoring the f  -## Tidying up +## Clean Up AWS Resources with Pulumi When you're finished experimenting, be sure to tear down the Redshift cluster and all other resources with a single `pulumi destroy`: @@ -785,7 +786,7 @@ Resources: - 16 deleted ``` -## See it in action +## Watch the AWS Glue ETL Pipeline in Action If you'd like to see a demonstration of what you've read in this blog post, then this [Modern Infrastructure video](https://www.youtube.com/playlist?list=PLyy8Vx2ZoWloyj3V5gXzPraiKStO2GGZw) covers the whole process: