Decisioning studio docs update - adding PRo docs #11398
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| --- | ||
| page_order: 2 | ||
| nav_title: Decisioning Studio Pro | ||
| article_title: Setting up Decisioning Studio Pro agents | ||
| description: "Learn how to set up Decisioning Studio Pro agents to make 1:1 AI decisions that maximize your business metrics." | ||
| --- | ||
|
|
||
| # Setting up Decisioning Studio Pro agents | ||
|
|
||
| > Think of a Decisioning Studio agent like the brain of your Martech stack: using your rich customer data, the agent orchestrates optimal and personalized decisions at a 1:1 level to maximize a target metric you choose. | ||
|
|
||
| AI Decisioning Pro is more flexible and more powerful than AI Decisioning Go. Setting up a Pro agent requires collaboration with our AI Decisioning Services team, which guides and assists with agent design throughout to drive maximum value for your business. | ||
|
|
||
|  | ||
|
|
||
| To set up an AI Decisioning Agent and complete the AI Decisioning Loop, at a high level, you will: | ||
|
|
||
| #### 1. Design your agent | ||
|
|
||
| You will make important decisions such as what metric the agent should optimize for, as well as defining the universe of possible decisions the agent could make. | ||
|
|
||
| #### 2. Set customer context | ||
|
|
||
| You will connect rich first-party data sources that describe your customer for the Decisioning Agent to learn from. | ||
|
|
||
| #### 3. Configure orchestration | ||
|
|
||
| You will connect Braze or a non-Braze Marketing Automation Platform that will carry out the Agent's decisions into action | ||
|
|
||
| #### 4. Close the AI Decisioning Loop | ||
|
|
||
| Finally, you will close the AI Decisioning Loop by connecting data sources that tell the Agent how a customer reacts to its decisions. | ||
|
|
||
| Decisioning Studio Pro includes AI Decisioning Services support from Braze’s forward deployed data science team, which will help you design and configure your agent to maximize your business outcomes. | ||
|
|
||
| ## Best practices | ||
|
|
||
| A few best practices for setting up Decisioning Studio agents: | ||
|
|
||
| 1. The more information agents have about your customers, the better they will perform. | ||
| 2. The more diverse the set of actions the agent can take, the more it can personalize its strategy for each user. | ||
| 3. The fewer constraints on your agents, the better. Constraints should be designed to respect business rules while freeing agent-led experimentation as much as possible. | ||
|
|
||
| {% alert important %} | ||
| This guide explains the most common integration patterns. Information Security will still need to vet all connection points and Solutions Consultants will be available to assist with implementation. | ||
| {% endalert %} | ||
|
|
||
| ## Next steps | ||
|
|
||
| 1. [Design your agent]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_pro/design_your_agent/) | ||
| 2. [Set Customer Context]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_pro/set_customer_context/) | ||
| 3. [Configure Orchestration]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_pro/configure_orchestration/) | ||
| 4. [Close the Feedback Loop]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_pro/create_feedback_loop/) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,103 @@ | ||||||
| --- | ||||||
| page_order: 2.3 | ||||||
| nav_title: Configure Orchestration | ||||||
| article_title: Configure Orchestration | ||||||
| description: "Learn how to configure orchestration for Decisioning Studio Pro agents to enable personalized communications." | ||||||
| --- | ||||||
|
|
||||||
| # Configure Orchestration | ||||||
|
|
||||||
| Decisioning agents need some means to orchestrate communications once they have ingested customer data and personalized at a 1:1 level. | ||||||
|
|
||||||
| While Decisioning Studio integrates most smoothly with Braze's orchestration capabilities, Decisioning Studio also supports native (code-free) integrations with Salesforce Marketing Cloud (SFMC) and Klaviyo. Custom integrations can also be configured with any other customer engagement platform (CEP). | ||||||
|
|
||||||
| {% alert note %} | ||||||
| The Decisioning Studio team is currently building even more seamless integrations between Decisioning Studio and Braze orchestration. This documentation will be updated as these steps are simplified. | ||||||
| {% endalert %} | ||||||
|
|
||||||
| ## If Customer Engagement Platform is Braze (Best case scenario) | ||||||
|
|
||||||
| Follow these steps to integrate a Braze Decisioning Studio agent with Braze's orchestration capabilities (and Braze's services team will be able to help): | ||||||
|
|
||||||
| ### Step 1: Create an API key | ||||||
|
|
||||||
| Go to **Settings** > **API Keys**, then create a new key with the following permissions: | ||||||
|
|
||||||
| {% multi_lang_include decisioning_studio/api_key_permissions.md %} | ||||||
|
There was a problem hiding this comment. I can't figure out where to find the source of this table, so I'm going to make some notes about the API permissions here:
|
||||||
|
|
||||||
| ### Step 2: Set up API-triggered campaigns | ||||||
|
|
||||||
| Set up an API-triggered campaign for each base template with API trigger properties for all optimized dimensions. | ||||||
|
There was a problem hiding this comment. I think I noted this in your original google doc - but I'd be careful of using the terminology "base template" here, since "templates" are actually a thing in the Braze product, and they are NOT necessarily what we are referring to here (i.e., we don't want people to think that they need to create a template in Braze AND a campaign; you can use a template when creating a campaign, but you don't have to). I'd suggest using language like "base creative". |
||||||
|
|
||||||
| A base template is any template that the Decisioning Agent might use for orchestrating messages. A Decisioning Agent might have 1 base template and multiple, in which case choosing the right base template for each customer will be one of the decisions the agent personalizes. | ||||||
|
There was a problem hiding this comment. I think this should be "or", right?
Suggested change
It might be worth saying that this could result in one or more campaigns per channel. For channels where the creative can be vastly different and not all handled via dynamic content (like email), you will likely need one campaign per base creative. But for channels like SMS/Push, you really only need one campaign for the entire channel, because all of the content can be easily passed through the API call (you have these in the examples, but I'm just expanding here in case you want to add any additional detail - totally up to you - I can be a bit verbose when I write things, and maybe simpler is better for now). |
||||||
|
|
||||||
| ### Step 3: Configure re-eligibility | ||||||
|
|
||||||
| Ensure all API Triggered Campaigns allow users to become re-eligible within **15 minutes**. | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| {% alert note %} | ||||||
| While the Decisioning Studio agent will never send the same campaign more than once a day, you will want to have the ability to send the same campaigns multiple times in a day for testing purposes. | ||||||
| {% endalert %} | ||||||
|
|
||||||
| ### Step 4: Add dynamic placeholders | ||||||
|
|
||||||
| These will serve as dynamic placeholders for decisions that the Decisioning Studio agent is optimizing. | ||||||
|
|
||||||
| Here are some examples: | ||||||
|
|
||||||
| #### Example #1: Email Campaign | ||||||
|
|
||||||
| Suppose the Decisioning Studio agent is optimizing an email campaign. This might be configured like this: | ||||||
|
There was a problem hiding this comment. I'd suggesting tying this back to "base creatives" - since you need to create a campaign for each
Suggested change
|
||||||
|
|
||||||
|  | ||||||
|
|
||||||
| Supposing the agent is optimizing for choice of templates and Call to Action (CTA) message, then an API-triggered campaign should be created for each template, and the CTA section of one template might look like: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| #### Example #2: Push campaign | ||||||
|
There was a problem hiding this comment. Do we think it's clear enough to the reader that all of these examples could potentially exist in the same use case? E.g., you are optimizing a multichannel first purchase journey - and this is how the various messages in that journey are configured for DS There was a problem hiding this comment. We may want to remove this section for now – Stephen & EMs are working on some standardization here and it's probably best to wait until he's done so we can do this section right from the start. |
||||||
|
|
||||||
| Suppose a Decisioning Studio agent is optimizing the message of a Push campaign. This might be configured like this: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| Resulting in the following message: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| #### Example #3: SMS Campaign | ||||||
|
|
||||||
| Suppose that the Decisioning Studio agent is optimizing for fields in an SMS campaign. This might be configured like this: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| Resulting in the following message: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| ## If Customer Engagement Platform is SFMC or Klaviyo | ||||||
|
|
||||||
| Decisioning Studio also supports native integrations with SFMC and Klaviyo. For SFMC, for example, Decisioning Studio triggers API events into a journey with data required to populate dynamic elements. | ||||||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF given that we're providing an example for SFMC, should we give one for Klaviyo as well? |
||||||
|
|
||||||
| For detailed steps to configure an agent that uses either of these customer engagement platforms, follow the [SFMC instructions]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_go/configure_sfmc/) or [Klaviyo instructions]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_go/configure_klaviyo/) | ||||||
|
|
||||||
| ## If another Customer Engagement Platform | ||||||
|
|
||||||
| Decisioning Studio can integrate with any customer engagement platform. However, this may require some custom engineering work from your team, since Decisioning Studio cannot trigger communications directly. | ||||||
|
There was a problem hiding this comment. You might want to be a little more direct here and just say "This will require custom engineering work." |
||||||
|
|
||||||
| In this scenario, the agent will deliver a "recommendation file." This file contains rows for each customer, with columns that indicate all of the personalized decisions for that customer. | ||||||
|
There was a problem hiding this comment. In the image, it says "Decisions file" but here we say "recommendation file" - suggest aligning on what you want to call it (when I made that slide, I had shifted toward calling it a "decisions" file - because we are a "decisioning" product, so it made better sense. And also, because it sounds less optional than "recommendations") |
||||||
|
|
||||||
| For example, the following recommendation file: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
| Might be used to optimize an email campaign that looks like the following: | ||||||
|
|
||||||
|  | ||||||
|
|
||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| --- | ||
| page_order: 2.4 | ||
| nav_title: Close the AI Decisioning Loop | ||
| article_title: Close the AI Decisioning Loop | ||
| description: "Learn how to close the AI decisioning loop for Decisioning Studio Pro agents to enable self-learning." | ||
| --- | ||
|
|
||
| # Close the AI Decisioning Loop | ||
|
|
||
| > Finally, while all customer data is important for the agent (and for information on how to configure, refer back to [Set Customer Context]({{site.baseurl}}/user_guide/brazeai/decisioning_studio/decisioning_studio_pro/set_customer_context/)), the *most important data assets* are those that tell the agent what happened after customer engagement decisions were sent. | ||
|
|
||
| These assets create the feedback loop that allows the agent to learn. | ||
|
|
||
| There are three critical assets for creating the feedback loop: conversions (including any relevant financial data, such as revenue); engagement data; and orchestration data. There are special requirements for each of these data assets. | ||
|
|
||
| {% alert note %} | ||
| Finally, note that while information on all three categories—conversions, engagement, and activations—is required for an agent to be successful, if the agent is natively integrated with the customer engagement platform, such as Braze, there may not be additional configuration steps necessary, since these may be sent with the customer data. | ||
| {% endalert %} | ||
|
|
||
| ## Conversions Data | ||
|
|
||
| The conversion asset describes what happened to the customer after orchestration. For example, supposing an agent is optimizing on Net Present Value (NPV) for customers receiving optimized campaigns, the conversion asset might include a daily update of changes to NPV. | ||
|
|
||
| The conversion asset must meet the following requirements: | ||
|
|
||
| | Requirements | Why? | | ||
| |-------------|------| | ||
| | Each record contains a unique customer identifier that is consistent with all data assets | Decisioning Studio needs to be able track the individual customer journey from recommendation, through activation, to conversion. | | ||
| | Each record has an associated timestamp. | Understanding the time between communication and sequence of customer actions is extremely important for model training and metric calculation. | | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF I know we've used |
||
| | If using a non-binary (e.g., converted vs. unconverted) target metric, the target metric value associated with each conversion is provided with each conversion event. | Decisioning Studio uses the target metric value to generate training experiences to appropriately reward/penalize the model based on the outcomes of the recommended actions. | | ||
| | If conversions can be uniquely attributed to communications (e.g., coupon redemption), fields needed to match conversions to activations are provided. | If a conversion event can be tied to a particular communication, this allows for clean and precise attribution. Direct attribution provides the clearest signal to the model, but if is not possible, as if often the case, proximity-based attribution will be used. | | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF same here regarding |
||
| {: .reset-td-br-1 .reset-td-br-2 role="presentation"} | ||
|
|
||
| ## Engagement Data | ||
|
|
||
| The engagement asset describes what happened to the customer, including clicks, opens, and other impressions. Engagement data may be included in the conversion data or it may be separate. It plays a similar role as conversions data—telling the agent what happened after customer engagement. | ||
|
|
||
| The engagement asset must meet similar requirements to the conversion asset: | ||
|
|
||
| | Requirements | Why? | | ||
| |-------------|------| | ||
| | Each record contains a unique customer identifier that is consistent with all data assets | Decisioning Studio needs to be able track engagement events for each individual customer. | | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF shouldn't we refer to the target audience members as |
||
| | Each record has an associated timestamp. | Understanding the time between communication and sequence of customer actions is extremely important for model training and metric calculation. | | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF same as my previous comment regarding |
||
| | If clicks, opens or other engagement data can be uniquely attributed to communications (e.g., coupon redemption), fields needed to match conversions to activations are provided. | As with conversion data, if engagement can be tied to a particular communication, this allows for clean and precise attribution. Direct attribution provides the clearest signal to the model. | | ||
| {: .reset-td-br-1 .reset-td-br-2 role="presentation"} | ||
|
|
||
| ## Activations Asset | ||
|
|
||
| The activations asset tells the agent which communications were sent. This is often necessary depending on how orchestration is configured. If the agent orchestrates via a direct integration with Braze, SFMC, or Klaviyo, then the agent may be able to pull activation data directly. Note that engagement data and activations data are very commonly found in the same data asset. | ||
|
|
||
| The activation asset must meet the following requirements: | ||
|
|
||
| | Requirements | Why? | | ||
| |-------------|------| | ||
| | Each record contains a unique customer identifier that is consistent with all data assets | Decisioning Studio needs to be able track the individual customer journey from recommendation, through activation, to conversion. | | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF same as my previous comment regarding |
||
| | Each record has an associated timestamp. | Understanding the time between communication and sequence of customer actions is extremely important for model training and metric calculation. | | ||
| | Fields needed to match communication content to activation events are provided (e.g., event_id). | Correctly matching communication characteristics to sends is necessary for model attribution and training. | | ||
| {: .reset-td-br-1 .reset-td-br-2 role="presentation"} | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| --- | ||
| page_order: 2.1 | ||
| nav_title: Design your agent | ||
| article_title: Design your agent | ||
| description: "Learn how to design your Decisioning Studio Pro agent with the AI Decisioning Services team." | ||
| --- | ||
|
|
||
| # Design your agent | ||
|
|
||
| The first step of agent setup is working with our AI Decisioning Services team to design your agent. | ||
|
|
||
| This includes making the following decisions: | ||
|
|
||
| 1. **Success metric:** What will the agent maximize when personalizing customer engagement? (e.g., revenue, LTM) | ||
| 2. **Audience:** For whom will the Decisioning Studio agent make customer engagement decisions? | ||
| 3. **Experiment groups:** How should Decisioning Studio's randomized controlled trials be structured? | ||
| 4. **Dimensions:** What decisions should the agent personalize? These could include time of day, subject line, frequency, offers, and more (the possibilities are vast!) | ||
| 5. **Options:** What options does the agent have to work with? For example, what email templates or offers can it send, and to whom? | ||
| 6. **Constraints:** What decisions should the agent *never* make? | ||
|
|
||
| Each of these decisions has implications for how much incremental uplift the agent may be able to generate, and how quickly. Our AI Decisioning Services team will work with you to design an agent that generates maximum value while respecting all of your business rules. | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF shouldn't it be AI Expert Services team? |
||
|
|
||
|  | ||
|
|
||
| Once key design decisions are made, we can proceed to configuration. | ||
|
Contributor
There was a problem hiding this comment. @chriswickhamOF do we have a page on how the technical design process looks like? I think it would be useful to have it included here to provide our customers with a good understanding of what designing a Decisioning Studio use case looks like from a technical perspective |
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chriswickhamOF I would suggest rephrasing the "assist with implementation" part to emphasize that while we can advise on certain aspects related to integration architecture, ultimately the customer technical teams are the ones who know their environment best. Outside of advising, there are additional considerations on our end wrt additional configuration paths we can take, but any action is limited to subsystems and environments managed by Braze.