[Observability AI Assistant] Create first Starter prompts #178907

CoenWarmer · 2024-03-18T20:27:59Z

Summary

This introduces starter prompts for users to get the conversation going.

Allows Kibana app developers to create starter prompts at any time using the setScreenContext method.

Up to 4 starter prompts will be shown to the user. Starter Prompts added by an app take precedence over the default starter prompts.

Screen.Recording.2024-04-05.at.15.15.48.mov

Default Starter Prompts (visible in all Observability apps when no additional prompts have been registered by app / page)

Prompt
`Give me examples of questions I can ask here.`
`Can you explain this page?`
`Do I have any alerts?`
`What are SLOs?`

Starter Prompts added to different specific Observability apps

App	Where	Condition	Prompt
Observability	Overview	`HasDataProvider` returns no data for an app	`Why don't I see any data for the {appsWithoutData} sections?`
Observability	Alerts	none	`Can you explain the rule types that are available?`
Observability	Rules	none	`Can you explain the rule types that are available?`
APM	All	No data returned	`Why don't I see any data?`
Infra	All	No data returned	`Why don't I see any data?`
Metrics	All	No data returned	`Why don't I see any data?`
Synthetics	All	No data returned	`Why don't I see any monitors?`
UX	All	No data returned	`Why don't I see any data?`

Guidance for Kibana engineers

The team owning a plugin can add the following code to add a starter prompt to the Assistant:

plugins.observabilityAiAssistant.service.setScreenContext(
  {
    starterPrompts: [
      {
        title: i18n.translate('xpack.app.foo.bar', defaultMessage: 'Explain' }), 
        prompt: i18n.translate('xpack.app.foo.baz', defaultMessage: 'How does feature X work?' }),
        icon: 'sparkles' // 'EuiIconType' 
      }
    ]
  }
)

You can use the optional screenDescription and data keys in setScreenContext to pass along additional information to the LLM which may be beneficial in answering the starter prompt that you configure. For example:

setScreenContext(
  {
    screenDescription: 'The user is looking at a no data page.' // Doesn't need to translated as this is passed to the LLM which doesn't need a translated string.
    data: [
      {
        name: 'config',
        description: 'The index configuration of the app',
        value: config,
      },
    ],
  }
)

As a rule of thumb, the more generic or 'high level' the starter prompt is, the higher in the React app tree it should be added.

More specific starter prompts that are relevant for pages (or even sections inside pages) should be added by adding setScreenContext further down in the React app tree, for instance in page components or even more specific. Be aware that only 4 starter prompts will be displayed so if you place more than that across components then they will not be displayed.

For instance, in the case of the Observability app, the guidance would be:

Starter prompt	Place to add in the React app
`What is Observability?`	`renderApp`
`What are SLOs?`	`renderApp`
`How do I set up Alerts?`	`renderApp`
`Can you describe the different rule types?`	Rules page
`Can you help me configure an SLO?`	SLO List
`Do I have any misconfigured SLOs?`	SLO List

Please validate whether or not the response given by the LLM is correct and helpful for the user.
If it does not, please contact the Observability AI Assistant team as there are multiple techniques to make the LLMs answer smarter. We can work with you to find the way that is most effective.

apmmachine · 2024-03-18T20:28:12Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
/oblt-deploy-serverless : Deploy a serverless Kibana instance using the Observability test environments.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-03-19T10:28:55Z

Pinging @elastic/obs-knowledge-team (Team:obs-knowledge)

elasticmachine · 2024-03-19T10:28:55Z

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

…ompts

x-pack/plugins/observability_solution/observability/public/application/index.tsx

x-pack/plugins/observability_solution/slo/public/pages/slos/components/slo_list.tsx

...s/observability_solution/observability_ai_assistant_app/public/components/chat/chat_body.tsx

...rvability_solution/observability_ai_assistant_app/public/components/chat/starter_prompts.tsx

...rvability_solution/observability_ai_assistant_app/public/components/chat/welcome_message.tsx

...k/plugins/observability_solution/observability_ai_assistant_app/public/utils/non_nullable.ts

…ompts

mgiota

LGTM!

CoenWarmer · 2024-03-19T15:45:10Z

Updated the design, it's now:

x-pack/plugins/observability_solution/observability/public/application/index.tsx

x-pack/plugins/observability_solution/observability_ai_assistant/public/plugin.tsx

...rvability_solution/observability_ai_assistant_app/public/components/chat/starter_prompts.tsx

x-pack/plugins/observability_solution/slo/public/application.tsx

…ompts

CoenWarmer · 2024-04-05T16:50:57Z

...servability_solution/observability/public/hooks/use_get_available_rules_with_descriptions.ts

+
+  const {
+    ruleTypesState: { data: ruleTypes },
+  } = useLoadRuleTypesQuery({


useLoadRuleTypesQuery() returns ids and a name, but no description

CoenWarmer · 2024-04-05T16:51:40Z

...servability_solution/observability/public/hooks/use_get_available_rules_with_descriptions.ts

+  });
+
+  return useMemo(() => {
+    const ruleTypesFromRuleTypeRegistry = ruleTypeRegistry.list();


ruleTypeRegistry.list() returns ids and descriptions, but no name 😅

x-pack/plugins/observability_solution/observability/public/pages/alerts/alerts.tsx

.../plugins/observability_solution/observability_ai_assistant/public/content/starter_prompts.ts

dgieselaar · 2024-04-05T17:10:20Z

...rvability_solution/observability_ai_assistant_app/public/components/chat/starter_prompts.tsx

+
+  const starterPrompts = uniq(
+    [...contexts]
+      .reverse()


I don't think you want to order this. We cannot depend on order, because components get mounted and unmounted, changing the order of when things are registered

Interesting. While working and testing the feature I haven't seen it not working. Can you give an example of when this does not work?

What do you expect the order to be based on? You can probably fairly easily reproduce this by adding useEffect hooks that register starting prompts, and invalidating the dependencies for one on every render, and noticing that that will always come up on top.

cauemarcondes · 2024-04-08T12:11:31Z

...plugins/observability_solution/apm/public/components/routing/templates/apm_main_template.tsx

@@ -118,21 +119,52 @@ export function ApmMainTemplate({
    isServerless: config?.serverlessOnboarding,
  });

+  let screenDescription = '';


Could we move the AI assistant code to its own file?

Same comments for the other files. I think we could clean up the components.

Not really. This functionality is intended for other plugins to give contextual information to the assistant about what the user is looking at right now.

I think what @cauemarcondes means is that it could be a separate hook or something similar. I've also done that in other places in APM (e.g. getThroughputScreenContext).

I saw those and didn't think it was super readable. But if its important for you, I will oblige

IMO it keeps the component clean and small making it easy to maintain.

IMO jumping between files hinders readability and leads engineers to just leave it alone because they don't see the contents anymore. Many, many examples of this in the Kibana codebase abound.

As stated though I will update.

IMO jumping between files hinders readability and leads engineers to just leave it alone because they don't see the contents anymore

I tend to agree and disagree with that 😄. The screen context/description is very specific for the AI assistant to know about this page/component. Me as a developer who is actively working on these components/pages I don't care what the assistant is doing. I know they are there though. What I want is to see a clean component, not having a bunch of static text on my way 😆.

For you, I think it'll be even easier as you'll have a clean file with only AI stuff.

...plugins/observability_solution/apm/public/components/routing/templates/apm_main_template.tsx

…ompts

tonyghiani

infra changes LGTM

dgieselaar · 2024-04-10T07:44:10Z

...lution/synthetics/public/apps/synthetics/components/getting_started/getting_started_page.tsx

+    return setScreenContext?.({
+      screenDescription: hasNoLocations
+        ? 'The user has no locations configured.'
+        : `The user has ${locations.length} locations configured: ${JSON.stringify(locations)}`,


any idea how much tokens this is?

dgieselaar · 2024-04-10T07:48:49Z

x-pack/plugins/observability_solution/observability/public/pages/alerts/alerts.tsx

+    return setScreenContext?.({
+      data: ruleTypesWithDescriptions.map((rule) => ({
+        name: rule.id,
+        value: `${rule.name} ${rule.description}`,


you can use value: rule. In this case it probably is overkill to use data (I wasn't aware of the shape, sorry). But it's also fine to leave it as-is, there's not a lot of tokens so it gets sent over automatically.

dgieselaar · 2024-04-12T12:28:16Z

@elasticmachine merge upstream

dgieselaar · 2024-04-12T13:22:26Z

I've created a follow-up issue for the ordering issue: #180698.

kibana-ci · 2024-04-12T13:45:21Z

💚 Build Succeeded

Buildkite Build
Commit: d2200a8

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`apm`	1675	1676	+1
`observability`	507	508	+1
`observabilityAIAssistant`	88	89	+1
`observabilityAIAssistantApp`	221	223	+2
total			+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	3.2MB	3.2MB	+572.0B
`infra`	1.4MB	1.4MB	+1002.0B
`observability`	282.7KB	285.6KB	+2.9KB
`observabilityAIAssistantApp`	141.9KB	142.5KB	+630.0B
`slo`	660.8KB	661.4KB	+603.0B
`synthetics`	848.7KB	849.3KB	+592.0B
`ux`	165.6KB	166.3KB	+750.0B
total			+7.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`observabilityAIAssistant`	44.6KB	45.7KB	+1.1KB

History

💚 Build #202086 succeeded e9a102b
💚 Build #201840 succeeded 37e89df
💛 Build #201510 was flaky 819ff0c
💔 Build #201419 failed 3ac8172
💔 Build #201392 failed 6b4ea31
💔 Build #201123 failed 3afa675

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @CoenWarmer

cauemarcondes

Obs UX LGTM

Create first Starter prompts

675ac0f

Merge branch 'main' into feat/starter-prompts

482d705

CoenWarmer marked this pull request as ready for review March 18, 2024 20:29

CoenWarmer requested review from a team as code owners March 18, 2024 20:29

CoenWarmer added the release_note:skip Skip the PR/issue when compiling release notes label Mar 18, 2024

Merge branch 'main' into feat/starter-prompts

a3d91ee

botelastic bot added Team:obs-knowledge Observability Experience Knowledge team Team:obs-ux-management Observability Management User Experience Team labels Mar 19, 2024

Cleanup

7567cf7

CoenWarmer force-pushed the feat/starter-prompts branch from 2d71c50 to 7567cf7 Compare March 19, 2024 10:30

Merge branch 'main' of github.com:elastic/kibana into feat/starter-pr…

aa8a7ed

…ompts

mgiota self-requested a review March 19, 2024 10:53

mgiota reviewed Mar 19, 2024

View reviewed changes

x-pack/plugins/observability_solution/observability/public/application/index.tsx Outdated Show resolved Hide resolved

mgiota reviewed Mar 19, 2024

View reviewed changes

x-pack/plugins/observability_solution/slo/public/pages/slos/components/slo_list.tsx Outdated Show resolved Hide resolved

dgieselaar reviewed Mar 19, 2024

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into feat/starter-pr…

7f0fcdc

…ompts

mgiota approved these changes Mar 19, 2024

View reviewed changes

Address feedback

7643392

CoenWarmer force-pushed the feat/starter-prompts branch from d288c27 to 7643392 Compare March 19, 2024 15:42

CoenWarmer requested a review from dgieselaar March 19, 2024 15:43

dgieselaar reviewed Mar 19, 2024

View reviewed changes

Address feedback

492707a

CoenWarmer requested review from a team as code owners April 4, 2024 13:54

Merge branch 'main' of github.com:elastic/kibana into feat/starter-pr…

3afa675

…ompts

CoenWarmer commented Apr 5, 2024

View reviewed changes

x-pack/plugins/observability_solution/observability/public/pages/alerts/alerts.tsx Outdated Show resolved Hide resolved

Fix test

819ff0c

CoenWarmer force-pushed the feat/starter-prompts branch from 6f9eeb6 to 819ff0c Compare April 5, 2024 17:04

CoenWarmer commented Apr 5, 2024

View reviewed changes

.../plugins/observability_solution/observability_ai_assistant/public/content/starter_prompts.ts Outdated Show resolved Hide resolved

dgieselaar reviewed Apr 5, 2024

View reviewed changes

dgieselaar assigned CoenWarmer Apr 6, 2024

cauemarcondes reviewed Apr 8, 2024

View reviewed changes

...plugins/observability_solution/apm/public/components/routing/templates/apm_main_template.tsx Outdated Show resolved Hide resolved

CoenWarmer added 2 commits April 8, 2024 16:31

Address feedback

9a60c35

Merge branch 'main' of github.com:elastic/kibana into feat/starter-pr…

37e89df

…ompts

CoenWarmer force-pushed the feat/starter-prompts branch from 3316cbf to 37e89df Compare April 8, 2024 14:32

Address feedback

e9a102b

tonyghiani approved these changes Apr 9, 2024

View reviewed changes

dgieselaar reviewed Apr 10, 2024

View reviewed changes

Merge branch 'main' into feat/starter-prompts

d2200a8

CoenWarmer requested a review from a team as a code owner April 12, 2024 12:28

dgieselaar mentioned this pull request Apr 12, 2024

[Observability AI Assistant] Order of starter prompts is unpredictable #180698

Open

dgieselaar approved these changes Apr 12, 2024

View reviewed changes

dgieselaar enabled auto-merge (squash) April 12, 2024 13:47

dgieselaar approved these changes Apr 12, 2024

View reviewed changes

dgieselaar requested a review from cauemarcondes April 12, 2024 13:47

cauemarcondes approved these changes Apr 12, 2024

View reviewed changes

dgieselaar merged commit 5816d1a into elastic:main Apr 12, 2024
20 checks passed

kibanamachine added v8.14.0 backport:skip This commit does not require backporting labels Apr 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Observability AI Assistant] Create first Starter prompts #178907

[Observability AI Assistant] Create first Starter prompts #178907

CoenWarmer commented Mar 18, 2024 •

edited

Loading

apmmachine commented Mar 18, 2024

elasticmachine commented Mar 19, 2024

elasticmachine commented Mar 19, 2024

mgiota left a comment

CoenWarmer commented Mar 19, 2024

CoenWarmer Apr 5, 2024 •

edited

Loading

CoenWarmer Apr 5, 2024 •

edited

Loading

dgieselaar Apr 5, 2024

CoenWarmer Apr 6, 2024

dgieselaar Apr 6, 2024 •

edited

Loading

cauemarcondes Apr 8, 2024

cauemarcondes Apr 8, 2024

CoenWarmer Apr 8, 2024

dgieselaar Apr 8, 2024

CoenWarmer Apr 8, 2024

cauemarcondes Apr 9, 2024

CoenWarmer Apr 9, 2024 •

edited

Loading

cauemarcondes Apr 10, 2024

tonyghiani left a comment

dgieselaar Apr 10, 2024

dgieselaar Apr 10, 2024

dgieselaar commented Apr 12, 2024

dgieselaar commented Apr 12, 2024

kibana-ci commented Apr 12, 2024

cauemarcondes left a comment

[Observability AI Assistant] Create first Starter prompts #178907

[Observability AI Assistant] Create first Starter prompts #178907

Conversation

CoenWarmer commented Mar 18, 2024 • edited Loading

Summary

Default Starter Prompts (visible in all Observability apps when no additional prompts have been registered by app / page)

Starter Prompts added to different specific Observability apps

Guidance for Kibana engineers

apmmachine commented Mar 18, 2024

🤖 GitHub comments

elasticmachine commented Mar 19, 2024

elasticmachine commented Mar 19, 2024

mgiota left a comment

Choose a reason for hiding this comment

CoenWarmer commented Mar 19, 2024

CoenWarmer Apr 5, 2024 • edited Loading

Choose a reason for hiding this comment

CoenWarmer Apr 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgieselaar Apr 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CoenWarmer Apr 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tonyghiani left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgieselaar commented Apr 12, 2024

dgieselaar commented Apr 12, 2024

kibana-ci commented Apr 12, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

cauemarcondes left a comment

Choose a reason for hiding this comment

CoenWarmer commented Mar 18, 2024 •

edited

Loading

CoenWarmer Apr 5, 2024 •

edited

Loading

CoenWarmer Apr 5, 2024 •

edited

Loading

dgieselaar Apr 6, 2024 •

edited

Loading

CoenWarmer Apr 9, 2024 •

edited

Loading