diff --git a/docs/docs/experimentation-ab-testing.md b/docs/docs/experimentation-ab-testing.md index 1ac74ca5e555..95055fc8a9fa 100644 --- a/docs/docs/experimentation-ab-testing.md +++ b/docs/docs/experimentation-ab-testing.md @@ -23,7 +23,7 @@ By the end of this tutorial, you will be able to: To follow this tutorial, you will need: -- A basic understanding of [multivariate flags](/managing-flags/core-management) in Flagsmith. +- A basic understanding of [multivariate flags](/managing-flags/core-management) in Flagsmith. Remember: multivariate bucketing only works when users are identified (use real identities or persistent anonymous GUIDs). - Access to a third-party analytics platform (e.g., Amplitude, Mixpanel) where you can send custom events. You can explore Flagsmith [integrations](/third-party-integrations/analytics/segment) for this purpose. - A development environment for your application where you can implement changes and integrate the Flagsmith SDK. diff --git a/docs/docs/managing-flags/core-management.md b/docs/docs/managing-flags/core-management.md index 14ec0241a5c3..5d9b29691fe7 100644 --- a/docs/docs/managing-flags/core-management.md +++ b/docs/docs/managing-flags/core-management.md @@ -29,6 +29,26 @@ By clicking the **Create A/B/n Test** button, you can define values for A/B test ::: +### Multivariate flags (A/B/n) + +Multivariate flags let you define multiple variants with percentage weightings (A/B/n). Key points: + +- Require identified users: multivariate bucketing is calculated per identity, so you must identify users (or generate a unique anonymous identifier like a GUID/UUID) to serve a consistent variant. +- Deterministic per environment: the same identity gets the same variant within an environment unless you change weights. +- Stable weights matter: changing weightings during a live test can re-bucket users; avoid mid-test weight changes unless you intentionally want to reshuffle. +- Per-environment: identity bucketing is independent per environment; the same identity can land in different buckets across environments. +- Percentages must sum to 100% and you need at least two variants. +- The SDK returns both the boolean `enabled` state and the multivariate `value`; use the value to drive your experiment behaviour. + +If you need to test unauthenticated traffic, generate a unique identifier (like a GUID/UUID) for each anonymous user and store it persistently (e.g., in a cookie or localStorage) so the same user gets the same variant across sessions. + +Quick A/B/n setup checklist: +1. Define variants and weights (total 100%) +2. Enable the flag +3. Identify the user (or anonymous GUID) before fetching +4. Use the `value` of the feature when defining the branching logic in your code +5. Enable required [integrations](/third-party-integrations/) for analytics purposes + --- ## Editing a Feature Flag @@ -85,6 +105,11 @@ Deleting a feature flag is permanent and cannot be undone. Make sure your applic - Make sure you have saved your changes in each tab of the edit feature flag panel. - Check that you are in the correct project and environment. +### Multivariate Flags Giving Inconsistent Results + +- Ensure you identify users before fetching flags. For anonymous users, generate a unique identifier and store it persistently (e.g., cookie or localStorage) so the same user gets the same variant across sessions. +- Avoid changing weightings during a live experiment, as this can re-bucket users and invalidate results. + ### Permission Issues - You may need additional permissions to create, edit, clone, or delete feature flags. If you see permission errors or options are disabled, contact your Flagsmith administrator to review your access rights. For more information, see the [Permissions and Roles](/administration-and-security/access-control/rbac) page.