Data discrepancy between bootstrapped data and front-end data

[bjfresh]

Bug description

Data discrepancy between bootstrapped data and front-end data

How to reproduce

On server:

import { z } from "zod"
import { rpc } from "../../trpc"
import { PostHogClient } from "../../../clients/posthog"

import { PostHog } from "posthog-node"
import { POSTHOG_API_KEY } from "../constants"

export const PostHogClient = new PostHog(POSTHOG_API_KEY, {
	host: "https://us.i.posthog.com",
	flushAt: 1,
	flushInterval: 0,
})

const userFlagsSchema = z.object({
	profileTBA: z.string(),
})

export const userFlags = rpc.input(userFlagsSchema).query(async ({ input }) => {
	const userFlags = await PostHogClient.getAllFlags(input.profileTBA)
	console.log({ userFlags })
	await PostHogClient.shutdown()

	return userFlags
})

Bootstrap + init React Native client:

import { trpc } from "@/services/trpc"
import { usePermaUser } from "./usePermaUser"
import { useEffect, useState } from "react"
import { initializePostHogClient } from "@/services/posthog"
import type PostHog from "posthog-react-native"

export function usePostHogClient({
	profileTBA,
}: {
	profileTBA?: `0x${string}`
}): PostHog | null {
	const [PostHogClient, setPosthogClient] = useState<PostHog | null>(null)
	const permaUser = usePermaUser({ userAddress: profileTBA })

	// Fetch the user's feature flags from the server so we can bootstrap the PostHog client
	// This helps ensure that the PostHog client is ready to go with accurate flag data when we need it
	// Without this, the data may not be ready in time for the first feature flag check and the flag may be inaccurate
	const userFeatureFlags = trpc.featureFlags.userFlags.useQuery(
		{
			// biome-ignore lint/style/noNonNullAssertion: <We know profileTBA is defined because this hook is otherwise disabled>
			profileTBA: profileTBA!,
		},
		{
			enabled: !!profileTBA,
		},
	)

	// console.log({ userFeatureFlags })

	// Init the PostHog client and bootstrap with the user's feature flags
	useEffect(() => {
		const init = async () => {
			if (!!PostHogClient || !userFeatureFlags.data || !profileTBA) {
				// Exit early if the PostHog client is already initialized, or if we're missing data
				return
			}

			const posthog = await initializePostHogClient({
				profileTBA,
				username: permaUser?.user?.username?.label ?? "",
				featureFlags: userFeatureFlags.data,
			})
			await posthog.ready()

			setPosthogClient(posthog)
		}
		init()
	}, [
		PostHogClient,
		profileTBA,
		userFeatureFlags.data,
		permaUser?.user?.username?.label,
	])

	return PostHogClient
}

Client-side query for ** getFeatureFlag**:

Before fetching, logic confirms:

• PostHogClient is instantiated
• PostHogClient has distinctId

Both server-side and client-side clients are instantiated with profileTBA as distinctId.

So the data should be identical, but we're seeing results where the value differs. In effect, we bootstrap the client with the correct data, and then on refetch it makes the data incorrect. This is for an isAdmin flag attached to a cohort, so it's important as protection.

import {
	type UseQueryResult,
	useQuery,
	useQueryClient,
} from "@tanstack/react-query"
import type { JsonType } from "posthog-react-native/lib/posthog-core/src/types"
import type { ZodSchema } from "zod"
import { useOnAppBackground } from "./useOnAppStateBackgrounded"
import { useRefetchOnAppFocus } from "./useRefetchOnAppFocus"
import { isAddress } from "viem"
import { usePostHogClient } from "./usePostHogClient"

interface PostHogFlagState {
	isEnabled: boolean
	payloadQuery: UseQueryResult<JsonType | undefined, Error>
}

// Handles fetching the feature flag state and payload from PostHog
// and refetching when the app comes to the foreground
// Context: 'posthog-react-native' has a useFeatureFlagWithPayload hook, but it doesn't refresh
export function usePostHogFeatureFlag({
	key,
	schema,
	profileTBA,
}: {
	key: string
	schema?: ZodSchema
	profileTBA?: `0x${string}`
}): PostHogFlagState {
	const PostHogClient = usePostHogClient({ profileTBA })
	const queryClient = useQueryClient()

	const shouldFetch =
		!!PostHogClient && isAddress(PostHogClient.getDistinctId()) && !!profileTBA

	const isEnabledQuery = useQuery({
		queryKey: ["posthog", "getFeatureFlag", key, profileTBA],
		queryFn: () => {
			console.log({
				// isFeatureEnabled: PostHogClient?.isFeatureEnabled(key),
				distinctId: PostHogClient?.getDistinctId(),
				getFeatureFlag: PostHogClient?.getFeatureFlag(key),
			})

			return Boolean(
				PostHogClient?.onFeatureFlags(
					() => PostHogClient?.getFeatureFlag(key) ?? false,
				),
			)
		},
		enabled: shouldFetch,
		staleTime: 0,
	})

	const payLoadQueryKey = ["posthog", "getFeatureFlagPayload", key, profileTBA]

	const payloadQuery = useQuery({
		queryKey: payLoadQueryKey,
		queryFn: () => {
			const rawPayload = PostHogClient?.onFeatureFlags(() =>
				PostHogClient?.getFeatureFlagPayload(key),
			)
			if (schema) {
				const parseResult = schema.safeParse(rawPayload)
				return parseResult.success ? parseResult.data : null
			}
			return rawPayload ?? null
		},
		enabled: shouldFetch,
		staleTime: 0,
	})

	// Refetch when the app comes to the foreground
	useRefetchOnAppFocus(async () => {
		isEnabledQuery.refetch()
		payloadQuery.refetch()
	})

	useOnAppBackground(async () => {
		await queryClient.invalidateQueries({
			queryKey: payLoadQueryKey,
			exact: true,
		})
		PostHogClient?.reloadFeatureFlags() // Sadly not async
	})

	return {
		isEnabled: isEnabledQuery.data ?? false,
		payloadQuery,
	}
}

Related sub-libraries

• All of them
• posthog-web
• posthog-node
• posthog-react-native

Thanks in advance for any assistance.

[marandaneto]

@bjfresh hello, thanks for the issue.

// Fetch the user's feature flags from the server so we can bootstrap the PostHog client
// This helps ensure that the PostHog client is ready to go with accurate flag data when we need it
// Without this, the data may not be ready in time for the first feature flag check and the flag may be inaccurate

If you need fresh flags, you can just use reloadFeatureFlagsAsync and await the promise to be resolved, this is better with preloadFeatureFlags: false so you don't reload a 2nd time.
This is also only an issue for the very first time the SDK is installed and the app is opened, after the 1st successful flags have been pulled, the SDK caches in the disk and it will use the cached values on the next app restart in case the request to get new flags again is still in process or failed.

If you still want to do bootstrapping, here it is explained, and here it shows that you need to init the SDK with the bootstrap and featureFlags object.
I don't see the bootstrap values in your code snippet, so I am unsure this would work, also if you'd need feature flags with payloads, you'd need to bootstrap using the featureFlagPayloads value as well.

The bootstrap values are overwritten once the client requests and receives the fresh flags.
The flags will only be the same if the client is also using the same distinctId, so you also have to bootstrap with distinctId.

Since I don't have access to your code nor I can see how initializePostHogClient is created.
Can you provide a MRE so we can try and reproduce the issue? please remove all the logic that is not needed, the less the better, just a way to show off the problem.

Thanks.

[bjfresh]

Ah, whoops, meant to include initializePostHog. We are indeed bootstrapping with distinctId:

import { POSTHOG_HOST_SERVER, POSTHOG_API_KEY } from "@/constants"
import PostHog from "posthog-react-native"

import { getAddress } from "viem"

export async function initializePostHogClient({
	profileTBA,
	username,
	featureFlags,
}: {
	profileTBA: `0x${string}`
	username?: string
	featureFlags?: Record<string, string | boolean>
}) {
	const uniqueIdentifier = getAddress(profileTBA)
	const customIdentifiers = {
		profileTBA: uniqueIdentifier,
		username,
	}

	// Initialize the PostHog client with bootstrap options
	const initPostHog = async () => {
		const postHogClient = new PostHog(POSTHOG_API_KEY as string, {
			host: POSTHOG_HOST_SERVER,
			disabled: __DEV__,
			bootstrap: {
				distinctId: uniqueIdentifier,
				isIdentifiedId: true,
				featureFlags,
			},
		})

		await postHogClient.ready()

		// console.log({ distinctID: postHogClient.getDistinctId() })

		return postHogClient
	}

	const postHogClient = await initPostHog()

	// Only identify once we've confirmed the client is ready
	postHogClient.identify(uniqueIdentifier, customIdentifiers)

	return postHogClient
}

and all of the flag data is as expected before re-fetching. Before refetch in usePostHogFeatureFlag, we verify that the distinctId is indeed set on the PostHogClient as expected to set shouldFetch, and the refetch query is enabled based on that condition. So the client is bootstrapped with distinctId and the featureFlags with proper values, has the expected distinctId when calling getDistinctId, and then simply doesn't give the same vals when calling isEnabledQuery:

	const isEnabledQuery = useQuery({
		queryKey: ["posthog", "getFeatureFlag", key, profileTBA],
		queryFn: () => {
			console.log({
				// isFeatureEnabled: PostHogClient?.isFeatureEnabled(key),
				distinctId: PostHogClient?.getDistinctId(),
				getFeatureFlag: PostHogClient?.getFeatureFlag(key),
			})

			return Boolean(
				PostHogClient?.onFeatureFlags(
					() => PostHogClient?.getFeatureFlag(key) ?? false,
				),
			)
		},
		enabled: shouldFetch,
		staleTime: 0,
	})

I'm also seeing that I get different values based on whether getFeatureFlag is wrapped in onFeatureFlags

[marandaneto]

sorry for the delay, the sample has a bunch of conditions, and using bootstrap flags, it's hard to understand what is going on by just reading the code.
@bjfresh can you provide a MRE? I'd love to test it with this change once merged/released.