Skip to content

feat: extension_entry event that only fires once per instantiation#65

Merged
ggprior merged 2 commits intomainfrom
georg/eng-192-count-extension-entry-points-instead-of-model-calls
Mar 27, 2026
Merged

feat: extension_entry event that only fires once per instantiation#65
ggprior merged 2 commits intomainfrom
georg/eng-192-count-extension-entry-points-instead-of-model-calls

Conversation

@ggprior
Copy link
Copy Markdown
Contributor

@ggprior ggprior commented Mar 24, 2026

Adding a new extension_entry event that only fires once per instantiation, to get a feel for extension usage regardless of number of fit/predict calls.
Will bump version in extensions and create posthog insight afterwards.

@ggprior ggprior requested a review from safaricd March 24, 2026 11:26
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the telemetry system by introducing a new event, extension_entry, designed to provide more accurate insights into extension usage. Unlike existing events that might fire multiple times within a single extension's lifecycle, this new event ensures that each unique instantiation of an extension is recorded only once. This change will facilitate better understanding of how extensions are utilized by users.

Highlights

  • New Telemetry Event: Introduced a new ExtensionEntryEvent designed to track unique extension instantiations, providing a more accurate measure of extension usage compared to events that fire per model call.
  • Decorator Integration: Modified the @set_extension decorator to emit the ExtensionEntryEvent precisely once when an outermost extension is entered, including robust error handling for event capture failures.
  • Comprehensive Testing: Added extensive unit tests to verify the correct emission logic of the ExtensionEntryEvent, covering scenarios such as single calls, nested calls, sequential calls, and class decorators.
  • Dependency Updates: Relaxed upper version bounds for pandas, posthog, and typing-extensions dependencies in uv.lock, and removed some specific platform wheels for cffi.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new extension_entry telemetry event, which is designed to fire only once per extension instantiation. The implementation looks solid, with a new event class, modifications to the @set_extension decorator to fire the event, and comprehensive new tests for both the event and the decorator logic. However, I've noticed some potentially unintended changes in the uv.lock file. The version constraints for pandas, posthog, and typing-extensions have been relaxed, removing upper bounds. This could lead to future breakages if new major versions of these libraries with incompatible changes are released. I've added specific comments on these lines. Additionally, I've suggested a small improvement to a test name for clarity.

{ name = "numpy", specifier = ">=1.21.6" },
{ name = "nvidia-ml-py", specifier = ">=13.590.48" },
{ name = "pandas", specifier = ">=1.4.0,<3" },
{ name = "pandas", specifier = ">=1.4.0" },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The upper bound for the pandas dependency has been removed, changing from >=1.4.0,<3 to >=1.4.0. This allows pandas version 3.0 and higher, which may introduce breaking changes. To prevent potential compatibility issues, it is recommended to restore the upper bound unless compatibility with pandas v3 has been verified and is intended.

{ name = "pandas", specifier = ">=1.4.0" },
{ name = "platformdirs", specifier = ">=4" },
{ name = "posthog", specifier = "~=6.7" },
{ name = "posthog", specifier = ">=6.7" },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The version constraint for posthog has been relaxed from ~=6.7 to >=6.7. The ~= operator (equivalent to >=6.7, <7.0) protects against breaking changes in new major versions. The new constraint >=6.7 allows any future version, including major versions that may not be compatible. It is recommended to use a more restrictive constraint like ~=6.7 to ensure stability.

{ name = "scikit-learn", specifier = ">=1.2.0" },
{ name = "twine", marker = "extra == 'build'", specifier = ">=5.0.0" },
{ name = "typing-extensions", specifier = ">=4.12,<5" },
{ name = "typing-extensions", specifier = ">=4.12" },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The upper bound for typing-extensions has been removed, changing from >=4.12,<5 to >=4.12. This could allow version 5.x, which might introduce breaking API changes. To prevent potential issues, it's advisable to restore the upper bound unless compatibility with the new major version has been verified and is intended.

assert mock_capture.call_args[0][0].extension_name == "cls_ext"

@patch("tabpfn_common_utils.telemetry.core.decorators.capture_event")
def test_event_not_emitted_on_exception(self, mock_capture):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The name of this test, test_event_not_emitted_on_exception, is a bit misleading. It suggests that an exception within the decorated function would prevent event emission. However, the test actually verifies that an exception during the capture_event call does not crash the application and allows the decorated function to run. A more descriptive name like test_function_executes_when_capture_event_fails would better reflect the test's purpose.

@ggprior ggprior requested a review from safaricd March 26, 2026 09:09
Copy link
Copy Markdown
Collaborator

@safaricd safaricd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ggprior ggprior force-pushed the georg/eng-192-count-extension-entry-points-instead-of-model-calls branch 2 times, most recently from b4fc9f7 to bf3db80 Compare March 27, 2026 08:22
@ggprior ggprior force-pushed the georg/eng-192-count-extension-entry-points-instead-of-model-calls branch from bf3db80 to fb600a6 Compare March 27, 2026 08:25
@ggprior ggprior merged commit 59be002 into main Mar 27, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants