Skip to content

Add support for new GPT models from OpenAI #1717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

nehcneb
Copy link
Contributor

@nehcneb nehcneb commented Apr 14, 2025

Add support for gpt-4.1, gpt-4.1-mini, gpt-4.1-nano.
Let gpt-4.1-mini replace gpt-4o-mini as the default model.

  • [Not applicable] Closes #xxxx (Replace xxxx with the GitHub issue number).
  • [Not applicable ] Tests added and passed if fixing a bug or adding a new feature.
  • [Done] All code checks passed.

Important

Add support for new OpenAI GPT-4.1 models and set gpt-4.1-mini as the default model, updating tests accordingly.

  • Model support and defaults:
    • Add support for gpt-4.1, gpt-4.1-mini, and gpt-4.1-nano (and dated variants) to _supported_chat_models in openai.py.
    • Change default model in OpenAI class from gpt-4o-mini to gpt-4.1-mini.
  • Tests:
    • Update all model arguments in test_agent_llm_judge.py and .ipynb_checkpoints/test_agent_llm_judge-checkpoint.py from gpt-4o-mini to gpt-4.1-mini.
  • Docs:
    • Update docstring in OpenAI to list new supported models.

This description was created by Ellipsis for 65333f4. It will automatically update as commits are pushed.

nehcneb added 2 commits April 15, 2025 08:23
Add support for gpt-4.1, gpt-4.1-mini, gpt-4.1-nano.
Let gpt-4.1-mini perform the role previously performed by gpt-4o-mini.
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 65333f4 in 2 minutes and 40 seconds

More details
  • Looked at 287 lines of code in 3 files
  • Skipped 0 files when reviewing.
  • Skipped posting 13 drafted comments based on config settings.
1. extensions/llms/openai/pandasai_openai/openai.py:53
  • Draft comment:
    Default model changed to 'gpt-4.1-mini'. Ensure corresponding documentation is updated.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50%
    The comment asks to "ensure documentation is updated", but I can see that the docstring has already been updated to include the new model. The comment is asking for something that has already been done. Additionally, the comment starts with "Ensure that...", which violates our rules about not asking authors to verify things.
    Could there be other documentation outside this file that needs to be updated? Could this be a valid concern about maintaining consistency across the codebase?
    While there might be other documentation, we are instructed to ignore cross-file issues and only think about the file we are reviewing. In this file, the documentation has already been updated.
    The comment should be deleted because it asks for documentation updates that have already been made in this file, and we should ignore potential documentation needs in other files.
2. tests/unit_tests/agent/.ipynb_checkpoints/test_agent_llm_judge-checkpoint.py:93
  • Draft comment:
    Grammar fix: Change 'to connects' to 'to connect' in the prompt.
  • Reason this comment was not posted:
    Marked as duplicate.
3. tests/unit_tests/agent/.ipynb_checkpoints/test_agent_llm_judge-checkpoint.py:175
  • Draft comment:
    Typo: Change 'heart stoke' to 'heart stroke' in function docstring.
  • Reason this comment was not posted:
    Marked as duplicate.
4. tests/unit_tests/agent/test_agent_llm_judge.py:175
  • Draft comment:
    Typo: Change 'heart stoke' to 'heart stroke' in function docstring.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
5. extensions/llms/openai/pandasai_openai/openai.py:19
  • Draft comment:
    Docstring updated to include new GPT-4.1 models. Ensure this list stays in sync with _supported_chat_models.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. extensions/llms/openai/pandasai_openai/openai.py:53
  • Draft comment:
    Default model updated to 'gpt-4.1-mini' as specified. Verify that all components consuming this default are updated accordingly.
  • Reason this comment was not posted:
    Comment did not seem useful. Confidence is useful = 0% <= threshold 50%
    This comment is asking the PR author to verify that all components consuming the default model are updated accordingly. This falls under the rule of not asking the author to double-check things or ensure behavior is intended. Therefore, this comment should be removed.
7. tests/unit_tests/agent/.ipynb_checkpoints/test_agent_llm_judge-checkpoint.py:175
  • Draft comment:
    Typo: Replace 'heart stoke' with 'heart stroke' in the test description.
  • Reason this comment was not posted:
    Marked as duplicate.
8. tests/unit_tests/agent/.ipynb_checkpoints/test_agent_llm_judge-checkpoint.py:1
  • Draft comment:
    Consider excluding .ipynb_checkpoints files from version control to avoid redundant test executions.
  • Reason this comment was not posted:
    Comment was on unchanged code.
9. tests/unit_tests/agent/test_agent_llm_judge.py:80
  • Draft comment:
    The evaluation prompt is duplicated across test files. Consider refactoring it into a shared fixture or constant to DRY up the code.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
10. tests/unit_tests/agent/test_agent_llm_judge.py:205
  • Draft comment:
    Tests write an artifact file to the project root. Consider using a temporary directory to avoid polluting the repo.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
11. tests/unit_tests/agent/.ipynb_checkpoints/test_agent_llm_judge-checkpoint.py:93
  • Draft comment:
    Typographical error: In the evaluation_prompt string, change 'to connects' to 'to connect' to fix the grammar.
  • Reason this comment was not posted:
    Marked as duplicate.
12. tests/unit_tests/agent/test_agent_llm_judge.py:93
  • Draft comment:
    Typographical issue in the evaluation prompt text: Consider correcting 'pd.Dataframe' to 'pd.DataFrame' and changing 'connects' to 'connect' for clarity.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
13. tests/unit_tests/agent/test_agent_llm_judge.py:175
  • Draft comment:
    Typographical error in the docstring for test_combined_questions_with_type: 'heart stoke' should be corrected to 'heart stroke'.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.

Workflow ID: wflow_WNVnBMoeFJK5zJiA


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant