Open
Conversation
Added a profile to later be used for title generation, The idea here is to allow profiles to select a title generator profile that will be used for auto generating titles.
Inside the chat settings there is now a section for titles, with the only option currently being the selector for which profile to use for title generation. By default all profiles use the `titleGenerator` profile, but `jenLlama` has been overridden to use itself since it's not supposed to be reliant upon OpenAI.
Moved the prompt for title generation from the system prompt to a new field. This is necessary since Jen currently uses itself as its title generator, and thus we need to distinguish between the overall system prompt and the title prompt. This also allows other models to use custom prompts different from the default one for its title generator.
Newer models (o1 and newer) prefer to use the `developer` role over the `system` role.
Previously the overrides parameter was the full structure, which needlessly forced TypeScript into warning the some required members were not set. Since this is specifically for overriding some properties, it makes more sense for the is be a Partial, which makes all members optional.
Title requests potentially have some differences from summary requests, so it was created as a new option. Currently this disables image generation, allows system/developer prompts, and prevents continuous chat (which overwrites our system/developer prompt).
This moves from using the current model being used for title generation, to using the specific title generation profile. The reason for this is that you may be conversing with the very expensive GPT 4.5, or some other preview model, and don't want to be billed so harshly for a simple title. This also allows for overrides to the title generation prompt, in case the hardcoded one doesn't work well. The current Title Generator profile uses the cheapest GPT 4.1 nano model, but it's not great at following instructions. I would recommend the GPT 4o mini model, which has similar pricing and better titles. At the current time, this is not existent in the available models though, so we're sticking with GPT 4.1 nano for now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds configuration options to title generation.
Namely, this allows for profiles to assign another profile as its title generator. This is because it previously used the model from the current chat, which could become pricey if you were using GPT 4.5, or some other expensive preview.
Rather than add a substantial amount of configuration options to all profiles, this instead allows delegation to another profile. A profile specifically created for title generation was added just for this purpose, and it is the default title generator profile. The only exception is for the Jen profile, since that does not use OpenAI.
This also allows for a given profile to customize its title generation. This primarily came out of necessity for separating the title prompt from the existing system prompt. This was because the Jen profile just uses itself rather than another profile. Due to the nature of the design, this is a possibility for any profile an end user makes. As such a separate field was added for the title prompt.
One change I would make is to use GPT 4o mini instead of GPT 4.1 nano. Nano doesn't work well at generating titles, and costs only marginally less than 4o mini. The only reason I didn't do that, was 4o mini no longer exists as a selectable profile. #580 does add this model back, and it would be my recommendation to use that instead once #580 is mainlined.
This PR does bear some similarities to #567, but it also has some differences. This PR is a much narrower change, isolating itself solely to expanding the capabilities of title generation. #567 also adds automatic title generation, something I would like to also add, but which extends beyond the scope of this change.
#567 also limits the user to using GPT 4.1 nano, even if they are using LLaMa. This instead allows for greater flexibility in model choice.
#567 also chooses to filter only to user messages, whereas this continues using both user and assistant messages, this can easily be brought into conformity with #567 though.