Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Personal Message Context #33

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

RealistikDash
Copy link
Collaborator

This PR adds keeping track of personal context for queries done outside of threads, alongside the clear_context command.

This aims to make using the query command more viable for usecases where the bot cannot create a thread (such as group chats), allowing for more fluent conversations. The clear_context command was added to make context switching easier, as you dont want the bot bringing up your cooking queries when you want to ask it about code.

Potential issues

  • Personal messages are not counted in the monthly cost and even if they were, context clearing would invalidate them. Perhaps use a "deleted" field in the database to account for these.
  • Currently, context limits are handled by a constant (last 5 messages), which is pretty small. Perhaps using a time based one (messages from the last hour or so) would be more convenient and adding input token calculations into the fetch query directly (last n tokens until you hit the input token limit for the model) to max out the context.

@RealistikDash RealistikDash self-assigned this Feb 25, 2025
@RealistikDash RealistikDash requested a review from cmyui as a code owner February 25, 2025 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant