-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add an optional 'size' field to the Resource type. #132
Add an optional 'size' field to the Resource type. #132
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this is a great idea! Just one wording nit, but I'm happy with this.
Tagging in @dsp-ant for second approval.
Improved wording Co-authored-by: Justin Spahr-Summers <[email protected]>
This needs to rebased to be moved into draft |
@dsp-ant This is backwards compatible, so I think it can be additive to the last release, no? |
40c83f5
I'll update the appropriate schema pending the Audio Modality (93) PR. |
Thank you for this addition! |
@jspahrsummers - this hasn't made it's way in to the current "draft" schema - is there a merge process (assuming this isn't being removed?). want to check before trying to figure out what when wrong schema.ts on #188 |
Adds an optional 'size' field to the Resource type. The 'size' represents the raw size in bytes (before encoding or tokenization).
This can be used by the Client/Host Application to pre-emptively estimate Context Window usage and make informed decisions about resource handling before attempting to load or process resources.
Motivation and Context
When working with LLMs, it's crucial to manage context window limitations effectively. This addition allows Host/Client applications to make smarter decisions about resource handling before attempting to process them.
For example:
How Has This Been Tested?
Schema change only - no runtime implications to test. SDK implementations will need their own testing coverage when incorporating this field.
Breaking Changes
No breaking changes:
Types of changes
Checklist
Additional context
This change aligns with common patterns in file system and storage APIs where size information is readily available. It provides valuable metadata for client applications while maintaining backward compatibility.
The size field specifically represents raw byte size to avoid confusion with token counts or other encoded sizes. This distinction is documented in the schema comments to prevent misuse.