New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ref: batch size > max_request_per_batch happens post dataset files creation. #352

Open

adamoptimizer opened this issue Jan 12, 2025 · 0 comments

Assignees

Labels

batch-api low priority bug

Contributor

adamoptimizer commented Jan 12, 2025

We should error out prior to any dataset files creations.
All config related error checking should be part of config validation

curator/src/bespokelabs/curator/request_processor/batch/base_batch_request_processor.py

Line 82 in 7e4e6f1

if self.config.batch_size > self.max_requests_per_batch:

adamoptimizer added batch-api low priority bug labels

adamoptimizer self-assigned this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment