Skip to content

[Bug]: Incompatibility between ChatGeneration and GenerationCacheEntryModel in AutoLabeling with SQLAlchemyGenerationCache #960

@zmin1217

Description

@zmin1217

When using a Chat-type LLM (Large Language Model) and initializing AutoLabeling with the default SQLAlchemyGenerationCache, there's a compatibility issue in the get method of GenerationCacheEntryModel.

The problem occurs because the method uniformly constructs Generation objects using:
generations = [Generation(**gen) for gen in generations]
This implementation is incompatible with Chat-type LLM outputs that produce ChatGeneration objects instead of Generation objects, resulting in validation errors.

File "/opt/cloudera/parcels/Anaconda/envs/auto/lib/python3.8/site-packages/langchain_core/load/serializable.py", line 113, in __init__
    super().__init__(*args, **kwargs)
  File "pydantic/main.py", line 347, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Generation
type
  unexpected value; permitted: 'Generation' (type=value_error.const; given=ChatGeneration; permitted=('Generation',))

Is it possible to use the following approach to resolve this?

if generations[0]['type'] == 'Generation': 
    generations = [Generation(**gen) for gen in generations]
else:
    generations = [ChatGeneration(**gen) for gen in generations]

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions