When using a Chat-type LLM (Large Language Model) and initializing AutoLabeling with the default SQLAlchemyGenerationCache, there's a compatibility issue in the get method of GenerationCacheEntryModel.
The problem occurs because the method uniformly constructs Generation objects using:
generations = [Generation(**gen) for gen in generations]
This implementation is incompatible with Chat-type LLM outputs that produce ChatGeneration objects instead of Generation objects, resulting in validation errors.
File "/opt/cloudera/parcels/Anaconda/envs/auto/lib/python3.8/site-packages/langchain_core/load/serializable.py", line 113, in __init__
super().__init__(*args, **kwargs)
File "pydantic/main.py", line 347, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Generation
type
unexpected value; permitted: 'Generation' (type=value_error.const; given=ChatGeneration; permitted=('Generation',))
Is it possible to use the following approach to resolve this?
if generations[0]['type'] == 'Generation':
generations = [Generation(**gen) for gen in generations]
else:
generations = [ChatGeneration(**gen) for gen in generations]