Skip to content

[Serving] Save keras model with huge embedding layers. #1447

Open
@workingloong

Description

@workingloong

Now, we need to load the whole model variables in a single process to export the model using SavedModel. However, the ids may be very sparse in advertising and recommendation systems, so the embedding layer size may be huge and the model size may extend the memory. In this case, we should save variable shards in every parameter server and save the model definition as saved_model.pb in any one worker.
That is, we need to save the model definition and variables separately. But now, we can not separate the model definition and variables initialization by Keras API in the eager mode. The variables in the model will be initialized eagerly in the process when we build the model to get the complete model definition.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions