Multi model tensorflow container not working

[ag-labs-sys]

Hi,

I have deployed a multi model endpoint with tensorflow serving container 1.5.3 cpu but it's returning gibberish

7\n ],\n [\n -1.2174921414467445e61\n ],\n [\n 5.648687733862501e147\n ],\n [\n -2.2103468677006018e21\n ],\n [\n -4.0616676729872787e-168\n ],\n [\n 9.245780168734567e172\n ],\n [\n -4.97308126648274e195\n ],\n [\n -2.8919419839829766e-96\n ],\n [\n 1.149778695782349e-174\n ],\n [\n -3.754576890892577e-106\n ],\n [\n -5.147271276809809e123\n ],\n [\n 2.0358998311689856e62\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 2.5863434848353987e161\n ],\n [\n 1.376136905749849e-153\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ]\n ],\n [\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n 0.0\n ],\n [\n Is not object'}

Not really sure whats happening with this. Will appreciate any help with this. Cheers!

[jloux]

I had this same problem.

For single endpoints, my requests originally looked (roughly) like this:

 client.invoke_endpoint(
            EndpointName=endpoint_name,
            Body=json.dumps(SOME_NUMPY_ARRAY),
            ContentType='application/jsonlines'
        )

For MME, this didn't work. And my team found your issue. :)

I looked in the unit tests for this container, and found that MME requests all included a JSON object with the key "instances".

https://github.com/aws/sagemaker-tensorflow-serving-container/blob/dc1ccd1cb19114a0b357862aa2177e9d2a67fdf5/test/integration/local/test_multi_model_endpoint.py

This worked for us for invoking MMEs:

 client.invoke_endpoint(
           EndpointName=endpoint_name,
           TargetModel=model_name
            Body=json.dumps({'instances': SOME_NUMPY_ARRAY}),
            ContentType='application/jsonlines'
        )

I hope this helps. :)