Issues Encountered While Testing a Multi-Service gRPC Server with Runn

[xiao-yao-tn]

I am experiencing a consistent issue while using Runn for testing my gRPC services.
When attempting to test a specific service, let's refer to it as ServiceA/MethodA,
Runn unexpectedly reports an error related to a different service, which we will call ServiceB.
This error occurs even though ServiceB is not directly involved in the test case for ServiceA/MethodA.

error message

1) grpc/tests/ServiceA_test.yml
   └── grpc/tests/requestBase.yml
  Failure/Error: gRPC request failed on "Generated by `runn new`".steps[0]: failed to find descriptor: symbol not found: ServiceB
  Failure step (grpc/tests/requestBase.yml):
  ...

yaml file

desc: TestServiceA
vars:
  request:
    - "json://../test-data/ServiceA/0-request.json"
  response:
    - "json://../test-data/ServiceA/0-response.json"
steps:
  - loop:
      count: len(vars.request)
    include:
      path: requestBase.yml
      vars:
        ignoreField: "request_time"
        request: "{{ vars.request[i] }}"
        response: "{{ vars.response[i] }}"
        testPath: "ServiceA/MethodA""

requestBase.yml

runners:
  greq:
    addr: ${BASE_GRPC_URL} 
    tls: false 
steps:
  - greq:
      "{{ vars.testPath }}":
        message: "{{ vars.request }}"
    test: compare(current.res.message, vars.response, vars.ignoreField)

When ServiceB is removed from our gRPC server, and the gprc_cli return the service list like following.
Runn tests for ServiceA/MethodA proceed without any issues.

% ./grpc_cli ls localhost:8081
grpc.reflection.v1alpha.ServerReflection
ServiceA

I would greatly appreciate any insights or guidance on this issue.
It's unclear why Runn is requiring ServiceB for tests related to ServiceA.
Is there a potential issue in how Runn handles service dependencies, or could this be related to our specific setup?

[k1LoW]

Are you importing something of ServiceB in the .proto file of ServiceA?
In this case, you need ServiceB information.

[xiao-yao-tn]

Are you importing something of ServiceB in the .proto file of ServiceA? In this case, you need ServiceB information.

There are no dependencies in our .proto definitions between ServiceA and ServiceB.

Also after we removed Service B, we enabled the gRPC health check service, and the same issue happened.

 % ./grpc_cli ls localhost:8081
grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection
ServiceA

1) grpc/tests/ServiceA_test.yml 1c3d98b883cfd72343404d1ed56e5191afe0614b
   └── grpc/tests/requestBase.yml
  Failure/Error: gRPC request failed on "Generated by `runn new`".steps[0]: failed to find descriptor: symbol not found: grpc.health.v1.Health
  Failure step (grpc/tests/requestBase.yml):
  6   - greq:
  7       "{{ vars.testPath }}":
  8         message: "{{ vars.request }}"
  9     test: compare(current.res.message, vars.response, vars.ignoreField)

And FYI I tried to use buf curl with reflect to call ServiceA, everything is OK.
buf curl --protocol grpc --http2-prior-knowledge --reflect "http://localhost:8081/ServiceA/MethodA"

[k1LoW]

@xiao-yao-tn Thank you for your report!

Sounds like a problem with runn's Multi service + Reflection.

[k1LoW]

This error occurs even though ServiceB is not directly involved in the test case for ServiceA/MethodA.

Note that this is because gRPC Runner first performs reflection on all services.

This led me to believe that the information in Service B might be incomplete.

[k1LoW]

@xiao-yao-tn

I tried using grpcbin but could not reproduce it.
Is there any way to reproduce this event on my end?

$ docker run -it --rm -p 9000:9000 -p 9001:9001 moul/grpcbin &
[1] 82024
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
~ > unable to setup input stream: unable to set IO streams as raw terminal: interrupted system call

[1]  + exit 1     docker run -it --rm -p 9000:9000 -p 9001:9001 moul/grpcbin
$ grpcurl -plaintext localhost:9000 list
addsvc.Add
grpc.gateway.examples.examplepb.ABitOfEverythingService
grpc.reflection.v1alpha.ServerReflection
grpcbin.GRPCBin
hello.HelloService
$ grpcurl -plaintext -d '{"greeting": "alice"}' localhost:9000 hello.HelloService/SayHello
{
  "reply": "hello alice"
}
$ runn --version
runn version 0.91.4
~ > cat tmp.yml
desc: grpcbin
runners:
  greq:
    addr: localhost:9000
    tls: false
steps:
- greq:
    hello.HelloService/SayHello:
      message:
        greeting: alice
$ runn run tmp.yml
.

1 scenario, 0 skipped, 0 failures
$