Skip to content

Owner Resource Not Found (Gateway V2 Connection Mode and Direct Connection Mode): Remaps sub-status to 1003 for requests to child resources against non-existent container. #47604

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jeet1995 wants to merge 22 commits into
base: main
Choose a base branch
from

Addressing review comments.

c432e44
Select commit
Loading
Failed to load commit list.
Open

Owner Resource Not Found (Gateway V2 Connection Mode and Direct Connection Mode): Remaps sub-status to 1003 for requests to child resources against non-existent container. #47604

Addressing review comments.
c432e44
Select commit
Loading
Failed to load commit list.
Azure Pipelines / java - cosmos - ci succeeded Jan 10, 2026 in 1h 2m 46s

Build #20260109.4 had test failures

2 errors / 7,413 warnings

Details

Tests

  • Failed: 4 (0.01%)
  • Passed: 77,338 (99.06%)
  • Other: 727 (0.93%)
  • Total: 78,069
Code coverage

  • 2397 of 80592 lines covered (2.97%)

Annotations

Check failure on line 25 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

Build log #L25 

PowerShell exited with code '1'.

Check failure on line 25 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

Build log #L25 

PowerShell exited with code '1'.

Check failure on line 1 in c.a.c.i.c.e.PartitionControllerImplTests.handleMerge

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

c.a.c.i.c.e.PartitionControllerImplTests.handleMerge 

leaseManager.acquire(
    TestLease-8f468008-78bb-440d-9745-2d5b82c5f1d6 Owner='null' Continuation=null Version=EPK_RANGE_BASED_LEASE FeedRange=eyJSYW5nZSI6eyJtaW4iOiJBQSIsIm1heCI6IkNDIn19 Timestamp(local)=2026-01-09T23:39:12.360612908Z Timestamp(server)=1970-01-01T00:00:12Z[UTC]
);
Wanted 1 time:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204)
But was 2 times:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)
Raw output 
org.mockito.exceptions.verification.TooManyActualInvocations: 

leaseManager.acquire(
    TestLease-8f468008-78bb-440d-9745-2d5b82c5f1d6 Owner='null' Continuation=null Version=EPK_RANGE_BASED_LEASE FeedRange=eyJSYW5nZSI6eyJtaW4iOiJBQSIsIm1heCI6IkNDIn19 Timestamp(local)=2026-01-09T23:39:12.360612908Z Timestamp(server)=1970-01-01T00:00:12Z[UTC]
);
Wanted 1 time:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204)
But was 2 times:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)


	at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:136)
	at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:658)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:219)
	at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:923)
	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:192)
	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.testng.TestRunner.privateRun(TestRunner.java:808)
	at org.testng.TestRunner.run(TestRunner.java:603)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:429)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:423)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:383)
	at org.testng.SuiteRunner.run(SuiteRunner.java:326)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1249)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
	at org.testng.TestNG.runSuites(TestNG.java:1092)
	at org.testng.TestNG.run(TestNG.java:1060)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:308)
	at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:71)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)

Check failure on line 1 in c.a.c.i.c.e.PartitionControllerImplTests.handleMerge

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

c.a.c.i.c.e.PartitionControllerImplTests.handleMerge 

 leaseManager.acquire(     TestLease-8f468008-78bb-440d-9745-2d5b82c5f1d6 Owner='null' Continuation=null Version=EPK_RANGE_BASED_LEASE FeedRange=eyJSYW5nZSI6eyJtaW4iOiJBQSIsIm1heCI6IkNDIn19 Timestamp(local)=2026-01-09T23:39:12.360612908Z Timestamp(server)=1970-01-01T00:00:12Z[UTC] ); Wanted 1 time: -> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204) But was 2 times: -> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70) -> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)
Raw output 
org.mockito.exceptions.verification.TooManyActualInvocations: 
leaseManager.acquire(
    TestLease-8f468008-78bb-440d-9745-2d5b82c5f1d6 Owner='null' Continuation=null Version=EPK_RANGE_BASED_LEASE FeedRange=eyJSYW5nZSI6eyJtaW4iOiJBQSIsIm1heCI6IkNDIn19 Timestamp(local)=2026-01-09T23:39:12.360612908Z Timestamp(server)=1970-01-01T00:00:12Z[UTC]
);
Wanted 1 time:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204)
But was 2 times:
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)
-> at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImpl.addOrUpdateLease(PartitionControllerImpl.java:70)


at com.azure.cosmos.implementation.changefeed.epkversion.PartitionControllerImplTests.handleMerge(PartitionControllerImplTests.java:204)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:136)
at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:658)
at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:219)
at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:923)
at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:192)
at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:808)
at org.testng.TestRunner.run(TestRunner.java:603)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:429)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:423)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:383)
at org.testng.SuiteRunner.run(SuiteRunner.java:326)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1249)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
at org.testng.TestNG.runSuites(TestNG.java:1092)
at org.testng.TestNG.run(TestNG.java:1060)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:308)
at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:71)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495)

Check failure on line 1 in c.a.c.s.ChangeFeedPartitionReaderITest.change feed partition reader should honor endLSN during split and should hang

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

c.a.c.s.ChangeFeedPartitionReaderITest.change feed partition reader should honor endLSN during split and should hang 

false did not equal true
Raw output 
      org.scalatest.exceptions.TestFailedException: false did not equal true
      at org.scalatest.matchers.MatchersHelper$.indicateFailure(MatchersHelper.scala:344)
      at org.scalatest.matchers.should.Matchers$AnyShouldWrapper.shouldEqual(Matchers.scala:6860)
      at com.azure.cosmos.spark.ChangeFeedPartitionReaderITest.$anonfun$new$3(ChangeFeedPartitionReaderITest.scala:225)
      at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
      at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
      at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
      at org.scalatest.Transformer.apply(Transformer.scala:22)
      at org.scalatest.Transformer.apply(Transformer.scala:20)
      at org.scalatest.flatspec.AnyFlatSpecLike$$anon$5.apply(AnyFlatSpecLike.scala:1684)
      at org.scalatest.TestSuite.withFixture(TestSuite.scala:196)
      at org.scalatest.TestSuite.withFixture$(TestSuite.scala:195)
      at org.scalatest.flatspec.AnyFlatSpec.withFixture(AnyFlatSpec.scala:1685)
      at org.scalatest.flatspec.AnyFlatSpecLike.invokeWithFixture$1(AnyFlatSpecLike.scala:1682)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTest$1(AnyFlatSpecLike.scala:1694)
      at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTest(AnyFlatSpecLike.scala:1694)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTest$(AnyFlatSpecLike.scala:1676)
      at com.azure.cosmos.spark.IntegrationSpec.org$scalatest$BeforeAndAfterEach$$super$runTest(IntegrationSpec.scala:10)
      at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
      at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
      at com.azure.cosmos.spark.IntegrationSpec.runTest(IntegrationSpec.scala:10)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$runTests$1(AnyFlatSpecLike.scala:1752)
      at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
      at scala.collection.immutable.List.foreach(List.scala:431)
      at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
      at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:390)
      at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:427)
      at scala.collection.immutable.List.foreach(List.scala:431)
      at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
      at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
      at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTests(AnyFlatSpecLike.scala:1752)
      at org.scalatest.flatspec.AnyFlatSpecLike.runTests$(AnyFlatSpecLike.scala:1751)
      at org.scalatest.flatspec.AnyFlatSpec.runTests(AnyFlatSpec.scala:1685)
      at org.scalatest.Suite.run(Suite.scala:1112)
      at org.scalatest.Suite.run$(Suite.scala:1094)
      at org.scalatest.flatspec.AnyFlatSpec.org$scalatest$flatspec$AnyFlatSpecLike$$super$run(AnyFlatSpec.scala:1685)
      at org.scalatest.flatspec.AnyFlatSpecLike.$anonfun$run$1(AnyFlatSpecLike.scala:1797)
      at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
      at org.scalatest.flatspec.AnyFlatSpecLike.run(AnyFlatSpecLike.scala:1797)
      at org.scalatest.flatspec.AnyFlatSpecLike.run$(AnyFlatSpecLike.scala:1795)
      at com.azure.cosmos.spark.IntegrationSpec.org$scalatest$BeforeAndAfterAll$$super$run(IntegrationSpec.scala:10)
      at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
      at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
      at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
      at com.azure.cosmos.spark.IntegrationSpec.run(IntegrationSpec.scala:10)
      at org.scalatest.Suite.callExecuteOnSuite$1(Suite.scala:1175)
      at org.scalatest.Suite.$anonfun$runNestedSuites$1(Suite.scala:1222)
      at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
      at scala.collection.Ind

Check failure on line 1 in c.a.c.s.PointWriterITest.Point Writer can create item with duplicates

See this annotation in the file changed.

@azure-pipelines azure-pipelines / java - cosmos - ci

c.a.c.s.PointWriterITest.Point Writer can create item with duplicates 

{"innerErrorMessage":"{\"innerErrorMessage\":\"ChannelHandlerContext(RntbdRequestManager#0, [id: 0x42ecb5bb, L:/127.0.0.1:58389 - R:/127.0.0.1:10253]) closed exceptionally with 1 pending requests\"}","cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.77.0-snapshot.1 WindowsServer2022/10.0 JRE/11.0.29","activityId":"fd7ac986-417a-4d53-ac70-df8a33cedd7f","requestLatencyInMs":108,"requestStartTimeUTC":"2026-01-09T23:58:25.885791800Z","requestEndTimeUTC":"2026-01-09T23:58:25.994465900Z","responseStatisticsList":[{"storeResult":{"storePhysicalAddress":"rntbd://127.0.0.1:10253/apps/DocDbApp/services/DocDbServer14/partitions/a4cb495a-38c8-11e6-8106-8cdcd42c33be/replicas/1p/","lsn":0,"quorumAckedLSN":-1,"currentReplicaSetSize":-1,"globalCommittedLsn":-1,"partitionKeyRangeId":null,"isValid":true,"statusCode":410,"subStatusCode":0,"isGone":true,"isNotFound":false,"isInvalidPartition":false,"isThroughputControlRequestRateTooLarge":false,"requestCharge":0.0,"itemLSN":-1,"sessionToken":null,"backendLatencyInMs":null,"retryAfterInMs":null,"exceptionMessage":"ChannelHandlerContext(RntbdRequestManager#0, [id: 0x42ecb5bb, L:/127.0.0.1:58389 - R:/127.0.0.1:10253]) closed exceptionally with 1 pending requests","exceptionResponseHeaders":"{x-ms-substatus=0}","replicaStatusList":{"Ignoring":["10253:S:Connected"],"Attempting":["10253:P:Connected"]},"transportRequestTimeline":[{"eventName":"created","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":0.0},{"eventName":"queued","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":0.0},{"eventName":"channelAcquisitionStarted","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":54.6745},{"eventName":"pipelined","startTimeUTC":"2026-01-09T23:58:25.940466300Z","durationInMilliSecs":49.0},{"eventName":"transitTime","startTimeUTC":"2026-01-09T23:58:25.989466300Z","durationInMilliSecs":3.9937},{"eventName":"decodeTime","startTimeUTC":null,"durationInMilliSecs":0.0},{"eventName":"received","startTimeUTC":null,"durationInMilliSecs":0.0},{"eventName":"completed","startTimeUTC":"2026-01-09T23:58:25.993460Z","durationInMilliSecs":0.0}],"rntbdRequestLengthInBytes":510,"rntbdResponseLengthInBytes":-1,"requestPayloadLengthInBytes":103,"responsePayloadLengthInBytes":-1,"channelStatistics":{"channelId":"42ecb5bb","channelTaskQueueSize":0,"pendingRequestsCount":0,"lastReadTime":"2026-01-09T23:58:25.940466300Z","waitForConnectionInit":false},"serviceEndpointStatistics":{"availableChannels":37,"acquiredChannels":1,"executorTaskQueueSize":96,"inflightRequests":126,"lastSuccessfulRequestTime":"2026-01-09T23:58:25.885Z","lastRequestTime":"2026-01-09T23:58:25.887Z","createdTime":"2026-01-09T23:57:59.250646400Z","isClosed":false,"cerMetrics":{}}},"requestResponseTimeUTC":"2026-01-09T23:58:25.994465900Z","requestStartTimeUTC":"2026-01-09T23:58:25.885791800Z","requestResourceType":"Document","requestOperationType":"Create","requestSessionToken":null,"e2ePolicyCfg":null,"excludedRegions":null,"sessionTokenEvaluationResults":[],"perPartitionCircuitBreakerInfoHolder":null,"perPartitionFailoverInfoHolder":null}],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{"0f208c54-6698-4eff-9912-e79923919491":{"startTimeUTC":"2026-01-09T23:58:26.001464800Z","endTimeUTC":null,"targetEndpoint":"https://127.0.0.1:8081/addresses/?$resolveFor=dbs%2F3%2BIpAA%3D%3D%2Fcolls%2F3%2BIpAL4l-ck%3D%2Fdocs&$filter=protocol%20eq%20rntbd&$partitionKeyRangeIds=1","exceptionMessage":null,"forceRefresh":true,"forceCollectionRoutingMapRefresh":false,"inflightRequest":true}},"regionsContacted":["south central us"],"retryContext":{"statusAndSubStatusCodes":[[410,0],[408,0]],"retryCount":2,"retryLatency":7},"metadataDiagnosticsContext":{"metadataDiagnosticList":null,"empty":true},"serializationDiagnosticsContext":{"serializationDiagnosticsList":null},"gatewayStatisticsList":[],"samplingRateSnapshot":1.0,"bloomFilterInsertionCountSnapshot":0,"systemInformation":{"usedMemory":"292251 KB","a
Raw output 
      {"ClassName":"CosmosException","userAgent":"azsdk-java-cosmos/4.77.0-snapshot.1 WindowsServer2022/10.0 JRE/11.0.29","statusCode":408,"resourceAddress":null,"innerErrorMessage":"{\"innerErrorMessage\":\"ChannelHandlerContext(RntbdRequestManager#0, [id: 0x42ecb5bb, L:/127.0.0.1:58389 - R:/127.0.0.1:10253]) closed exceptionally with 1 pending requests\"}","causeInfo":"[class: class com.azure.cosmos.implementation.GoneException, message: {\"innerErrorMessage\":\"ChannelHandlerContext(RntbdRequestManager#0, [id: 0x42ecb5bb, L:/127.0.0.1:58389 - R:/127.0.0.1:10253]) closed exceptionally with 1 pending requests\"}]","responseHeaders":"{x-ms-substatus=0}","cosmosDiagnostics":{"userAgent":"azsdk-java-cosmos/4.77.0-snapshot.1 WindowsServer2022/10.0 JRE/11.0.29","activityId":"fd7ac986-417a-4d53-ac70-df8a33cedd7f","requestLatencyInMs":176,"requestStartTimeUTC":"2026-01-09T23:58:25.885791800Z","requestEndTimeUTC":"2026-01-09T23:58:26.062467500Z","responseStatisticsList":[{"storeResult":{"storePhysicalAddress":"rntbd://127.0.0.1:10253/apps/DocDbApp/services/DocDbServer14/partitions/a4cb495a-38c8-11e6-8106-8cdcd42c33be/replicas/1p/","lsn":0,"quorumAckedLSN":-1,"currentReplicaSetSize":-1,"globalCommittedLsn":-1,"partitionKeyRangeId":null,"isValid":true,"statusCode":410,"subStatusCode":0,"isGone":true,"isNotFound":false,"isInvalidPartition":false,"isThroughputControlRequestRateTooLarge":false,"requestCharge":0.0,"itemLSN":-1,"sessionToken":null,"backendLatencyInMs":null,"retryAfterInMs":null,"exceptionMessage":"ChannelHandlerContext(RntbdRequestManager#0, [id: 0x42ecb5bb, L:/127.0.0.1:58389 - R:/127.0.0.1:10253]) closed exceptionally with 1 pending requests","exceptionResponseHeaders":"{x-ms-substatus=0}","replicaStatusList":{"Ignoring":["10253:S:Connected"],"Attempting":["10253:P:Connected"]},"transportRequestTimeline":[{"eventName":"created","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":0.0},{"eventName":"queued","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":0.0},{"eventName":"channelAcquisitionStarted","startTimeUTC":"2026-01-09T23:58:25.885791800Z","durationInMilliSecs":54.6745},{"eventName":"pipelined","startTimeUTC":"2026-01-09T23:58:25.940466300Z","durationInMilliSecs":49.0},{"eventName":"transitTime","startTimeUTC":"2026-01-09T23:58:25.989466300Z","durationInMilliSecs":3.9937},{"eventName":"decodeTime","startTimeUTC":null,"durationInMilliSecs":0.0},{"eventName":"received","startTimeUTC":null,"durationInMilliSecs":0.0},{"eventName":"completed","startTimeUTC":"2026-01-09T23:58:25.993460Z","durationInMilliSecs":0.0}],"rntbdRequestLengthInBytes":510,"rntbdResponseLengthInBytes":-1,"requestPayloadLengthInBytes":103,"responsePayloadLengthInBytes":-1,"channelStatistics":{"channelId":"42ecb5bb","channelTaskQueueSize":0,"pendingRequestsCount":0,"lastReadTime":"2026-01-09T23:58:25.940466300Z","waitForConnectionInit":false},"serviceEndpointStatistics":{"availableChannels":37,"acquiredChannels":1,"executorTaskQueueSize":96,"inflightRequests":126,"lastSuccessfulRequestTime":"2026-01-09T23:58:25.885Z","lastRequestTime":"2026-01-09T23:58:25.887Z","createdTime":"2026-01-09T23:57:59.250646400Z","isClosed":false,"cerMetrics":{}}},"requestResponseTimeUTC":"2026-01-09T23:58:25.994465900Z","requestStartTimeUTC":"2026-01-09T23:58:25.885791800Z","requestResourceType":"Document","requestOperationType":"Create","requestSessionToken":null,"e2ePolicyCfg":null,"excludedRegions":null,"sessionTokenEvaluationResults":[],"perPartitionCircuitBreakerInfoHolder":null,"perPartitionFailoverInfoHolder":null}],"supplementalResponseStatisticsList":[],"addressResolutionStatistics":{"0f208c54-6698-4eff-9912-e79923919491":{"startTimeUTC":"2026-01-09T23:58:26.001464800Z","endTimeUTC":"2026-01-09T23:58:26.062467500Z","targetEndpoint":"https://127.0.0.1:8081/addresses/?$resolveFor=dbs%2F3%2BIpAA%3D%3D%2Fcolls%2F3%2BIpAL4l-ck%3D%2Fdocs&$filter=protocol%20eq%20rntbd&$partitionKeyRangeIds=1","exceptionMessage":null,"forceRefresh"
View more details on Azure Pipelines