feat!: confirm indexing via Curio piece-status (#689)

* feat: confirm indexing via Curio piece-status instead of filecoinpin.contact

- poll Curio's per-provider piece-status endpoint until synced, then a single
  bounded cid.contact check to confirm the advertisement is actually indexed
- treat a Curio-synced-but-cid.contact-disagrees case as a distinct mismatch,
  not a plain timeout
- checkIpniIndexer's ipniIndexerUrl is now required (previously defaulted to
  filecoinpin.contact, which is being retired)

* fix indexing confirmation gaps found in review

* fix indexer failure classification and provider validation

* fix(ipni): make retry waits abort-aware

* fix(ipni): fail fast on pre-v1.28.6 piece status

* test(ipni): drop alias identity assertion

* refactor(ipni)!: drop waitForIpniProviderResults alias

* refactor(ipni): carry mismatch details on the error

* chore: lint:fix

* Update documentation/content-routing-faq.md

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* Update documentation/content-routing-faq.md

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* Update documentation/behind-the-scenes-of-adding-a-file.md

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* chore: drop orphaned deprecated JSDoc

---------

Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com>
Co-authored-by: Steve Loeppky <biglep@filoz.org>

Author: 3 people

documentation/behind-the-scenes-of-adding-a-file.md

@@ -113,9 +113,9 @@ This is a function of the CAR size and the throughput between the client and the
 
 *What/why:*
 
-At some point after receiving the uploaded [CAR](glossary.md#car), an SP indexing task processes the CAR and creates a local mapping of CIDs to offsets within the CAR so it can serve IPFS style retrievals.  Following that, an SP [IPNI](glossary.md#ipni) tasks picks up the local index, makes and IPNI advertisement chain, and then announces the advertisement chain to IPNI indexers like filecoinpin.contact and cid.contact so they know to come and get the advertisement chain to build up their own index.
+At some point after receiving the uploaded [CAR](glossary.md#car), an SP indexing task processes the CAR and creates a local mapping of CIDs to offsets within the CAR so it can serve IPFS style retrievals.  Following that, an SP [IPNI](glossary.md#ipni) tasks picks up the local index, makes and IPNI advertisement chain, and then announces the advertisement chain to IPNI indexers like cid.contact so they know to come and get the advertisement chain to build up their own index.
 
-Filecoin Pin validates the IPNI advertisement process by polling `https://filecoinpin.contact/cid/$cid` (NOT cid.contact due to [negative caching issues discussed below](#how-long-does-an-ipni-indexer-cache-results)). 
+Filecoin Pin validates the IPNI advertisement process in two steps: first it polls the SP's `GET /pdp/piece/{pieceCid}/status` endpoint until it reports `synced: true`. (Behind the scenes, the SP checks the IPNI instance's own sync-status endpoint on Filecoin Pin's behalf.  This was added in [curio#1450](https://github.com/filecoin-project/curio/pull/1450)). Then, once synced, Filecoin Pin makes a single confirming query to `https://cid.contact/cid/$cid` to verify the expected provider actually shows up. Querying cid.contact directly before that confirmation is avoided due to [negative caching issues](content-routing-faq.md#how-long-does-an-ipni-indexer-cache-results).
 
 *Outputs:*
 

documentation/content-routing-faq.md

@@ -23,16 +23,7 @@ In the event that an SP wipes their existing index state, the previously announc
 
 ## How long does an IPNI indexer cache results?
 
-This depends on both the [IPNI](glossary.md#ipni) indexer instance (e.g., cid.contact, filecoinpin.contact) and whether there is a cache hit or cache miss.
+[cid.contact](http://cid.contact) tends to cache hits for multiple hours and cache misses (negative cache) for minutes. (cid.contact caching teraform is [here](https://github.com/ipni/storetheindex/blob/main/deploy/infrastructure/prod/us-east-2/cloudfront.tf).)  As a result of this, there are "gotchas" we have to be careful to avoid or can unavoidably fall into.
 
-[cid.contact](http://cid.contact) for example tends to cache hits for multiple hours and cache misses (negative cache) for minutes.  As a result of this, there are "gotchas" we have to be careful to avoid or can unavoidably fall into.
-
-- [cid.contact](http://cid.contact) cache miss "gotcha" - Because cid.contact caches misses (i.e., negative cache), it's important for Filecoin Pin to not poll cid.contact after an advertisement has been announced.  The act of polling could cause the empty result set to get cached for minutes.  Instead, Filecoin Pin polls [filecoinpin.contact](http://filecoinpin.contact) which doesn't have negative caching.  Once Filecoin Pin sees the expected results from filecoinpin.contact it then proceeds to give IPFS Mainnet retrieval URLs since it should be safe to invoke a request path that hits cid.contact because cid.contact should now not get a non-empty result.
-- [cid.contact](http://cid.contact) cache hit "gotcha" - If cid.contact has a provider record(s) for CID X, but CID X is not currently from any of those provider(s), then cid.contact could be caching non-retrievable result for hours even though filecoinpin.contact has a provider that makes CID X retrievable. We currently don't have a workaround for this…
-
-## Why is there filecoinpin.contact and cid.contact?
-
-[filecoinpin.contact](http://filecoinpin.contact) serves two purposes currently:
-
-1. Serve as a fallback in case [cid.contact](http://cid.contact) has issues keeping its global index updated.  To help with availability, cid.contact has the ability to delegate requests to other [IPNI](glossary.md#ipni) indexers like [filecoinpin.contact](http://filecoinpin.contact) in case they have results.
-2. Validate IPNI announcing/advertising independently of [cid.contact](http://cid.contact).  Per the "[cid.contact](http://cid.contact) cache miss gotcha" above, the act of polling cid.contact can actually delay how long it takes before cid.contact returns a non-empty result for a given CID.  [filecoinpin.contact](http://filecoinpin.contact) has different caching configuration so that polling can be done safely.
+- cid.contact cache miss "gotcha" - Because cid.contact caches misses (i.e., negative cache), it's important for Filecoin Pin to not poll `GET /cid/{cid}` before the advertisement has actually been processed.  The act of polling could cause the empty result set to get cached for minutes.  Instead, Filecoin Pin first polls the storage provider's `GET /pdp/piece/{pieceCid}/status` until it reports `synced: true`. The SP checks the IPNI instance's own sync-status endpoint on Filecoin Pin's behalf, which isn't a CID lookup and isn't subject to the same negative caching, so it's safe for the SP to do that patiently.  Only once the SP confirms sync does Filecoin Pin make a single confirming `GET /cid/{cid}` query, to confirm the expected provider's record actually shows up.  If a provider never reports `synced: true`, Filecoin Pin reports failure without ever falling back to `GET /cid/{cid}`. The SP already did the patient, safe check on its end, so a further `/cid/{cid}` fallback would be redundant and risk the exact negative-cache problem this whole flow exists to avoid.
+- cid.contact cache hit "gotcha" - If cid.contact has a provider record(s) for CID X, but CID X is not currently retrievable from any of those provider(s), then cid.contact could be caching a non-retrievable result for hours. We currently don't have a workaround for this…

documentation/glossary.md

@@ -124,7 +124,7 @@ As a "trustless" protocol, retrieval of IPFS data using this mechanism provides
 
 See https://docs.ipfs.tech/concepts/glossary/#ipni.
 
-IPNI is the content routing system that [Filecoin Pin](#filecoin-pin) relies upon for retrieval to work for [standard IPFS tooling](#standard-ipfs-tooling).  [Service Providers](#service-provider) announce their advertisement changes to IPNI indexer like [filecoinpin.contact](http://filecoinpin.contact) and cid.contact, and the advertised CIDs become discoverable for IPFS Standard tooling.
+IPNI is the content routing system that [Filecoin Pin](#filecoin-pin) relies upon for retrieval to work for [standard IPFS tooling](#standard-ipfs-tooling).  [Service Providers](#service-provider) announce their advertisement changes to IPNI indexers like cid.contact, and the advertised CIDs become discoverable for IPFS Standard tooling.
 
 ## Metadata
 
@@ -196,7 +196,7 @@ See [Retrieving Your Data](retrieval.md) for how to use each CID to fetch your c
 
 [IPNI](#ipni) advertisements include a `ContextID` that encodes the [Piece CID](#piece-cid). You can use this to reverse-map an IPFS CID back to the Piece it lives in.
 
-1. Look up the IPFS CID in an IPNI indexer, e.g. `https://cid.contact/cid/<ipfs-cid>` (or use [filecoinpin.contact](https://filecoinpin.contact) for data stored via Filecoin Pin).
+1. Look up the IPFS CID in an IPNI indexer, e.g. `https://cid.contact/cid/<ipfs-cid>`.
 2. Find the `ContextID` field in one of the provider records. It is base64-encoded.
 3. Decode the base64, drop the first byte (a version prefix), and treat the remaining bytes as a CID:
 

src/common/upload-flow.ts

@@ -594,6 +594,22 @@ export async function performUpload(
     return `Checking for IPNI provider records (${overallPart}${cidPart})`
   }
 
+  function pieceSyncOpId(providerIndex: number): string {
+    return `piece-sync-${providerIndex}`
+  }
+
+  // Only shown when there's more than one provider — keeps the common single-provider
+  // case identical to the original single-op wording.
+  function providerPrefix(providerIndex: number, providerCount: number): string {
+    return providerCount > 1 ? `[${providerIndex}/${providerCount}] ` : ''
+  }
+
+  function discardPieceSyncOps(providerCount: number): void {
+    for (let i = 1; i <= providerCount; i++) {
+      flow.discardOperation(pieceSyncOpId(i))
+    }
+  }
+
   const network = getNetworkSlug(synapse.chain)
 
   const uploadResult = await executeUpload(synapse, carData, rootCid, {
@@ -683,6 +699,56 @@ export async function performUpload(
           break
         }
 
+        case 'pieceSyncStatus:retryUpdate': {
+          const { serviceURL, providerIndex, providerCount, providerAttempt, providerMaxAttempts } = event.data
+          const suffix = providerCount > 1 ? ` on ${serviceURL}` : ''
+          flow.addOperation(
+            pieceSyncOpId(providerIndex),
+            `${providerPrefix(providerIndex, providerCount)}Waiting for advertisement to be indexed${suffix} (attempt ${providerAttempt}/${providerMaxAttempts})`
+          )
+          break
+        }
+        case 'pieceSyncStatus:providerSynced': {
+          const { serviceURL, providerIndex, providerCount } = event.data
+          // Single-provider case: leave the op open — pieceSyncStatus:complete closes it below.
+          if (providerCount > 1) {
+            flow.completeOperation(
+              pieceSyncOpId(providerIndex),
+              `${providerPrefix(providerIndex, providerCount)}Advertisement confirmed indexed on ${serviceURL}`,
+              { type: 'success' }
+            )
+          }
+          break
+        }
+        case 'pieceSyncStatus:complete': {
+          if (event.data.providerCount > 1) {
+            discardPieceSyncOps(event.data.providerCount)
+            flow.printSection(
+              pc.green(`✓ Advertisement confirmed indexed on all ${event.data.providerCount} providers`),
+              []
+            )
+          } else {
+            flow.completeOperation(pieceSyncOpId(1), 'Advertisement confirmed indexed', { type: 'success' })
+          }
+          break
+        }
+        case 'pieceSyncStatus:failed': {
+          if (event.data.providerCount > 1) {
+            discardPieceSyncOps(event.data.providerCount)
+            flow.printSection(pc.yellow('⚠ Advertisement not confirmed indexed in time.'), [
+              pc.gray(event.data.error.message),
+            ])
+          } else {
+            flow.completeOperation(pieceSyncOpId(1), 'Advertisement not confirmed indexed in time.', {
+              type: 'warning',
+              details: {
+                title: 'Reason',
+                content: [pc.gray(event.data.error.message)],
+              },
+            })
+          }
+          break
+        }
         case 'ipniProviderResults:retryUpdate': {
           const attempt = event.data.attempt ?? (event.data.retryCount === 0 ? 1 : event.data.retryCount + 1)
           flow.addOperation(
@@ -718,6 +784,16 @@ export async function performUpload(
           })
           break
         }
+        case 'indexingConfirmation:mismatch': {
+          // The underlying ipniProviderResults:failed is suppressed for this call, so
+          // 'ipni' never gets completed on its own — discard it before reporting here.
+          flow.discardOperation('ipni')
+          flow.printSection(
+            pc.yellow('⚠ Storage provider reported sync, but a direct indexer lookup still disagrees'),
+            [pc.gray(event.data.error.message)]
+          )
+          break
+        }
         default: {
           break
         }

src/core/upload/index.ts

@@ -1,4 +1,4 @@
-import type { FilecoinChain, PDPProvider, Synapse } from '@filoz/synapse-sdk'
+import type { FilecoinChain, PDPProvider, PieceCID, Synapse } from '@filoz/synapse-sdk'
 import { calibration, mainnet } from '@filoz/synapse-sdk'
 import type { StorageContext } from '@filoz/synapse-sdk/storage'
 import type { CID } from 'multiformats/cid'
@@ -19,9 +19,9 @@ import { isSessionKeyMode } from '../synapse/index.js'
 import { recordUploadResult } from '../telemetry/index.js'
 import type { ProgressEvent, ProgressEventHandler } from '../utils/types.js'
 import {
-  type ValidateIPNIProgressEvents,
-  type WaitForIpniProviderResultsOptions,
-  waitForIpniProviderResults,
+  type IndexingConfirmationProgressEvents,
+  type WaitForIndexingConfirmationOptions,
+  waitForIndexingConfirmation,
 } from '../utils/validate-ipni-advertisement.js'
 import {
   type SynapseUploadData,
@@ -232,7 +232,7 @@ export interface UploadExecutionOptions {
   /** Optional identifier to help correlate logs. */
   contextId?: string
   /** Optional umbrella onProgress receiving child progress events. */
-  onProgress?: ProgressEventHandler<(UploadProgressEvents | ValidateIPNIProgressEvents) & {}>
+  onProgress?: ProgressEventHandler<(UploadProgressEvents | IndexingConfirmationProgressEvents) & {}>
   /** Optional metadata to associate with the upload (per-piece). */
   pieceMetadata?: Record<string, string>
   /**
@@ -250,7 +250,7 @@ export interface UploadExecutionOptions {
      * @default: true
      */
     enabled?: boolean
-  } & Omit<WaitForIpniProviderResultsOptions, 'onProgress'>
+  } & Omit<WaitForIndexingConfirmationOptions, 'onProgress'>
 
   /** Number of storage copies to create (default determined by SDK). */
   copies?: number
@@ -306,7 +306,8 @@ export interface UploadExecutionResult extends SynapseUploadResult {
   /** Active network derived from the Synapse instance. */
   network: string
   /**
-   * True if the IPFS Root CID was observed on filecoinpin.contact (IPNI).
+   * True if the IPNI indexer confirmed the piece's advertisement(s) as indexed and
+   * the IPFS Root CID was confirmed present on the indexer.
    *
    * You should block any displaying, or attempting to access, of IPFS
    * download URLs unless the IPNI validation is successful.
@@ -349,27 +350,37 @@ export async function executeUpload(
     )
   }
 
-  // Collect providers from `providerSelected` events for IPNI validation
-  const selectedProviders: PDPProvider[] = []
+  // pieceCid is identical across providers for the same upload, so any `stored` event works.
+  // contexts carry their provider directly — no providerSelected event fires for them.
+  const selectedProviders: PDPProvider[] = options.contexts?.map((context) => context.provider) ?? []
+  let pieceCidForValidation: PieceCID | undefined
   let ipniValidationPromise: Promise<boolean> | undefined
 
-  const emitProgress: ProgressEventHandler<UploadProgressEvents | ValidateIPNIProgressEvents> = (event) => {
+  const emitProgress: ProgressEventHandler<UploadProgressEvents | IndexingConfirmationProgressEvents> = (event) => {
     switch (event.type) {
       case 'providerSelected': {
         selectedProviders.push(event.data.provider)
         break
       }
+      case 'stored': {
+        pieceCidForValidation ??= event.data.pieceCid
+        break
+      }
       case 'piecesAdded': {
-        // Begin IPNI validation on the first piecesAdded event
+        // Begin indexing confirmation on the first piecesAdded event
         if (options.ipniValidation?.enabled !== false && ipniValidationPromise == null) {
+          if (pieceCidForValidation == null) {
+            logger.warn('piecesAdded fired before stored; skipping indexing confirmation (no pieceCid to poll)')
+            break
+          }
           const {
             enabled: _enabled,
             expectedProviders,
             signal: ipniSignal,
             ...restOptions
           } = options.ipniValidation ?? {}
 
-          const validationOptions: WaitForIpniProviderResultsOptions = {
+          const validationOptions: WaitForIndexingConfirmationOptions = {
             ...restOptions,
             logger,
             signal: ipniSignal ?? options.signal,
@@ -379,18 +390,20 @@ export async function executeUpload(
             validationOptions.onProgress = options.onProgress
           }
 
-          // Use providers collected from selection events for IPNI validation
+          // Use providers collected from selection events for indexing confirmation
           if (expectedProviders != null) {
             validationOptions.expectedProviders = expectedProviders
           } else if (selectedProviders.length > 0) {
             validationOptions.expectedProviders = selectedProviders
           }
 
-          ipniValidationPromise = waitForIpniProviderResults(rootCid, validationOptions).catch((error) => {
-            validationOptions.signal?.throwIfAborted()
-            logger.warn({ error }, 'IPNI provider results check was rejected')
-            return false
-          })
+          ipniValidationPromise = waitForIndexingConfirmation(rootCid, pieceCidForValidation, validationOptions).catch(
+            (error) => {
+              validationOptions.signal?.throwIfAborted()
+              logger.warn({ error }, 'Indexing confirmation check was rejected')
+              return false
+            }
+          )
         }
         break
       }
@@ -447,7 +460,7 @@ export async function executeUpload(
       ipniValidated = await ipniValidationPromise
     } catch (error) {
       options.signal?.throwIfAborted()
-      logger.error({ error }, 'Could not validate IPNI provider records')
+      logger.error({ error }, 'Could not confirm indexing')
       ipniValidated = false
     }
   }