Introduce type InvalidationDataInfoOrFuture to replace InvalidationDa…

…taReference.

This change will replace `InvalidationDataReference` with a new type
hierarchy, `InvalidationDataInfoOrFuture`, to manage information about
remotely stored invalidation data for files, directory listings, and
nested file system operation nodes.

`InvalidationDataInfoOrFuture` is not hooked up to anything in this
change but broken out for ease of review.

**Motivation:**

The previous `InvalidationDataReference` model relied on immediate
availability of Maximum Transitive Source Versions (MTSVs) from Skyframe,
however, that approach creates too much coupling. Computing MTSVs will
now require a full traversal of the Skyframe dependency graph. This
traversal is parallelized, leading to potential race conditions where
multiple threads might attempt to compute the same value redundantly.

**Solution:**

The `InvalidationDataInfoOrFuture` hierarchy addresses this by:

1. **Memoization:** Employing `SettableFutureKeyedValue` to memoize the
results of the parallelized MTSV computation, preventing redundant work.

2. **Type Safety:** Introducing a type system that clearly distinguishes
between constant (no persisted data), value (persisted data with cache
key and write status), and future (in-progress computation) states for
each of the three basic types: files, directory listings, and nodes
(nested sets of files and directory listings). This improved type safety
allows for more precise handling of invalidation data.

**Details:**

- The hierarchy uses sealed interfaces to represent the different states
  and types of invalidation data.

- `FileDataInfo` includes the MTSV and resolved real path for ancestor
  resolution and symlink handling.

- `Constant*` types (e.g., `ConstantFileData`) indicate that no
  invalidation data is associated, enabling a collection type declaration
  to simply exclude (uninformative) constants.

`InvalidationDataReference` will be removed in a subsequent change.

PiperOrigin-RevId: 716224019
Change-Id: I6ee05721ff8a220cce0643a34d5c8a53a5ba4d0f

src/main/java/com/google/devtools/build/lib/skyframe/serialization/analysis/BUILD

@@ -105,6 +105,18 @@ java_library(
     ],
 )
 
+java_library(
+    name = "invalidation_data_reference_or_future",
+    srcs = ["InvalidationDataInfoOrFuture.java"],
+    deps = [
+        "//src/main/java/com/google/devtools/build/lib/concurrent:settable_future_keyed_value",
+        "//src/main/java/com/google/devtools/build/lib/skyframe:filesystem_keys",
+        "//src/main/java/com/google/devtools/build/lib/skyframe/serialization",
+        "//src/main/java/com/google/devtools/build/lib/vfs",
+        "//third_party:guava",
+    ],
+)
+
 java_library(
     name = "file_dependency_deserializer",
     srcs = [

src/main/java/com/google/devtools/build/lib/skyframe/serialization/analysis/InvalidationDataInfoOrFuture.java

@@ -0,0 +1,183 @@
+// Copyright 2024 The Bazel Authors. All rights reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//    http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package com.google.devtools.build.lib.skyframe.serialization.analysis;
+
+import com.google.common.util.concurrent.ListenableFuture;
+import com.google.devtools.build.lib.concurrent.SettableFutureKeyedValue;
+import com.google.devtools.build.lib.skyframe.DirectoryListingKey;
+import com.google.devtools.build.lib.skyframe.FileKey;
+import com.google.devtools.build.lib.skyframe.NestedFileSystemOperationNodes;
+import com.google.devtools.build.lib.skyframe.serialization.PackedFingerprint;
+import com.google.devtools.build.lib.vfs.RootedPath;
+import java.util.function.BiConsumer;
+
+/**
+ * Information about remotely stored invalidation data.
+ *
+ * <p>There are 3 distinct type families, associated with files, directory listings and nodes
+ * (nested sets of files and directory listings).
+ *
+ * <p>Each family has 3 types, a constant type (no persisted data), information about stored
+ * invalidation data or a future. Information about stored invalidation data always includes a cache
+ * key and a {@link ListenableFuture<Void>} write status.
+ *
+ * <p>In the case of {@link FileInvalidationData}, a Max Transitive Source Version (MTSV) and fully
+ * resolved path is also included. These fields are used for ancestor resolution.
+ */
+@SuppressWarnings("InterfaceWithOnlyStatics") // sealed hierarchy root
+sealed interface InvalidationDataInfoOrFuture
+    permits InvalidationDataInfoOrFuture.InvalidationDataInfo,
+        InvalidationDataInfoOrFuture.FileDataInfoOrFuture,
+        InvalidationDataInfoOrFuture.ListingDataInfoOrFuture,
+        InvalidationDataInfoOrFuture.NodeDataInfoOrFuture {
+
+  /** Non-future, immediate value sub-types of {@link InvalidationDataInfoOrFuture}. */
+  sealed interface InvalidationDataInfo extends InvalidationDataInfoOrFuture
+      permits FileDataInfo, ListingDataInfo, NodeDataInfo {}
+
+  /** Base implementation of a {@link InvalidationDataInfoOrFuture} value. */
+  abstract static sealed class BaseInvalidationDataInfo<T> {
+    private final T cacheKey;
+    private final ListenableFuture<Void> writeStatus;
+
+    BaseInvalidationDataInfo(T cacheKey, ListenableFuture<Void> writeStatus) {
+      this.cacheKey = cacheKey;
+      this.writeStatus = writeStatus;
+    }
+
+    /** Key for {@link com.google.devtools.build.lib.serialization.FingerprintValueService}. */
+    final T cacheKey() {
+      return cacheKey;
+    }
+
+    /** Transitively inclusive status of writing this data to the cache. */
+    final ListenableFuture<Void> writeStatus() {
+      return writeStatus;
+    }
+  }
+
+  sealed interface FileDataInfoOrFuture extends InvalidationDataInfoOrFuture
+      permits FileDataInfo, FutureFileDataInfo {}
+
+  sealed interface FileDataInfo extends FileDataInfoOrFuture, InvalidationDataInfo
+      permits ConstantFileData, FileInvalidationDataInfo {}
+
+  /** The file doesn't change and isn't associated with any invalidation data. */
+  enum ConstantFileData implements FileDataInfo {
+    CONSTANT_FILE;
+  }
+
+  /** Information about transitive upload of invalidation data for a certain {@link FileKey}. */
+  static final class FileInvalidationDataInfo extends BaseInvalidationDataInfo<String>
+      implements FileDataInfo {
+    private final long mtsv;
+    private final RootedPath realPath;
+
+    FileInvalidationDataInfo(
+        String cacheKey, ListenableFuture<Void> writeStatus, long mtsv, RootedPath realPath) {
+      super(cacheKey, writeStatus);
+      this.mtsv = mtsv;
+      this.realPath = realPath;
+    }
+
+    /**
+     * The MTSV.
+     *
+     * <p>Used by dependents to create parent references. This information is already incorporated
+     * into the {@link #cacheKey} value.
+     */
+    long mtsv() {
+      return mtsv;
+    }
+
+    /**
+     * The resolved real path.
+     *
+     * <p>Used for symlink resolution.
+     */
+    RootedPath realPath() {
+      return realPath;
+    }
+  }
+
+  static final class FutureFileDataInfo
+      extends SettableFutureKeyedValue<FutureFileDataInfo, FileKey, FileDataInfo>
+      implements FileDataInfoOrFuture {
+    FutureFileDataInfo(FileKey key, BiConsumer<FileKey, FileDataInfo> consumer) {
+      super(key, consumer);
+    }
+  }
+
+  sealed interface ListingDataInfoOrFuture extends InvalidationDataInfoOrFuture
+      permits ListingDataInfo, FutureListingDataInfo {}
+
+  sealed interface ListingDataInfo extends ListingDataInfoOrFuture, InvalidationDataInfo
+      permits ConstantListingData, ListingInvalidationDataInfo {}
+
+  /** This listing doesn't change and isn't associated with invalidation data. */
+  enum ConstantListingData implements ListingDataInfo {
+    CONSTANT_LISTING;
+  }
+
+  /**
+   * Information about transitive upload of invalidation data for a certain {@link
+   * DirectoryListingKey}.
+   */
+  static final class ListingInvalidationDataInfo extends BaseInvalidationDataInfo<String>
+      implements ListingDataInfo {
+    ListingInvalidationDataInfo(String cacheKey, ListenableFuture<Void> writeStatus) {
+      super(cacheKey, writeStatus);
+    }
+  }
+
+  static final class FutureListingDataInfo
+      extends SettableFutureKeyedValue<FutureListingDataInfo, DirectoryListingKey, ListingDataInfo>
+      implements ListingDataInfoOrFuture {
+    FutureListingDataInfo(
+        DirectoryListingKey key, BiConsumer<DirectoryListingKey, ListingDataInfo> consumer) {
+      super(key, consumer);
+    }
+  }
+
+  sealed interface NodeDataInfoOrFuture extends InvalidationDataInfoOrFuture
+      permits NodeDataInfo, FutureNodeDataInfo {}
+
+  sealed interface NodeDataInfo extends NodeDataInfoOrFuture, InvalidationDataInfo
+      permits ConstantNodeData, NodeInvalidationDataInfo {}
+
+  enum ConstantNodeData implements NodeDataInfo {
+    CONSTANT_NODE;
+  }
+
+  /** Information about remotely persisted {@link NestedFileSystemOperationNodes}. */
+  static final class NodeInvalidationDataInfo extends BaseInvalidationDataInfo<PackedFingerprint>
+      implements NodeDataInfo {
+    NodeInvalidationDataInfo(PackedFingerprint key, ListenableFuture<Void> writeStatus) {
+      super(key, writeStatus);
+    }
+  }
+
+  static final class FutureNodeDataInfo
+      extends SettableFutureKeyedValue<
+          FutureNodeDataInfo, NestedFileSystemOperationNodes, NodeDataInfo>
+      implements NodeDataInfoOrFuture {
+    FutureNodeDataInfo(NestedFileSystemOperationNodes key) {
+      super(key, FutureNodeDataInfo::setNodeDataInfo);
+    }
+
+    private static void setNodeDataInfo(NestedFileSystemOperationNodes key, NodeDataInfo value) {
+      key.setSerializationScratch(value);
+    }
+  }
+}