oppia · BenHenning · Aug 6, 2021 · Aug 6, 2021 · Aug 10, 2021 · Aug 10, 2021
diff --git a/BUILD b/BUILD
@@ -0,0 +1,35 @@
+"""
+TODO: fill in
+"""
+
+package_group(
+    name = "api_visibility",
+    packages = [
+        "//",
+    ],
+)
+
+package_group(
+    name = "proto_impl_visibility",
+    packages = [
+        "//proto/...",
+    ],
+)
+
+java_library(
+    name = "android_java_protos",
+    visibility = ["//visibility:public"],
+    exports = [
+      "//proto/v1/api:android_java_proto",
+      "//proto/v1/structure:structure_java_proto",
+    ],
+)
+
+java_library(
+    name = "android_java_lite_protos",
+    visibility = ["//visibility:public"],
+    exports = [
+      "//proto/v1/api:android_java_proto_lite",
+      "//proto/v1/structure:structure_java_proto_lite",
+    ],
+)
diff --git a/README.md b/README.md
@@ -1,2 +1,37 @@
-# oppia-android-api
-Published Android-specific API for the Oppia backend.
+# oppia-proto-api
+Published required API for the Oppia backend.
+
+## Versioning
+Versioning starts getting a bit complicated when considering the backend VersionedModels, the model structures themselves, schema structures, proto structures, the API, snapshots of the API, and other things. To try and keep things simple, this design intends to define a _separate_ set of versions that are specific to this API. How these map to Oppia backend's many versions is an implementation detail not actually relevant to this design.
+
+### API version
+The API itself is versioned and has separate releases. To keep things simple, it has a major & minor version (e.g. 1.0). The major version is represented directly in the directory structure (e.g. 'v1') since the repository could, in the future, house multiple major versions (which is necessary for the backend to maintain long-term backward compatibility with older API versions). The major version is only incremented if a breaking change must be introduced into the API (generally this will only come when we want to make substantial changes to the API/redesign it).
+
+The API's minor version essentially counts the number of releases that major version has been changed. This means we increment the minor version anytime we make a compatible change to the API, and then release it. The main benefit of having distinct releases is it allows the backend & frontends to opt into taking on a particular version of the API. Due to the backward/forward interoperability nature of protobuf, it's unlikely there will ever be an incompatibility so long as structure versions are respected (see below), and even in those cases there should be potential for graceful failures. Note that the minor version is only ever represented in the release numbers and never in code form since it only serves the purpose of cataloging.
+
+### Content versions
+Content versions are meant to be the version of the entity itself corresponding to a particular structure (such as a topic, skill, or exploration). These are used to track whether there are content updates for a particular structure. These are expected to map exactly to the versions stored in the corresponding structure's VersionedModel in the backend, but how this versioning behaves is up to the backend so long as it's monotonically increasing and positive.
+
+### Structure versions
+Structure versions correspond to groups of proto structures themselves. These are unique versions to the proto structures and are **not** shared with schema structure versions in the backend. Generally, these only need to be incremented when a proto structure is updated in such a way that the default data will break existing logical assumptions (thus requiring a data migration). Proto structures should never be updated in an incompatible way (without incrementing the major version--see the API version section above), but sometimes data migrations are necessary and these versions help track when those need to occur. Unlike schema versions in the backend, these are likely to not increment as often since most proto changes can be safely ignored by older clients.
+
+Note that one aspect of this is a bit complicated: some of the substructures are shared across high-level structures (such as SubtitledHtml, or other language-based substructures). To ensure these are properly tracked, some substructures are grouped into their own combined version to track changes. The following are the list of structure types whose versions are tracked:
+- TopicSummaryStructureVersion
+- RevisionCardStructureVersion
+- ConceptCardStructureVersion
+- ExplorationStructureVersion
+- QuestionStructureVersion
+- StateStructureVersion
+- LanguageStructuresVersion
+- ImageStructureVersion
+
+#### A note on interaction versioning
+I did want to briefly note that it was discovered during the creation of this PR & structure versions that separate versioning for interactions is not actually needed since, per the strong typing of State's substructures, the structures of individual interactions & how they relate to the exploration/question experience is implied as part of State's structure version. This does not mean the backend won't need additional version tracking for interactions in order to compute compatibility with specific proto State structure versions, but it does serendipitously simplify version tracking at the sub-State level a bit.
+
+## Support
+If you have any feature requests or bug reports, please log them on our [issue tracker](https://github.com/oppia/oppia-proto-api/issues/new?title=Describe%20your%20feature%20request%20or%20bug%20report%20succinctly&body=If%20you%27d%20like%20to%20propose%20a%20feature,%20describe%20what%20you%27d%20like%20to%20see.%0A%0AIf%20you%27re%20reporting%20a%20bug,%20please%20be%20sure%20to%20include%20the%20expected%20behaviour,%20the%20observed%20behaviour,%20and%20steps%20to%20reproduce%20the%20problem.%20Console%20copy-pastes%20and%20any%20background%20on%20the%20environment%20would%20also%20be%20helpful.%0A%0AThanks!).
+
+Please report security issues directly to [email protected].
+
+## Licence
+The Oppia-Proto-API code is released under the [Apache v2 license](https://github.com/oppia/oppia-proto-api/blob/master/LICENSE).
diff --git a/WORKSPACE b/WORKSPACE
@@ -0,0 +1,60 @@
+"""
+This file lists and imports all external dependencies needed to build Oppia Proto API.
+"""
+
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+
+# Set up Python (as a prerequisite dependency for Protobuf).
+
+PYTHON_SHA256_HASH = "934c9ceb552e84577b0faf1e5a2f0450314985b4d8712b2b70717dc679fdc01b"
+PYTHON_VERSION = "0.3.0"
+
+http_archive(
+    name = "rules_python",
+    url = "https://github.com/bazelbuild/rules_python/releases/download/{0}/rules_python-{0}.tar.gz".format(PYTHON_VERSION),
+    sha256 = PYTHON_SHA256_HASH,
+)
+
+# Set up proto & its toolchain.
+
+PROTOBUF_SHA256_HASH = "c6003e1d2e7fefa78a3039f19f383b4f3a61e81be8c19356f85b6461998ad3db"
+PROTOBUF_VERSION = "3.17.3"
+
+http_archive(
+    name = "com_google_protobuf",
+    sha256 = PROTOBUF_SHA256_HASH,
+    strip_prefix = "protobuf-%s" % PROTOBUF_VERSION,
+    urls = ["https://github.com/protocolbuffers/protobuf/archive/v%s.tar.gz" % PROTOBUF_VERSION]
+)
+
+# Set up rules_proto to allow defining proto libraries.
+
+# Note that rules_proto doesn't have any shipped versions, so the workspace needs to pin to a
+# specific commit hash.
+RULES_PROTO_VERSION = "97d8af4dc474595af3900dd85cb3a29ad28cc313"
+RULES_PROTO_SHA256_HASH = "602e7161d9195e50246177e7c55b2f39950a9cf7366f74ed5f22fd45750cd208"
+
+http_archive(
+    name = "rules_proto",
+    sha256 = RULES_PROTO_SHA256_HASH,
+    strip_prefix = "rules_proto-%s" % RULES_PROTO_VERSION,
+    urls = ["https://github.com/bazelbuild/rules_proto/archive/%s.tar.gz" % RULES_PROTO_VERSION],
+)
+
+load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
+rules_proto_dependencies()
+rules_proto_toolchains()
+
+# Set up rules_java to enable the Java proto & Java proto lite rules.
+RULES_JAVA_VERSION = "0.1.1"
+RULES_JAVA_SHA256_HASH = "220b87d8cfabd22d1c6d8e3cdb4249abd4c93dcc152e0667db061fb1b957ee68"
+
+http_archive(
+    name = "rules_java",
+    sha256 = RULES_JAVA_SHA256_HASH,
+    url = "https://github.com/bazelbuild/rules_java/releases/download/{0}/rules_java-{0}.tar.gz".format(RULES_JAVA_VERSION),
+)
+
+load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains")
+rules_java_dependencies()
+rules_java_toolchains()
diff --git a/proto/BUILD b/proto/BUILD
@@ -0,0 +1,3 @@
+"""
+TODO: fill in
+"""
diff --git a/proto/v1/BUILD b/proto/v1/BUILD
@@ -0,0 +1,3 @@
+"""
+TODO: fill in
+"""
diff --git a/proto/v1/api/BUILD b/proto/v1/api/BUILD
@@ -0,0 +1,29 @@
+"""
+This library contains the proto_library(), java_lite_proto_library() and java_proto_library rules.
+The proto_library() rule creates a proto file library for the android API proto file to be used in multiple languages. 
+The java_lite_proto_library() rule takes in a proto_library target and generates Java code.
+The java_proto_library() rule takes in a proto_library target and generate Java code.
+"""
+
+load("@rules_java//java:defs.bzl", "java_lite_proto_library", "java_proto_library")
+load("@rules_proto//proto:defs.bzl", "proto_library")
+
+proto_library(
+    name = "android_proto",
+    srcs = ["android.proto"],
+    deps = [
+      "//proto/v1/structure:structure_proto"
+    ],
+)
+
+java_proto_library(
+    name = "android_java_proto",
+    visibility = ["//:api_visibility"],
+    deps = [":android_proto"],
+)
+
+java_lite_proto_library(
+    name = "android_java_proto_lite",
+    visibility = ["//:api_visibility"],
+    deps = [":android_proto"],
+)
diff --git a/proto/v1/api/android.proto b/proto/v1/api/android.proto
@@ -0,0 +1,113 @@
+syntax = "proto3";
+
+package org.oppia.proto.v1.api;
+
+import "proto/v1/structure/concept_card.proto";
+import "proto/v1/structure/exploration.proto";
+import "proto/v1/structure/question.proto";
+import "proto/v1/structure/revision_card.proto";
+import "proto/v1/structure/topic_summary.proto";
+import "proto/v1/structure/versions.proto";
+
+option java_package = "org.oppia.v1.api";
+option java_multiple_files = true;
+
+message TopicListRequest {
+  TopicListRequestResponseStructureVersion structure_version = 1;
+
+  ClientContext client_context = 2;
+  ClientCompatibilityContext compatibility_context = 3;
+  ClientDownloadStateContext download_context = 4;
+}
+
+message TopicListResponse {
+  TopicListRequestResponseStructureVersion structure_version = 1;
+
+  // All topic summaries that are newer & compatible with the client.
+  repeated org.oppia.proto.v1.structure.TopicSummary topic_summaries = 2;
+}
+
+message TopicContentRequest {
+  TopicContentRequestResponseStructureVersion structure_version = 1;
+
+  ClientContext client_context = 2;
+  repeated DownloadRequestStructureIdentifier identifiers = 3;
+  int32 requested_max_payload_size = 4;
+}
+
+message TopicContentResponse {
+  TopicContentRequestResponseStructureVersion structure_version = 1;
+
+  repeated DownloadResult download_results = 2;
+
+  // Structure corresponds to TopicContentRequestResponseStructureVersion.
+  message DownloadResult {
+    DownloadRequestStructureIdentifier identifier = 1;
+
+    oneof structure_type {
+      bool skipped_suggest_retry = 2;
+      org.oppia.proto.v1.structure.RevisionCard revision_card = 3;
+      org.oppia.proto.v1.structure.ConceptCard concept_card = 4;
+      org.oppia.proto.v1.structure.Exploration exploration = 5;
+      QuestionIdList question_id_list = 6;
+      org.oppia.proto.v1.structure.Question question = 7;
+    }
+  }
+
+  // Structure corresponds to TopicContentRequestResponseStructureVersion.
+  message QuestionIdList {
+    repeated string question_ids = 1;
+  }
+}
+
+message ClientContext {
+  string version_name = 1;
+  int64 version_code = 2;
+}
+
+message ClientCompatibilityContext {
+  org.oppia.proto.v1.structure.TopicSummaryStructureVersion topic_summary_structure_version = 1;
+  org.oppia.proto.v1.structure.RevisionCardStructureVersion revision_card_structure_version = 2;
+  org.oppia.proto.v1.structure.ConceptCardStructureVersion concept_card_structure_version = 3;
+  org.oppia.proto.v1.structure.ExplorationStructureVersion exploration_structure_version = 4;
+  org.oppia.proto.v1.structure.QuestionStructureVersion question_structure_version = 5;
+  org.oppia.proto.v1.structure.StateStructureVersion state_structure_version = 6;
+  org.oppia.proto.v1.structure.LanguageStructuresVersion language_structures_version = 7;
+  org.oppia.proto.v1.structure.ImageStructureVersion thumbnail_structure_version = 8;
+}
+
+message ClientDownloadStateContext {
+  message DownloadState {
+    int32 content_version = 1;
+    oneof structure_type {
+      string topic_id = 2;
+      org.oppia.proto.v1.structure.SubtopicId revision_card_id = 3;
+      string concept_card_id = 4;
+      string story_id = 5; 
+      string exploration_id = 6; 
+      string question_id = 7;
+      string skill_id = 8;
+    }
+  }
+
+  repeated DownloadState current_downloads = 1;
+}
+
+// This structure's version corresponds to TopicContentRequestResponseStructureVersion.
+message DownloadRequestStructureIdentifier {
+  oneof structure_type {
+    org.oppia.proto.v1.structure.SubtopicId revision_card_id = 1;
+    string concept_card_id = 2;
+    string exploration_id = 3;
+    string question_list_skill_id = 4;
+    string question_id = 5;
+  }
+}
+
+message TopicListRequestResponseStructureVersion {
+  int32 version = 1;
+}
+
+message TopicContentRequestResponseStructureVersion {
+  int32 version = 1;
+}
diff --git a/proto/v1/structure/BUILD b/proto/v1/structure/BUILD
@@ -0,0 +1,38 @@
+"""
+This library contains the proto_library(), java_lite_proto_library() and java_proto_library rules.
+The proto_library() rule creates a proto file library for the core proto files to be used in multiple languages. 
+The java_lite_proto_library() rule takes in a proto_library target and generates Java code.
+The java_proto_library() rule takes in a proto_library target and generate Java code.
+"""
+
+load("@rules_java//java:defs.bzl", "java_lite_proto_library", "java_proto_library")
+load("@rules_proto//proto:defs.bzl", "proto_library")
+
+proto_library(
+    name = "structure_proto",
+    srcs = [
+        "concept_card.proto",
+        "exploration.proto",
+        "image.proto",
+        "languages.proto",
+        "objects.proto",
+        "question.proto",
+        "revision_card.proto",
+        "state.proto",
+        "topic_summary.proto",
+        "versions.proto",
+    ],
+    visibility = ["//:proto_impl_visibility"],
+)
+
+java_proto_library(
+    name = "structure_java_proto",
+    visibility = ["//:api_visibility"],
+    deps = [":structure_proto"],
+)
+
+java_lite_proto_library(
+    name = "structure_java_proto_lite",
+    visibility = ["//:api_visibility"],
+    deps = [":structure_proto"],
+)
diff --git a/proto/v1/structure/concept_card.proto b/proto/v1/structure/concept_card.proto
@@ -0,0 +1,28 @@
+syntax = "proto3";
+
+package org.oppia.proto.v1.structure;
+
+import "proto/v1/structure/image.proto";
+import "proto/v1/structure/languages.proto";
+import "proto/v1/structure/versions.proto";
+
+option java_package = "org.oppia.v1.structure";
+option java_multiple_files = true;
+
+message ConceptCard {
+  ConceptCardStructureVersion structure_version = 1;
+
+  string skill_id = 2;
+  string description = 3;
+  SubtitledHtml explanation = 4;
+  repeated WorkedExample worked_examples = 5;
+  RecordedVoiceovers recorded_voiceovers = 6;
+  WrittenTranslations written_translations = 7;
+  ReferencedImageList referenced_image_list = 8;
+
+  // Proto version corresponds to ConceptCardStructureVersion.
+  message WorkedExample {
+    SubtitledHtml question = 1;
+    SubtitledHtml explanation = 2;
+  }
+}
diff --git a/proto/v1/structure/exploration.proto b/proto/v1/structure/exploration.proto
@@ -0,0 +1,21 @@
+syntax = "proto3";
+
+package org.oppia.proto.v1.structure;
+
+import "proto/v1/structure/image.proto";
+import "proto/v1/structure/state.proto";
+import "proto/v1/structure/versions.proto";
+
+option java_package = "org.oppia.v1.structure";
+option java_multiple_files = true;
+
+message Exploration {
+  ExplorationStructureVersion structure_version = 1;
+
+  string id = 2;
+  string title = 3;
+  string init_state_name = 4;
+  map<string, State> states = 5;
+  int32 content_version = 6;
+  ReferencedImageList referenced_image_list = 7;
+}