Merge pull request #91500 from rh-tokeefe/OLS-1622

OLS-1622: Document token quota feature

Author: ShaunaDiaz

configure/ols-configuring-openshift-lightspeed.adoc

@@ -28,3 +28,5 @@ include::modules/ols-about-cluster-interaction.adoc[leveloffset=+1]
 include::modules/ols-enabling-cluster-interaction.adoc[leveloffset=+2]
 include::modules/ols-about-the-byo-knowledge-tool.adoc[leveloffset=+1]
 include::modules/ols-providing-custom-knowledge-to-the-llm.adoc[leveloffset=+2]
+include::modules/ols-tokens-and-token-quota-limits.adoc[leveloffset=+1]
+include::modules/ols-activating-token-quota-limits.adoc[leveloffset=+2]

modules/ols-activating-token-quota-limits.adoc

@@ -0,0 +1,61 @@
+// Module included in the following assemblies:
+// * lightspeed-docs-main/configure/ols-configuring-openshift-lightspeed.adoc
+
+:_mod-docs-content-type: PROCEDURE
+[id="ols-activating-token-quota-limits_{context}"]
+= Activating token quota limits
+
+Activate token quota limits for the {ols-long} service by defining key-value pairs in the `ConfigMap` resource. The {ols-long} pod mounts the `ConfigMap` resource as a volume, enabling access to the file stored within it. The `OLSConfig` custom resource (CR) references the `ConfigMap` resource to obtain the quota limit information.
+
+.Prerequisites
+
+* You have installed the the {ols-long} Operator.
+
+* You have configured a large language model provider.
+
+* A PostgreSQL database is configured and the {ols-long} service can access the database.
+
+.Procedure
+
+. Open the {ols-long} `OLSconfig` CR file by running the following command:
++
+[source,terminal]
+----
+$ oc edit olsconfig cluster
+----
+
+. Modify the `spec.ols.quotaHandlersConfig` specification to include token quota limit information.
++
+.Example {ols-long} `OLSConfig` CR
+[source,yaml]
+----
+apiVersion: ols.openshift.io/v1alpha1
+kind: OLSConfig
+metadata:
+  name: cluster
+spec:
+  ols:
+    quotaHandlersConfig:
+      limitersConfig:
+      - name: user_limits # <1>
+        type: user_limiter
+        initialQuota: 100000 # <2>
+        quotaIncrease: 1000 # <3>
+        period: 30 days
+      - name: cluster_limits # <4>
+        type: cluster_limiter 
+        initialQuota: 1000000 # <5>
+        quotaIncrease: 100000 # <6>
+        period: 30 days # <7>
+----
+<1> Specifies the token limit for user account. 
+<2> Specifies a token quota limit of 100,000 for each user over the time period specified in the `period` field.
+<3> Increases the token quota limit for the user by 1,000 at the end of the time period specified in the `period` field.
+<4> Specifies the token limit for a cluster.
+<5> Specifies a token quota limit of 1,000,000 for each cluster over the time period specified in the `period` field.
+<6> Increases the token quota limit for the cluster by 100,000 at the end of the time period specified in the `period` field.
+<7> Defines the amount of time that the scheduler waits before the period resets or the quota limit increases.
+
+. Click *Save*. 
++
+The save operation saves the file and applies the changes to activate the token quota limits.

modules/ols-tokens-and-token-quota-limits.adoc

@@ -0,0 +1,12 @@
+// Module included in the following assemblies:
+// * lightspeed-docs-main/configure/ols-configuring-openshift-lightspeed.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="ols-tokens-and-token-quota-limits_{context}"]
+= Tokens and token quota limits
+
+Tokens are small chunks of text, which can be as small as one character or as large as one word. Tokens are the units of measurement used to quantify the amount of text that the {ols-long} service sends to, or receives from, a large language model (LLM). Every interaction with the service and the LLM is counted in tokens.
+
+Token quota limits define the number of tokens that can be used in a certain timeframe. Implementing token quota limits helps control costs, encourage more efficient use of queries, and regulate system demands. In a multi-user configuration, token quota limits help provide equal access to all users ensuring everyone has an opportunity to submit queries. 
+
+You can define token quota limits for {ocp-short-name} clusters or {ocp-short-name} user accounts.