Skip to content

Commit

Permalink
Merge branch 'vector-restructure' of https://github.com/opensearch-pr…
Browse files Browse the repository at this point in the history
…oject/documentation-website into vector-restructure
  • Loading branch information
kolchfa-aws committed Feb 14, 2025
2 parents 73f4fd7 + 7f549c4 commit e2a1cf9
Show file tree
Hide file tree
Showing 8 changed files with 118 additions and 51 deletions.
1 change: 1 addition & 0 deletions _about/version-history.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ permalink: /version-history/

OpenSearch version | Release highlights | Release date
:--- | :--- | :---
[2.19.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.19.0.md) | Adds workload management, additional query insights, and template queries. Introduces a query insights page to OpenSearch Dashboards. Includes improvements and bug fixes to snapshots, search statistics, star-tree search, and index management. For a full list of release highlights, see the Release Notes. | 11 February 2025
[2.18.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.18.0.md) | Adds a redesigned home page, updated Discover interface, and collaborative workspaces to OpenSearch Dashboards. Includes improvements to ML inference processor and query grouping. Introduces reranking by field and paginated CAT APIs. Includes experimental OpenSearch Dashboards Assistant capabilities. For a full list of release highlights, see the Release Notes. | 05 November 2024
[2.17.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.17.1.md) | Includes bug fixes for ML Commons, anomaly detection, k-NN, and security analytics. Adds various infrastructure and maintenance updates. For a full list of release highlights, see the Release Notes. | 1 October 2024
[2.17.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.17.0.md) | Includes disk-optimized vector search, binary quantization, and byte vector encoding in k-NN. Adds asynchronous batch ingestion for ML tasks. Provides search and query performance enhancements and a new custom trace source in trace analytics. Includes application-based configuration templates. For a full list of release highlights, see the Release Notes. | 17 September 2024
Expand Down
6 changes: 3 additions & 3 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ baseurl: "/docs/latest" # the subpath of your site, e.g. /blog
url: "https://opensearch.org" # the base hostname & protocol for your site, e.g. http://example.com
permalink: /:path/

opensearch_version: '2.18.0'
opensearch_dashboards_version: '2.18.0'
opensearch_major_minor_version: '2.18'
opensearch_version: '2.19.0'
opensearch_dashboards_version: '2.19.0'
opensearch_major_minor_version: '2.19'
lucene_version: '9_12_0'

# Build settings
Expand Down
5 changes: 3 additions & 2 deletions _data/versions.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
{
"current": "2.18",
"current": "2.19",
"all": [
"2.18",
"2.19",
"1.3"
],
"archived": [
"2.18",
"2.17",
"2.16",
"2.15",
Expand Down
4 changes: 2 additions & 2 deletions _field-types/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ Type | Description
:--- | :---
`null` | A `null` field can't be indexed or searched. When a field is set to null, OpenSearch behaves as if the field has no value.
`boolean` | OpenSearch accepts `true` and `false` as Boolean values. An empty string is equal to `false.`
`float` | A single-precision, 32-bit floating-point number.
`double` | A double-precision, 64-bit floating-point number.
`float` | A single-precision, 32-bit IEEE 754 floating-point number, restricted to finite values.
`double` | A double-precision, 64-bit IEEE 754 floating-point number, restricted to finite values.
`integer` | A signed 32-bit number.
`object` | Objects are standard JSON objects, which can have fields and mappings of their own. For example, a `movies` object can have additional properties such as `title`, `year`, and `director`.
`array` | OpenSearch does not have a specific array data type. Arrays are represented as a set of values of the same data type (for example, integers or strings) associated with a field. When indexing, you can pass multiple values for a field, and OpenSearch will treat it as an array. Empty arrays are valid and recognized as array fields with zero elements---not as fields with no values. OpenSearch supports querying and filtering arrays, including checking for values, range queries, and array operations like concatenation and intersection. Nested arrays, which may contain complex objects or other arrays, can also be used for advanced data modeling.
Expand Down
13 changes: 12 additions & 1 deletion _ml-commons-plugin/api/model-apis/delete-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,15 @@ DELETE /_plugins/_ml/models/MzcIJX8BA7mbufL6DOwl
"_seq_no" : 27,
"_primary_term" : 18
}
```
```

## Safely deleting a model
Introduced 2.19
{: .label .label-purple }

To prevent accidental deletion of models in active use by agents, search pipelines, ingest pipelines, or other components, you can enable a safety check. If the safety check is enabled and you attempt to delete a model that is in current use, OpenSearch returns an error message. To proceed with deletion:

- Identify any components using the model and either delete them or update them so that they use other models.
- Once all dependencies are cleared, delete the model.

For information about enabling this feature, see [Safely delete models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/cluster-settings/#safely-delete-models).
84 changes: 51 additions & 33 deletions _ml-commons-plugin/cluster-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ We recommend setting `plugins.ml_commons.only_run_on_ml_node` to `true` on produ

### Setting

```
```yaml
plugins.ml_commons.only_run_on_ml_node: true
```

Expand All @@ -52,7 +52,7 @@ plugins.ml_commons.only_run_on_ml_node: true

### Setting

```
```yaml
plugins.ml_commons.task_dispatch_policy: round_robin
```

Expand All @@ -68,7 +68,7 @@ Sets the number of ML tasks that can run on each ML node. When set to `0`, no ML

### Setting

```
```yaml
plugins.ml_commons.max_ml_task_per_node: 10
```

Expand All @@ -83,7 +83,7 @@ Sets the number of ML models that can be deployed to each ML node. When set to `

### Setting

```
```yaml
plugins.ml_commons.max_model_on_node: 10
```

Expand All @@ -99,7 +99,7 @@ When returning runtime information with the [Profile API]({{site.url}}{{site.bas

### Setting

```
```yaml
plugins.ml_commons.sync_up_job_interval_in_seconds: 3
```

Expand All @@ -114,7 +114,7 @@ Controls how many predict requests are monitored on one node. If set to `0`, Ope

### Setting

```
```yaml
plugins.ml_commons.monitoring_request_count: 100
```

Expand All @@ -129,7 +129,7 @@ Controls how many register model tasks can run in parallel on one node. If set t

### Setting

```
```yaml
plugins.ml_commons.max_register_model_tasks_per_node: 10
```

Expand All @@ -146,7 +146,7 @@ Controls how many deploy model tasks can run in parallel on one node. If set to

### Setting

```
```yaml
plugins.ml_commons.max_deploy_model_tasks_per_node: 10
```

Expand All @@ -157,11 +157,11 @@ plugins.ml_commons.max_deploy_model_tasks_per_node: 10

## Register models using URLs

This setting gives you the ability to register models using a URL. By default, ML Commons only allows registration of [pretrained]({{site.url}}{{site.baseurl}}//ml-commons-plugin/pretrained-models/) models from the OpenSearch model repository.
This setting gives you the ability to register models using a URL. By default, ML Commons only allows registration of [pretrained]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models/) models from the OpenSearch model repository.

### Setting

```
```yaml
plugins.ml_commons.allow_registering_model_via_url: false
```

Expand All @@ -172,11 +172,11 @@ plugins.ml_commons.allow_registering_model_via_url: false

## Register models using local files

This setting gives you the ability to register a model using a local file. By default, ML Commons only allows registration of [pretrained]({{site.url}}{{site.baseurl}}//ml-commons-plugin/pretrained-models/) models from the OpenSearch model repository.
This setting gives you the ability to register a model using a local file. By default, ML Commons only allows registration of [pretrained]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models/) models from the OpenSearch model repository.

### Setting

```
```yaml
plugins.ml_commons.allow_registering_model_via_local_file: false
```

Expand All @@ -196,7 +196,7 @@ The default URL value for this trusted URL setting is not secure. For security,
{: .warning }


```
```yaml
plugins.ml_commons.trusted_url_regex: <model-repository-url>
```

Expand All @@ -211,7 +211,7 @@ Assigns how long in seconds an ML task will live. After the timeout, the task wi

### Setting

```
```yaml
plugins.ml_commons.ml_task_timeout_in_seconds: 600
```

Expand All @@ -230,7 +230,7 @@ Starting with OpenSearch 2.5, ML Commons runs a native memory circuit breaker to

### Setting

```
```yaml
plugins.ml_commons.native_memory_threshold: 90
```

Expand All @@ -247,7 +247,7 @@ Values are based on the percentage of JVM heap memory available. When set to `0`

### Setting

```
```yaml
plugins.ml_commons.jvm_heap_memory_threshold: 85
```

Expand All @@ -264,7 +264,7 @@ Valid values are in byte units. To disable the circuit breaker, set this value t

### Setting

```
```yaml
plugins.ml_commons.disk_free_space_threshold: 5G
```

Expand All @@ -279,7 +279,7 @@ Use this setting to specify the names of nodes on which you don't want to run ML

### Setting

```
```yaml
plugins.ml_commons.exclude_nodes._name: node1, node2
```

Expand All @@ -289,7 +289,7 @@ When enabled, this setting grants users the ability to deploy models to specific

### Setting

```
```yaml
plugins.ml_commons.allow_custom_deployment_plan: false
```

Expand All @@ -304,7 +304,7 @@ This setting is applicable when you send a prediction request for an externally

### Setting

```
```yaml
plugins.ml_commons.model_auto_deploy.enable: false
```

Expand All @@ -319,7 +319,7 @@ This setting automatically redeploys deployed or partially deployed models upon

### Setting

```
```yaml
plugins.ml_commons.model_auto_redeploy.enable: true
```

Expand All @@ -334,7 +334,7 @@ This setting sets the limit for the number of times a deployed or partially depl

### Setting

```
```yaml
plugins.ml_commons.model_auto_redeploy.lifetime_retry_times: 3
```

Expand All @@ -349,7 +349,7 @@ This setting sets the ratio of success for the auto-redeployment of a model base

### Setting

```
```yaml
plugins.ml_commons.model_auto_redeploy_success_ratio: 0.8
```

Expand All @@ -364,22 +364,40 @@ When set to `true`, this setting enables the ability to run Python-based models

### Setting

```
```yaml
plugins.ml_commons.enable_inhouse_python_model: false
```

### Values

- Default value: false
- Default value: `false`
- Valid values: `false`, `true`

## Safely delete models
Introduced 2.19
{: .label .label-purple }

When set to `true`, this setting enables a safety feature that checks for downstream dependencies before deleting a model. This helps prevent accidental deletion of models in active use by agents, search pipelines, ingest pipelines, and other downstream tasks. If this setting is enabled and you attempt to delete a model that has active downstream dependencies, you'll receive an error message and the model will not be deleted.

### Setting

```yaml
plugins.ml_commons.safe_delete_model: true
```

### Values

- Default value: `false`
- Valid values: `false`, `true`


## Enable access control for connectors

When set to `true`, the setting allows admins to control access and permissions to the connector API using `backend_roles`.

### Setting

```
```yaml
plugins.ml_commons.connector_access_control_enabled: true
```

Expand All @@ -394,7 +412,7 @@ This setting allows a cluster admin to enable running local models on the cluste

### Setting

```
```yaml
plugins.ml_commons.local_model.enabled: true
```

Expand All @@ -409,7 +427,7 @@ This setting allows a cluster admin to control the types of nodes on which exter

### Setting

```
```yaml
plugins.ml_commons.task_dispatcher.eligible_node_role.remote_model: ["ml"]
```

Expand All @@ -424,7 +442,7 @@ This setting allows a cluster admin to control the types of nodes on which local

### Setting

```
```yaml
plugins.ml_commons.task_dispatcher.eligible_node_role.remote_model: ["ml"]
```

Expand All @@ -438,7 +456,7 @@ This setting allows a cluster admin to enable remote inference on the cluster. I

### Setting

```
```yaml
plugins.ml_commons.remote_inference.enabled: true
```

Expand All @@ -453,7 +471,7 @@ When set to `true`, this setting enables the agent framework (including agents a

### Setting

```
```yaml
plugins.ml_commons.agent_framework_enabled: true
```

Expand All @@ -468,7 +486,7 @@ When set to `true`, this setting enables conversational memory, which stores all

### Setting

```
```yaml
plugins.ml_commons.memory_feature_enabled: true
```

Expand All @@ -484,7 +502,7 @@ When set to `true`, this setting enables the search processors for retrieval-aug

### Setting

```
```yaml
plugins.ml_commons.rag_pipeline_feature_enabled: true
```

Expand Down
Loading

0 comments on commit e2a1cf9

Please sign in to comment.