Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/docs/cloud/data-sources/knowledge-base.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ This is a perfect use case for the Knowledge Base:

![Knowledge Base Edit Document](/docs/cloud/data-sources/knowledge-base-edit-document.png)

Once you've added your documents, you will be able to edit or delete them at any time. If you write a lot of documents, you can always search them by keywork or by meaning.
Once you've added your documents, you will be able to edit or delete them at any time. If you write a lot of documents, you can always search them by keyword or by meaning.

![Knowledge Base Edit Document](/docs/cloud/data-sources/knowledge-base-with-docs.png)
8 changes: 4 additions & 4 deletions content/docs/cloud/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,22 @@ Before you get started with Orama Cloud, it's important to understand some key c

A project is a set of [**data sources**](#data-sources) that you want to use for RAG, search, or any other method of retrieval. Each project has its own set of data sources, and depending on your plan, you can create as many projects as you need.

> **Why do I need different data sources? And why not just use a single collection for all of my data?**
> **Why do I need different data sources? And why not just use a single project for all of my data?**

That is a fair question, and the answer is simple: imagine you're building a support agent for your company. You will have some data coming from Slack, some from Notion, some from your documentation, and so on.

You could put everything in a single collection, but that could complicate things when it comes to updating, deleting, or adding new data.
You could put everything in a single project, but that could complicate things when it comes to updating, deleting, or adding new data.

For example, you may want to scrape your documentation every day, so you'd have to remove the old data and add the new data. But what if you also want to keep the data from Slack and Notion? You'd have to re-import all of that data every time you update your documentation, which would be a pain.

Instead, you can put data that will be updated frequently in a single [data source](#data-sources) (let's call it `docs`), and put data that follows a different update schedule in another data source (let's call it `slack`). This way, you can update the `docs` data source every day, and the `slack` data source every hour, for example. You can also add new data sources as needed, without having to worry about updating everything at once.

When you create a collection, you can choose its language (English, Spanish, etc.), and the embedding model you want to use. You can always change these settings in the future, but remember that they will be shared across all of the data sources in that collection.
When you create a project, you can choose its language (English, Spanish, etc.), and the embedding model you want to use. You can always change these settings in the future, but remember that they will be shared across all of the data sources in that project.

### Data Sources

A data source identifies a specific set of documents, ideally coming from a single origin. Continuing with the example above, you could have a data source for your documentation, one for your Slack messages, and one for your Notion pages.

Every data source is segregated from the others, so you can update, delete, or add new data without affecting the data in the other data sources.

At retrieval time (be it search, RAG, or any other method of retrieval), you can choose to use a single data source, multiple sources, or all of them. This flexibility allows you to have a single collection with multiple data sources and use them as needed for different purposes.
At retrieval time (be it search, RAG, or any other method of retrieval), you can choose to use a single data source, multiple sources, or all of them. This flexibility allows you to have a single project with multiple data sources and use them as needed for different purposes.
2 changes: 1 addition & 1 deletion content/docs/cloud/projects/create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Create a project in Orama Cloud

Once you have signed up for Orama Cloud, you can create a project. A project is a set of data sources that you can use to bring data into Orama Cloud.

If you're not familiar with the concept of collections, check out the [introduction to Orama Cloud](/docs/cloud#projects) for more information.
If you're not familiar with the concept of projects, check out the [introduction to Orama Cloud](/docs/cloud#projects) for more information.

## Create a project

Expand Down
10 changes: 5 additions & 5 deletions content/docs/oramacore/apis/advanced-reasoning.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ API Key type: **`read_api_key`**. Safe to expose to the public.

OramaCore has a built-in **Advanced Reasoning** feature, that may be used to perform a bit more advanced tasks.

When performing an [**Answer Session**](/docs/apis/answer-session), OramaCore will:
When performing an [**Answer Session**](/docs/oramacore/apis/answer-session), OramaCore will:

1. Get the query from the user.
2. Optimize it to perform the best possible search on your data.
Expand All @@ -21,7 +21,7 @@ When performing an [**Answer Session**](/docs/apis/answer-session), OramaCore wi

With **Advanced Reasoning**, OramaCore takes a step forward and tries to understand the user's query in a more advanced way.

Under the hood, it uses a built-in library called [**Party Planner**](/docs/architecture/party-planner).
Under the hood, it uses a built-in library called [**Party Planner**](/docs/oramacore/architecture/party-planner).

Party Planner will generate an action plan based on the user query, then will execute each step of the plan to get to the best possible answer.

Expand Down Expand Up @@ -94,7 +94,7 @@ In the example above, OramaCore takes a slightly different route from the previo

## Advanced Reasoning APIs

Using **Advanced Reasoning** with OramaCore is not very different from performing [Answer Sessions](/docs/apis/answer-session).
Using **Advanced Reasoning** with OramaCore is not very different from performing [Answer Sessions](/docs/oramacore/apis/answer-session).

First of all, you have a to create a new answer session:

Expand Down Expand Up @@ -134,7 +134,7 @@ const answerSession = collection.createAnswerSession({
})
```

You can read more about events and `state` structure [here](/docs/apis/answer-session#creating-an-answer-session).
You can read more about events and `state` structure [here](/docs/oramacore/answer-session#creating-an-answer-session).

### Performing an Advanced Reasoning Answer Session

Expand Down Expand Up @@ -229,7 +229,7 @@ In the example above, the console will log the following events:
}
]
```
2. Second state change: OramaCore comes up with a plan to answer the question. The plan is added to the current interaction. The plan is generated by [Party Planner](/docs/architecture/party-planner).
2. Second state change: OramaCore comes up with a plan to answer the question. The plan is added to the current interaction. The plan is generated by [Party Planner](/docs/oramacore/architecture/party-planner).
```json
[
{
Expand Down
4 changes: 2 additions & 2 deletions content/docs/oramacore/apis/answer-session.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ const answerSession = collection.createAnswerSession({

The `initialMessages` field is an array of messages that will be sent to the answer session as soon as it is created. This is useful if you are resuming a conversation, or if you want to provide some context to the answer session.

The `LLMConfig` allows you to specify a remote LLM to invoke for this particular answer session. It depends on the [OramaCore configuration](/docs/guide/configuration). By default, OramaCore will use the local LLM.
The `LLMConfig` allows you to specify a remote LLM to invoke for this particular answer session. It depends on the [OramaCore configuration](docs/oramacore/configuration). By default, OramaCore will use the local LLM.

The `events` field is an object that can contain the following event handlers:

Expand Down Expand Up @@ -111,7 +111,7 @@ Let's see what each field means:
| `sources` | The sources that were used to generate the response. | `object` | `[ { id: '123', score: 0.8, document: { ... } } ]` |
| `loading` | Whether the interaction is loading. | `boolean` | `false` |
| `planned` | Whether the interaction was planned. | `boolean` | `false` |
| `plan` | The plan of actions that OramaCore will take to answer the question. Generated by [Party Planner](/docs/architecture/party-planner). | `object` | `[ { step: 'OPTIMIZE_QUERY', description: 'Write an optimized OramaCore query for the user input' } ]` |
| `plan` | The plan of actions that OramaCore will take to answer the question. Generated by [Party Planner](/docs/oramacore/architecture/party-planner). | `object` | `[ { step: 'OPTIMIZE_QUERY', description: 'Write an optimized OramaCore query for the user input' } ]` |
| `error` | Whether the interaction has an error. | `boolean` | `false` |
| `errorMessage` | The error message, if any. | `string` | `The document could not be found` |
| `aborted` | Whether the interaction was aborted. | `boolean` | `false` |
Expand Down
2 changes: 1 addition & 1 deletion content/docs/oramacore/apis/custom-system-prompts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Custom System Prompts"
description: "Adopting custom system prompts for OramaCore Answer Sessions"
---

When performing [Answer Sessions](/docs/apis/answer-session) and [Advanced Reasoning Answer Sessions](/docs/apis/advanced-reasoning), OramaCore uses a standard **system prompt** to instruct the LLM on how to reply, provide guardrails, and ensure a low rate of hallucinations in the response.
When performing [Answer Sessions](/docs/oramacore/apis/answer-session) and [Advanced Reasoning Answer Sessions](/docs/oramacore/apis/advanced-reasoning), OramaCore uses a standard **system prompt** to instruct the LLM on how to reply, provide guardrails, and ensure a low rate of hallucinations in the response.

If you're curious, you can find the full system prompt [on GitHub](https://github.com/oramasearch/oramacore/blob/develop/src/prompts/v1/answer/system.md).

Expand Down
6 changes: 3 additions & 3 deletions content/docs/oramacore/apis/insert-documents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Insert Documents"
description: "APIs to insert documents into an OramaCore collection."
---

To insert documents into an OramaCore index, you need to [create a collection](/docs/apis/create-collection) first and then an [index](/docs/apis/create-collection#creating-an-index).
To insert documents into an OramaCore index, you need to [create a collection](/docs/oramacore/apis/create-collection) first and then an [index](/docs/oramacore/apis/create-collection#creating-an-index).

## APIs

Expand Down Expand Up @@ -56,7 +56,7 @@ await index.insertDocuments([
```
</Tabs>

OramaCore will [automatically generate generate text embeddings](/docs/customizations/text-embeddings) for the document in real-time, allowing you to perform full-text, hybrid, and vector search immediately.
OramaCore will [automatically generate generate text embeddings](/docs/oramacore/customizations/text-embeddings) for the document in real-time, allowing you to perform full-text, hybrid, and vector search immediately.

## Optimizations

Expand All @@ -66,7 +66,7 @@ Since OramaCore uses two immutable data structures (**FST** for full-text search

OramaCore will insert all the new documents into a buffer, and once the buffer reaches a certain size, it will be flushed to the immutable data structures.

The buffer size is configurable via the [`insert_batch_commit_size`](/docs/guide/configuration#writer_side) configuration in the `config.yaml` file.
The buffer size is configurable via the [`insert_batch_commit_size`](/docs/oramacore/guide/configuration#writer_side) configuration in the `config.yaml` file.

By default, OramaCore will commit (so will push the buffer to the immutable data structures) every 5000 documents. You can change this value to better suit your use case.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/oramacore/apis/introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Introduction"
description: "An introduction to the OramaCore REST APIs and SDKs."
---

With OramaCore we aim to provide a set of APIs that are backward compatible - or at least, as similar as possible - to those exposed by [Orama Cloud](https://docs.orama.com/cloud/performing-search).
With OramaCore we aim to provide a set of APIs that are backward compatible - or at least, as similar as possible - to those exposed by [Orama Cloud](/docs/cloud).

At the time of writing, with OramaCore being in beta, we are still working on the APIs and SDKs. We are also working on the documentation, so please bear with us.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/oramacore/apis/upsert-documents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,4 @@ await index.insertDocuments([
```
</Tabs>

Just like during insertion, at upsert-time OramaCore will [automatically generate generate text embeddings](/docs/customizations/text-embeddings) for the document in real-time, allowing you to perform full-text, hybrid, and vector search immediately.
Just like during insertion, at upsert-time OramaCore will [automatically generate generate text embeddings](/docs/oramacore/customizations/text-embeddings) for the document in real-time, allowing you to perform full-text, hybrid, and vector search immediately.
2 changes: 1 addition & 1 deletion content/docs/oramacore/architecture/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Future versions of OramaCore will move away from this approach by either integra

### Embeddings Generation

OramaCore automatically generates embeddings for your data. You can configure which models to use via the [configuration](/docs/guide/configuration).
OramaCore automatically generates embeddings for your data. You can configure which models to use via the [configuration](/docs/oramacore/guide/configuration).

Current benchmarks indicate this implementation can generate up to 1,200 embeddings per second on an RTX 4080 Super. We acknowledge this seems optimistic and will release reproducible benchmarks soon.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/oramacore/customizations/text-embeddings.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ These models are optimized for speed and memory usage, making them suitable for
</Callout>


The current OramaCore implementation is designed to support multiple text embedding models, allowing users to choose the best model for their specific use case. Therefore, different collections can use different models.
The current OramaCore implementation is designed to support multiple text embedding models, allowing users to choose the best model for their specific use case. Therefore, different projects can use different models.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if this should remain "collections"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that should remain collections as it's in OramaCore


---

Expand Down
24 changes: 12 additions & 12 deletions content/docs/oramacore/guide/api-key.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "API keys are used to authenticate requests to the OramaCore API"
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';

As explained in the [Architecture](/docs/architecture/write-read) section, OramaCore is split in two sides: the **reader side** and the **writer side**.
As explained in the [Architecture](/docs/oramacore/architecture/overview) section, OramaCore is split in two sides: the **reader side** and the **writer side**.

Therefore, depending on the operation you want to perform, you will need to use different API keys.

Expand All @@ -18,7 +18,7 @@ In total, OramaCore will give you access to three different kinds API keys:

The **master API key** is an essential key that allows you to configure the OramaCore instance and create or delete new collections.

It's configurable via the [`config.yaml`](/docs/guide/configuration) file under the `writer_side` section:
It's configurable via the [`config.yaml`](/docs/oramacore/guide/configuration) file under the `writer_side` section:

```yaml
# ...
Expand Down Expand Up @@ -60,15 +60,15 @@ When using the JavaScript client, OramaCore will automatically set the correct A
**Not safe to share**. Never share the write API key publicly. Treat it as a password.
</Callout>

The **write API key** is used to insert, update, or delete documents in a collection or index, as well as to create new hooks or tools.
The **write API key** is used to insert, update, or delete documents in a project or index, as well as to create new hooks or tools.

Every collection has its own write API key, which is generated when you create the collection.
Every project has its own write API key, which is generated when you create the project.

You will need this API key to:

- Insert one or more documents into a collection/index
- Update one or more documents in a collection/index
- Delete one or more documents from a collection/index
- Insert one or more documents into a project/index
- Update one or more documents in a project/index
- Delete one or more documents from a project/index
- Create a new hook or tool

In all the cases above, you will need to pass the write API key in the `Authorization` header of the request as a `Bearer` token.
Expand All @@ -78,7 +78,7 @@ Example:
<Tabs groupId='insert' persist items={['cURL']}>
```bash tab="cURL"
curl -X POST \
http://localhost:8080/v1/collections/{COLLECTION_ID}/indexes/create` \
http://localhost:8080/v1/collections/{PROJECT_ID}/indexes/create` \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <write_api_key>' \
-d '{ "id": "products", "embedding": "automatic" }'
Expand All @@ -93,14 +93,14 @@ If you're connecting to your OramaCore instance from a web browser, please make
**Safe to share**. This API key performs read operations only. You can share it publicly.
</Callout>

The **read API key** is used to perform read operations on a collection.
The **read API key** is used to perform read operations on a project.

Every collection has its own read API key, which is generated when you create the collection.
Every project has its own read API key, which is generated when you create the project.

You will need this API key to:

- Perform full-text, hybrid, or vector search
- Read the documents in a collection
- Read the documents in a project
- Perform answer sessions
- Run a tool

Expand All @@ -111,7 +111,7 @@ Example:
<Tabs groupId='search' persist items={['cURL']}>
```bash tab="cURL"
curl -X POST \
http://localhost:8080/v1/collections/{COLLECTION_ID}/search?api-key=<read_api_key> \
http://localhost:8080/v1/collections/{PROJECT_ID}/search?api-key=<read_api_key> \
-H 'Content-Type: application/json' \
-d '{ "term": "The quick brown fox" }'
```
Expand Down
Loading