generated from oracle-devrel/repo-template
-
Notifications
You must be signed in to change notification settings - Fork 46
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into synuora-data-development-devops-db-supp-feat…
…ures
- Loading branch information
Showing
48 changed files
with
1,229 additions
and
60 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
46 changes: 46 additions & 0 deletions
46
...odernisation/ai-services/generative-ai-service/generative-ai-workshop/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Generative AI Approach Workshop | ||
|
|
||
| ## Overview | ||
|
|
||
| This Approach workshop is targeted to LOB and Digital Officers to discover and design powerful use cases for Oracle Generative AI. | ||
|
|
||
| ## Common Challenges | ||
|
|
||
| Many clients are currently seeking to identify use cases for Generative AI. This workshop aims to guide participants in understanding the main advantages of Generative AI and the common solution architectures observed in the enterprise sector. More importantly, it will help pinpoint those use cases within your own organization. Furthermore, we'll look more closely at a specific use case, incorporating elements of design thinking. We will then view it from a solution architecture standpoint to identify its essential components. | ||
|
|
||
| ## Benefits | ||
|
|
||
| - Deeper understanding of the value of Generative AI; use cases exploration. | ||
| - Persona and Design thinking based to uncover real value. | ||
| - High level understanding of target architecture; what are the data sources, how to deal with guardrails and agents. | ||
|
|
||
| ## More Details | ||
|
|
||
| - How to request this workshop: Please reach out to your Oracle Account Team | ||
| - Delivered by: Cloud Solution Specialists - AI | ||
| Services | ||
| - How it is delivered: Zoom & Whiteboarding Tools | ||
| - Customer audience: LoB Representatives, Digital | ||
| Officers | ||
| - Workshop Duration: 2-4 Hours | ||
|
|
||
| ## AGENDA | ||
|
|
||
| 1. Introduction & Goals | ||
| 2. Ideate: | ||
| - Persona’s – understand the users & their needs | ||
| - Data sources | ||
| - UI/channels | ||
| 3. Solution Design: | ||
| - Prompting / Guardrails | ||
| - Integrations/Agents | ||
| 4. Next Steps | ||
|
|
||
| # License | ||
|
|
||
| Copyright (c) 2023 Oracle and/or its affiliates. | ||
|
|
||
| Licensed under the Universal Permissive License (UPL), Version 1.0. | ||
|
|
||
| See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. | ||
|
|
35 changes: 35 additions & 0 deletions
35
ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/LICENSE.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| Copyright (c) 2024 Oracle and/or its affiliates. | ||
|
|
||
| The Universal Permissive License (UPL), Version 1.0 | ||
|
|
||
| Subject to the condition set forth below, permission is hereby granted to any | ||
| person obtaining a copy of this software, associated documentation and/or data | ||
| (collectively the "Software"), free of charge and under any and all copyright | ||
| rights in the Software, and any and all patent rights owned or freely | ||
| licensable by each licensor hereunder covering either (i) the unmodified | ||
| Software as contributed to or provided by such licensor, or (ii) the Larger | ||
| Works (as defined below), to deal in both | ||
|
|
||
| (a) the Software, and | ||
| (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if | ||
| one is included with the Software (each a "Larger Work" to which the Software | ||
| is contributed by such licensors), | ||
|
|
||
| without restriction, including without limitation the rights to copy, create | ||
| derivative works of, display, perform, and distribute the Software and make, | ||
| use, sell, offer for sale, import, export, have made, and have sold the | ||
| Software and the Larger Work(s), and to sublicense the foregoing rights on | ||
| either these or other terms. | ||
|
|
||
| This license is subject to the following condition: | ||
| The above copyright notice and either this complete permission notice or at | ||
| a minimum a reference to the UPL must be included in all copies or | ||
| substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
75 changes: 75 additions & 0 deletions
75
ai-and-app-modernisation/ai-services/generative-ai-service/rag-genai/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
|
|
||
| # Creating a RAG (Retrieval-Augmented Generation) with Oracle Generative AI Service in just 21 lines of code | ||
|
|
||
| ## Introduction | ||
| In this article, we'll explore how to create a Retrieval-Augmented Generation (RAG) model using Oracle Gen AI, llama index, Qdrant Vector Database, and SentenceTransformerEmbeddings. This 21-line code will allow you to scrape through web pages, use llama index for indexing, Oracle Generative AI Service for question generation, and Qdrant for vector indexing. | ||
|
|
||
| <img src="./RagArchitecture.svg"> | ||
| </img> | ||
|
|
||
| ## Limited Availability | ||
|
|
||
| Oracle Generative AI Service is in Limited Availability as of today when we are creating this repo. | ||
|
|
||
| Customers can easily enter in the LA programs. To test these functionalities you need to enrol in the LA programs and install the proper versions of software libraries. | ||
|
|
||
| Code and functionalities can change, as a result of changes and new features | ||
|
|
||
| ## Prerequisites | ||
| Before getting started, make sure you have the following installed: | ||
|
|
||
| - Oracle Generative AI Service | ||
| - llama index | ||
| - qdrant client | ||
| - SentenceTransformerEmbeddings | ||
|
|
||
| ## Setting up the Environment | ||
| 1. Install the required packages: | ||
| ```bash | ||
| pip install oci==2.112.1+preview.1.1649 llama-index qdrant-client sentence-transformers | ||
| ``` | ||
|
|
||
| ## Loading data | ||
|
|
||
| You need to create a sitemap.xml file where you can specify or list the webpages which you want to include in your RAG. | ||
| Here we have used SentenceTransformerEmbeddings to create the embeddings but you can easily use any embeddings model . In the next blog we will show how easily you can use Oracle Generative AI Service embeddings model. | ||
|
|
||
| In this example we have used some Oracle documentation pages and created a xml file for the same and have placed it in Oracle object storage. | ||
|
|
||
| sitemap used : https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frpj5kvxryk1/b/thisIsThePlace/o/combined.xml | ||
|
|
||
| ## Entire code | ||
|
|
||
| ```bash | ||
| from genai_langchain_integration.langchain_oci import OCIGenAI | ||
| from llama_index import VectorStoreIndex | ||
| from llama_index import ServiceContext | ||
| from llama_index.vector_stores.qdrant import QdrantVectorStore | ||
| from llama_index.storage.storage_context import StorageContext | ||
| from qdrant_client import qdrant_client | ||
| from langchain.embeddings import SentenceTransformerEmbeddings | ||
| from llama_hub.web.sitemap import SitemapReader | ||
| loader = SitemapReader() | ||
| documents = loader.load_data(sitemap_url='https://objectstorage.eu-frankfurt-1.oraclecloud.com/n/frpj5kvxryk1/b/thisIsThePlace/o/combined.xml') | ||
| client = qdrant_client.QdrantClient(location=":memory:") | ||
| embeddings = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2") | ||
| llm = OCIGenAI(model_id="cohere.command",service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com",compartment_id = "ocid1.tenancy.oc1..aaaaaaaa5hwtrus75rauufcfvtnjnz3mc4xm2bzibbigva2bw4ne7ezkvzha",temperature=0.0) | ||
| system_prompt="As a support engineer, your role is to leverage the information in the context provided. Your task is to respond to queries based strictly on the information available in the provided context. Do not create new information under any circumstances. Refrain from repeating yourself. Extract your response solely from the context mentioned above. If the context does not contain relevant information for the question, respond with 'How can I assist you with questions related to the document?" | ||
| service_context = ServiceContext.from_defaults(llm=llm, chunk_size=1000, chunk_overlap=100, embed_model=embeddings,system_prompt=system_prompt) | ||
| vector_store = QdrantVectorStore(client=client, collection_name="ansh") | ||
| storage_context = StorageContext.from_defaults(vector_store=vector_store) | ||
| index = VectorStoreIndex.from_documents(documents, storage_context=storage_context, service_context=service_context) | ||
| query_engine = index.as_query_engine() | ||
| response = query_engine.query("can i use OCI document understanding for files in french ?") | ||
| print(response) | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| ## Conclusion | ||
|
|
||
| In this article, we've covered the process of creating a RAG model using Oracle Generative AI Service, llama index, Qdrant, and SentenceTransformerEmbeddings. Feel free to experiment with different web pages and datasets to enhance the capabilities of your model. | ||
|
|
||
| In a future blog post, we'll explore how to integrate Oracle Vector Database and Oracle Gen AI embeddings model into this RAG setup. | ||
|
|
||
| Feel free to modify and expand upon this template according to your specific use case and preferences. Good luck with your article! |
4 changes: 4 additions & 0 deletions
4
...p-modernisation/ai-services/generative-ai-service/rag-genai/RagArchitecture.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
ai-and-app-modernisation/content-management/content-management-cloud/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # ansible-jenkins | ||
|
|
||
| ## Getting started | ||
|
|
||
| This Terraform code provisions a new OCI instance and installs Jenkins directly through an Ansible playbook. | ||
| To optimize and be more cost-efficient, the instance shape is locked to VM.Standard.A1.Flex, but this code | ||
| can eventually be modified or forked. | ||
|
|
||
| ## Features and limitations | ||
| * Get quickly started with the latest Jenkins version on OCI | ||
| * Manage plugins and the installation through Ansible and Jenkins Configuration as Code | ||
| * Tested on Oracle Linux 8 | ||
| * Instance generated only if it is in a public subnet network | ||
| * Port 22 must be opened on the instance, as OCI Resource Manager will need to connect to the instance through SSH | ||
| * Jenkins port can't be between 0 and 1024, as those are Linux reserved ports and would require further configurations to be exposed | ||
| * To access Jenkins, the instance and Jenkins port must be reachable | ||
| * As the instance will be updated, it will take a while during the first run | ||
|
|
||
| Although these limitations might not fit every use case, the code can be used as a reference and there are ways to lift them. | ||
|
|
||
| [](https://cloud.oracle.com/resourcemanager/stacks/create?zipUrl=https://github.com/oracle-devrel/technology-engineering/raw/main/app-dev/devops/ansible-jenkins/ansible-jenkins-rm.zip) | ||
|
|
Binary file not shown.
Oops, something went wrong.