diff --git a/ai/ai-speech/podcast-generator/files/README.md b/ai/ai-speech/podcast-generator/Podcast-md/README.md similarity index 96% rename from ai/ai-speech/podcast-generator/files/README.md rename to ai/ai-speech/podcast-generator/Podcast-md/README.md index c86aaa879..5222f1bb7 100644 --- a/ai/ai-speech/podcast-generator/files/README.md +++ b/ai/ai-speech/podcast-generator/Podcast-md/README.md @@ -60,6 +60,8 @@ Identify the compartment you're currently working within. Navigate to 'Identity' * Automatically a podcast script will be created and will generate the audio podcast using the Cindy and Bob voices ![alt text](generated.jpg) +## Notes +* It is important to note that, as of the time of writing this tutorial, the Speech-to-Text service is available exclusively in the Phoenix region. ## I hope you liked it. Author: Jesús Brasero \ No newline at end of file diff --git a/ai/ai-speech/podcast-generator/files/ak1.png b/ai/ai-speech/podcast-generator/Podcast-md/ak1.png similarity index 100% rename from ai/ai-speech/podcast-generator/files/ak1.png rename to ai/ai-speech/podcast-generator/Podcast-md/ak1.png diff --git a/ai/ai-speech/podcast-generator/files/ak2.png b/ai/ai-speech/podcast-generator/Podcast-md/ak2.png similarity index 100% rename from ai/ai-speech/podcast-generator/files/ak2.png rename to ai/ai-speech/podcast-generator/Podcast-md/ak2.png diff --git a/ai/ai-speech/podcast-generator/files/ak3.png b/ai/ai-speech/podcast-generator/Podcast-md/ak3.png similarity index 100% rename from ai/ai-speech/podcast-generator/files/ak3.png rename to ai/ai-speech/podcast-generator/Podcast-md/ak3.png diff --git a/ai/ai-speech/podcast-generator/files/ak4.png b/ai/ai-speech/podcast-generator/Podcast-md/ak4.png similarity index 100% rename from ai/ai-speech/podcast-generator/files/ak4.png rename to ai/ai-speech/podcast-generator/Podcast-md/ak4.png diff --git a/ai/ai-speech/podcast-generator/files/ak5.png b/ai/ai-speech/podcast-generator/Podcast-md/ak5.png similarity index 100% rename from ai/ai-speech/podcast-generator/files/ak5.png rename to ai/ai-speech/podcast-generator/Podcast-md/ak5.png diff --git a/ai/ai-speech/podcast-generator/files/compartmentid.jpg b/ai/ai-speech/podcast-generator/Podcast-md/compartmentid.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/compartmentid.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/compartmentid.jpg diff --git a/ai/ai-speech/podcast-generator/files/edit_tts.jpg b/ai/ai-speech/podcast-generator/Podcast-md/edit_tts.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/edit_tts.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/edit_tts.jpg diff --git a/ai/ai-speech/podcast-generator/files/edit_tts_2.jpg b/ai/ai-speech/podcast-generator/Podcast-md/edit_tts_2.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/edit_tts_2.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/edit_tts_2.jpg diff --git a/ai/ai-speech/podcast-generator/files/generated.jpg b/ai/ai-speech/podcast-generator/Podcast-md/generated.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/generated.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/generated.jpg diff --git a/ai/ai-speech/podcast-generator/files/import_project.jpg b/ai/ai-speech/podcast-generator/Podcast-md/import_project.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/import_project.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/import_project.jpg diff --git a/ai/ai-speech/podcast-generator/files/preview.jpg b/ai/ai-speech/podcast-generator/Podcast-md/preview.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/preview.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/preview.jpg diff --git a/ai/ai-speech/podcast-generator/files/services.jpg b/ai/ai-speech/podcast-generator/Podcast-md/services.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/services.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/services.jpg diff --git a/ai/ai-speech/podcast-generator/files/signature.jpg b/ai/ai-speech/podcast-generator/Podcast-md/signature.jpg similarity index 100% rename from ai/ai-speech/podcast-generator/files/signature.jpg rename to ai/ai-speech/podcast-generator/Podcast-md/signature.jpg diff --git a/ai/ai-speech/podcast-generator/README.md b/ai/ai-speech/podcast-generator/README.md index 24f7d4653..6c8320cf6 100644 --- a/ai/ai-speech/podcast-generator/README.md +++ b/ai/ai-speech/podcast-generator/README.md @@ -5,15 +5,70 @@ The application is designed to streamline podcast production through advanced AI This application is built using Oracle Visual Builder Cloud Service (VBCS), a powerful low-code platform that simplifies development and accelerates the creation of robust applications without extensive coding. With this low-code approach, even complex workflows are straightforward to set up, allowing developers to focus on leveraging AI's potential for high-quality audio synthesis. This AI-powered solution not only automates and optimizes the podcast creation process but also allows content creators to deliver professional audio content at scale efficiently. -Reviewed: 12.11.2024 +Reviewed: 26.11.2024 -# When to use this asset? - -See the README document in the /files folder. - -# How to use this asset? +# **1. Prepare your user** + + In Oracle Cloud Infrastructure (OCI), API keys are used for secure authentication when accessing OCI resources through REST APIs. OCI API keys consist of two parts: a Public key and a Private key. You use the OCI console to generate the Private/Public key pair. + Generate API Keys using OCI Console + To Generate the API Keys using OCI Console: + + - Login into your OCI Account. + ![alt text]![alt text](Podcast-md/ak1.png) + - Click on the Profile icon at the top-right corner and select your Profile hyperlink. + ![alt text](Podcast-md/ak2.png) + - Under Resources section at the bottom-left, select API Keys and then click Add API Key. + ![alt text](Podcast-md/ak3.png) + - The Add API Key dialog is displayed. Select Generate API Key Pair to create a new key pair. + ![alt text](Podcast-md/ak4.png) + - Click Download Private Key. A .pem file is saved to your local device. You do not need to download the public key and click Add button. + ![alt text](Podcast-md/ak5.png) + + + +# **2.Pick you compartment** +Identify the compartment you're currently working within. Navigate to 'Identity' -> 'Compartments'. Locate your compartment and make a note of its OCID (Oracle Cloud Identifier) + +# **3.Open Visual Builder** +## Import Visual Builder project +* Open Visual Builder and click on the "Import" button. Choose "Application from file". +* Drop the zip project file +* Provide a name and an ID, for example "Podcast_Generator". Click on Import button. + ![alt text](Podcast-md/import_project.jpg) + +## Configure REST APIs authentication +* Open the recently created project. -See the README document in the /files folder. +* Click on Services button (left side) and click on "Backends" + ![alt text](Podcast-md/services.jpg) + +* Now, click on TTS, and Servers to edit server authentication. +![alt text](Podcast-md/edit_tts.jpg) + +* Click the pencil to provide the OCI Crendentials + ![alt text](Podcast-md/edit_tts_2.jpg) + +* Provide the crendentials you got during the step 1. + ![alt text](Podcast-md/signature.jpg) + +* Repeat the same process with the GenAI backend. + +## Provide your compartmentId +* Provide compartmentId default value in the project variable named "comparmentid" that you got during the step 2. + ![alt text](Podcast-md/compartmentid.jpg) + +# **4.Preview the application** +* Now can provide a topic in the text area and click "generate" button. + ![alt text](Podcast-md/preview.jpg) + +* Automatically a podcast script will be created and will generate the audio podcast using the Cindy and Bob voices + ![alt text](Podcast-md/generated.jpg) + +## Notes +* It is important to note that, as of the time of writing this tutorial, the Speech-to-Text service is available exclusively in the Phoenix region. + +## I hope you liked it. +Author: Jesús Brasero # License diff --git a/ai/ai-speech/podcast-generator/SECURITY.md b/ai/ai-speech/podcast-generator/SECURITY.md new file mode 100644 index 000000000..fb42c9453 --- /dev/null +++ b/ai/ai-speech/podcast-generator/SECURITY.md @@ -0,0 +1,38 @@ +# Reporting security vulnerabilities + +Oracle values the independent security research community and believes that +responsible disclosure of security vulnerabilities helps us ensure the security +and privacy of all our users. + +Please do NOT raise a GitHub Issue to report a security vulnerability. If you +believe you have found a security vulnerability, please submit a report to +[secalert_us@oracle.com][1] preferably with a proof of concept. Please review +some additional information on [how to report security vulnerabilities to Oracle][2]. +We encourage people who contact Oracle Security to use email encryption using +[our encryption key][3]. + +We ask that you do not use other channels or contact the project maintainers +directly. + +Non-vulnerability related security issues including ideas for new or improved +security features are welcome on GitHub Issues. + +## Security updates, alerts and bulletins + +Security updates will be released on a regular cadence. Many of our projects +will typically release security fixes in conjunction with the +Oracle Critical Patch Update program. Additional +information, including past advisories, is available on our [security alerts][4] +page. + +## Security-related information + +We will provide security related information such as a threat model, considerations +for secure use, or any known security issues in our documentation. Please note +that labs and sample code are intended to demonstrate a concept and may not be +sufficiently hardened for production use. + +[1]: mailto:secalert_us@oracle.com +[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html +[3]: https://www.oracle.com/security-alerts/encryptionkey.html +[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file diff --git a/ai/ai-speech/podcast-generator/files/vbcs_project/podcast_1_0-1.0.zip b/ai/ai-speech/podcast-generator/VBCS Project/podcast_ai-1.0.zip similarity index 97% rename from ai/ai-speech/podcast-generator/files/vbcs_project/podcast_1_0-1.0.zip rename to ai/ai-speech/podcast-generator/VBCS Project/podcast_ai-1.0.zip index c5c5a9608..730a702cb 100644 Binary files a/ai/ai-speech/podcast-generator/files/vbcs_project/podcast_1_0-1.0.zip and b/ai/ai-speech/podcast-generator/VBCS Project/podcast_ai-1.0.zip differ diff --git a/ai/ai-speech/podcast-generator/demo/on_the_moon.mov b/ai/ai-speech/podcast-generator/demo/on_the_moon.mov new file mode 100644 index 000000000..7aa2a1303 Binary files /dev/null and b/ai/ai-speech/podcast-generator/demo/on_the_moon.mov differ diff --git a/ai/generative-ai-service/README.md b/ai/generative-ai-service/README.md index c72069362..49730aed3 100644 --- a/ai/generative-ai-service/README.md +++ b/ai/generative-ai-service/README.md @@ -58,7 +58,7 @@ Reviewed: 13.11.2024 ## Reusable Assets Overview - [Podcast Generator](https://github.com/oracle-devrel/technology-engineering/tree/main/ai/ai-speech/podcast-generator) - +- [Decode Images and Videos with OCI GenAI](https://github.com/oracle-devrel/technology-engineering/tree/main/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI) # Useful Links diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/README.md b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/README.md new file mode 100644 index 000000000..39ffc3b8a --- /dev/null +++ b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/README.md @@ -0,0 +1,98 @@ + +# Decode Images and Videos with OCI GenAI + +This is an AI-powered application designed to unlock insights hidden within media files using the Oracle Cloud Infrastructure (OCI) Generative AI services. This application enables users to analyze images and videos, generating detailed summaries in multiple languages. Whether you are a content creator, researcher, or media enthusiast, this app helps you interpret visual content with ease. + + + +--- + +## Features + +### 🌍 **Multi-Language Support** +- Receive summaries in your preferred language, including: + - English, French, Arabic, Spanish, Italian, German, Portuguese, Japanese, Korean, and Chinese. + +### 🎥 **Customizable Frame Processing for Videos** +- Extract video frames at user-defined intervals. +- Analyze specific frame ranges to tailor your results for precision. + +### ⚡ **Parallel Processing** +- Uses efficient parallel computation for quick and accurate frame analysis. + +### 🖼️ **Image Analysis** +- Upload images to generate detailed summaries based on your input prompt. + +### 🧠 **Cohesive Summaries** +- Combines individual frame insights to create a seamless, cohesive summary of the video’s overall theme, events, and key details. + +--- + +## Technologies Used +- **[Streamlit](https://streamlit.io/):** For building an interactive user interface. +- **[Oracle Cloud Infrastructure (OCI) Generative AI](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm):** For powerful image and video content analysis. +- **[OpenCV](https://opencv.org/):** For video frame extraction and processing. +- **[Pillow (PIL)](https://pillow.readthedocs.io/):** For image handling and processing. +- **[tqdm](https://tqdm.github.io/):** For progress visualization in parallel processing. + +--- + +## Installation + +1. **Clone the repository:** + + +2. **Install dependencies:** + Make sure you have Python 3.8+ installed. Then, install the required libraries: + ```bash + pip install -r requirements.txt + ``` + +3. **Configure OCI:** + - Set up your OCI configuration by creating or updating the `~/.oci/config` file with your credentials and profile. + - Replace placeholders like `compartmentId`, `llm_service_endpoint`, and `visionModel` in the code with your actual values. + +--- + +## Usage + +1. **Run the application:** + ```bash + streamlit run app.py + ``` + +2. **Upload a file:** + - Use the sidebar to upload an image (`.png`, `.jpg`, `.jpeg`) or a video (`.mp4`, `.avi`, `.mov`). + +3. **Set parameters:** + - For videos, adjust the frame extraction interval and select specific frame ranges for analysis. + +4. **Analyze and summarize:** + - Enter a custom prompt to guide the AI in generating a meaningful summary. + - Choose the output language from the sidebar. + +5. **Get results:** + - View detailed image summaries or cohesive video summaries directly in the app. + +--- + +## Screenshots +### Image Analysis + + + +### Video Analysis + + + +--- + + +## Acknowledgments +- Oracle Cloud Infrastructure Generative AI for enabling state-of-the-art visual content analysis. +- Open-source libraries like OpenCV, Pillow, and Streamlit for providing powerful tools to build this application. + +--- + +## Contact +If you have questions or feedback, feel free to reach out via [anshuman.p.panda@oracle.com](mailto:email@example.com). diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/app.py b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/app.py new file mode 100644 index 000000000..0aaa846ac --- /dev/null +++ b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/app.py @@ -0,0 +1,203 @@ +# Author: Ansh +import streamlit as st +import oci +import base64 +import cv2 +from PIL import Image +from concurrent.futures import ThreadPoolExecutor +from tqdm import tqdm + +# OCI Configuration +compartmentId = "ocid1.compartment.oc1..XXXXXXXXXXXXXxxxxxxxxxxxxxxxxxxxxxxxxxxx" +llm_service_endpoint = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com" +CONFIG_PROFILE = "DEFAULT" +visionModel = "meta.llama-3.2-90b-vision-instruct" +summarizeModel = "cohere.command-r-plus-08-2024" +config = oci.config.from_file('~/.oci/config', CONFIG_PROFILE) +llm_client = oci.generative_ai_inference.GenerativeAiInferenceClient( + config=config, + service_endpoint=llm_service_endpoint, + retry_strategy=oci.retry.NoneRetryStrategy(), + timeout=(10, 240) + ) + +# Functions for Image Analysis +def encode_image(image_path): + with open(image_path, "rb") as image_file: + return base64.b64encode(image_file.read()).decode("utf-8") + +# Functions for Video Analysis +def encode_cv2_image(frame): + _, buffer = cv2.imencode('.jpg', frame) + return base64.b64encode(buffer).decode("utf-8") + +# Common Functions +def get_message(encoded_image=None, user_prompt=None): + content1 = oci.generative_ai_inference.models.TextContent() + content1.text = user_prompt + + message = oci.generative_ai_inference.models.UserMessage() + message.content = [content1] + + if encoded_image: + content2 = oci.generative_ai_inference.models.ImageContent() + image_url = oci.generative_ai_inference.models.ImageUrl() + image_url.url = f"data:image/jpeg;base64,{encoded_image}" + content2.image_url = image_url + message.content.append(content2) + return message + +def get_chat_request(encoded_image=None, user_prompt=None): + chat_request = oci.generative_ai_inference.models.GenericChatRequest() + chat_request.messages = [get_message(encoded_image, user_prompt)] + chat_request.api_format = oci.generative_ai_inference.models.BaseChatRequest.API_FORMAT_GENERIC + chat_request.num_generations = 1 + chat_request.is_stream = False + chat_request.max_tokens = 500 + chat_request.temperature = 0.75 + chat_request.top_p = 0.7 + chat_request.top_k = -1 + chat_request.frequency_penalty = 1.0 + return chat_request + +def cohere_chat_request(encoded_image=None, user_prompt=None): + print(" i am here") + chat_request = oci.generative_ai_inference.models.CohereChatRequest() + chat_request.api_format = oci.generative_ai_inference.models.BaseChatRequest.API_FORMAT_COHERE + message = get_message(encoded_image, user_prompt) + chat_request.message = message.content[0].text + chat_request.is_stream = False + chat_request.preamble_override = "Make sure you answer only in "+ lang_type + chat_request.max_tokens = 500 + chat_request.temperature = 0.75 + chat_request.top_p = 0.7 + chat_request.top_k = 0 + chat_request.frequency_penalty = 1.0 + return chat_request + + +def get_chat_detail(chat_request,model): + chat_detail = oci.generative_ai_inference.models.ChatDetails() + chat_detail.serving_mode = oci.generative_ai_inference.models.OnDemandServingMode(model_id=model) + chat_detail.compartment_id = compartmentId + chat_detail.chat_request = chat_request + return chat_detail + +def extract_frames(video_path, interval=1): + frames = [] + cap = cv2.VideoCapture(video_path) + frame_rate = int(cap.get(cv2.CAP_PROP_FPS)) + success, frame = cap.read() + count = 0 + + while success: + if count % (frame_rate * interval) == 0: + frames.append(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)) + success, frame = cap.read() + count += 1 + cap.release() + return frames + +def process_frame(llm_client, frame, prompt): + encoded_image = encode_cv2_image(frame) + try: + llm_request = get_chat_request(encoded_image, prompt) + llm_payload = get_chat_detail(llm_request,visionModel) + llm_response = llm_client.chat(llm_payload) + return llm_response.data.chat_response.choices[0].message.content[0].text + except Exception as e: + return f"Error processing frame: {str(e)}" + +def process_frames_parallel(llm_client, frames, prompt): + with ThreadPoolExecutor() as executor: + results = list(tqdm( + executor.map(lambda frame: process_frame(llm_client, frame, prompt), frames), + total=len(frames), + desc="Processing frames" + )) + return results + +def generate_final_summary(llm_client, frame_summaries): + combined_summaries = "\n".join(frame_summaries) + final_prompt = ( + "You are a video content summarizer. Below are summaries of individual frames extracted from a video. " + "Using these frame summaries, create a cohesive and concise summary that describes the content of the video as a whole. " + "Focus on providing insights about the overall theme, events, or key details present in the video, and avoid referring to individual frames or images explicitly.\n\n" + f"{combined_summaries}" + ) + try: + llm_request = cohere_chat_request(user_prompt=final_prompt) + llm_payload = get_chat_detail(llm_request,summarizeModel) + llm_response = llm_client.chat(llm_payload) + return llm_response.data.chat_response.text + except Exception as e: + return f"Error generating final summary: {str(e)}" + +# Streamlit UI +st.title("Decode Images and Videos with OCI GenAI") +uploaded_file = st.sidebar.file_uploader("Upload an image or video", type=["png", "jpg", "jpeg", "mp4", "avi", "mov"]) +user_prompt = st.text_input("Enter your prompt for analysis:", value="Describe the content of this image.") +lang_type = st.sidebar.selectbox("Output Language", ["English", "French", "Arabic", "Spanish", "Italian", "German", "Portuguese", "Japanese", "Korean", "Chinese"]) + +if uploaded_file: + if uploaded_file.name.split('.')[-1].lower() in ["png", "jpg", "jpeg"]: + # Image Analysis + temp_image_path = "temp_uploaded_image.jpg" + with open(temp_image_path, "wb") as f: + f.write(uploaded_file.getbuffer()) + + st.image(temp_image_path, caption="Uploaded Image", width=500) + + if st.button("Generate image Summary"): + with st.spinner("Analyzing the image..."): + try: + encoded_image = encode_image(temp_image_path) + llm_request = get_chat_request(encoded_image, user_prompt) + llm_payload = get_chat_detail(llm_request,visionModel) + llm_response = llm_client.chat(llm_payload) + llm_text = llm_response.data.chat_response.choices[0].message.content[0].text + st.success("OCI gen AI Response:") + st.write(llm_text) + except Exception as e: + st.error(f"An error occurred: {str(e)}") + elif uploaded_file.name.split('.')[-1].lower() in ["mp4", "avi", "mov"]: + + # Video Analysis + temp_video_path = "temp_uploaded_video.mp4" + video_html = f""" + + """ + st.markdown(video_html, unsafe_allow_html=True) + with open(temp_video_path, "wb") as f: + f.write(uploaded_file.getbuffer()) + + # st.video(temp_video_path) + st.write("Processing the video...") + + frame_interval = st.sidebar.slider("Frame extraction interval (seconds)", 1, 10, 1) + frames = extract_frames(temp_video_path, interval=frame_interval) + num_frames = len(frames) + st.write(f"Total frames extracted: {num_frames}") + + frame_range = st.sidebar.slider("Select frame range for analysis", 0, num_frames - 1, (0, num_frames - 1)) + + if st.button("Generate Video Summary"): + with st.spinner("Analyzing selected frames..."): + try: + selected_frames = frames[frame_range[0]:frame_range[1] + 1] + waiting_message = st.empty() + waiting_message.write(f"Selected {len(selected_frames)} frames for processing.") + # st.write(f"Selected {len(selected_frames)} frames for processing.") + frame_summaries = process_frames_parallel(llm_client, selected_frames, user_prompt) + # st.write("Generating final video summary...") + waiting_message.empty() + waiting_message.write("Generating final video summary...") + final_summary = generate_final_summary(llm_client, frame_summaries) + waiting_message.empty() + st.success("Video Summary:") + st.write(final_summary) + except Exception as e: + st.error(f"An error occurred: {str(e)}") diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image.png b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image.png new file mode 100644 index 000000000..ccbfe61b0 Binary files /dev/null and b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image.png differ diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image2.png b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image2.png new file mode 100644 index 000000000..6eee9e0a9 Binary files /dev/null and b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image2.png differ diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image3.png b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image3.png new file mode 100644 index 000000000..5f666671b Binary files /dev/null and b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/image3.png differ diff --git a/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/requirements.txt b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/requirements.txt new file mode 100644 index 000000000..2e3cf2124 --- /dev/null +++ b/ai/generative-ai-service/decode-Images-and-Videos-with-OCI-GenAI/requirements.txt @@ -0,0 +1,5 @@ +streamlit==1.37.0 +oci==3.50.1 +Pillow +opencv-python-headless==4.10.0.84 +tqdm==4.66.3 \ No newline at end of file diff --git a/ai/generative-ai-service/image-to-text/CONTRIBUTING.md b/ai/generative-ai-service/image-to-text/CONTRIBUTING.md new file mode 100644 index 000000000..93cb5e813 --- /dev/null +++ b/ai/generative-ai-service/image-to-text/CONTRIBUTING.md @@ -0,0 +1,55 @@ +# Contributing to this repository + +We welcome your contributions! There are multiple ways to contribute. + +## Opening issues + +For bugs or enhancement requests, please file a GitHub issue unless it's +security related. When filing a bug remember that the better written the bug is, +the more likely it is to be fixed. If you think you've found a security +vulnerability, do not raise a GitHub issue and follow the instructions in our +[security policy](./SECURITY.md). + +## Contributing code + +We welcome your code contributions. Before submitting code via a pull request, +you will need to have signed the [Oracle Contributor Agreement][OCA] (OCA) and +your commits need to include the following line using the name and e-mail +address you used to sign the OCA: + +```text +Signed-off-by: Your Name +``` + +This can be automatically added to pull requests by committing with `--sign-off` +or `-s`, e.g. + +```text +git commit --signoff +``` + +Only pull requests from committers that can be verified as having signed the OCA +can be accepted. + +## Pull request process + +1. Ensure there is an issue created to track and discuss the fix or enhancement + you intend to submit. +1. Fork this repository. +1. Create a branch in your fork to implement the changes. We recommend using + the issue number as part of your branch name, e.g. `1234-fixes`. +1. Ensure that any documentation is updated with the changes that are required + by your change. +1. Ensure that any samples are updated if the base image has been changed. +1. Submit the pull request. *Do not leave the pull request blank*. Explain exactly + what your changes are meant to do and provide simple steps on how to validate. + your changes. Ensure that you reference the issue you created as well. +1. We will assign the pull request to 2-3 people for review before it is merged. + +## Code of conduct + +Follow the [Golden Rule](https://en.wikipedia.org/wiki/Golden_Rule). If you'd +like more specific guidelines, see the [Contributor Covenant Code of Conduct][COC]. + +[OCA]: https://oca.opensource.oracle.com +[COC]: https://www.contributor-covenant.org/version/1/4/code-of-conduct/ \ No newline at end of file diff --git a/ai/generative-ai-service/image-to-text/LICENSE b/ai/generative-ai-service/image-to-text/LICENSE.txt similarity index 100% rename from ai/generative-ai-service/image-to-text/LICENSE rename to ai/generative-ai-service/image-to-text/LICENSE.txt diff --git a/ai/generative-ai-service/image-to-text/README.md b/ai/generative-ai-service/image-to-text/README.md index ef44ab1b1..d82940c74 100644 --- a/ai/generative-ai-service/image-to-text/README.md +++ b/ai/generative-ai-service/image-to-text/README.md @@ -114,3 +114,26 @@ pip install -r requirements.txt - [Streamlit Documentation](https://docs.streamlit.io/) For questions or feedback, please contact [anshuman.p.panda@oracle.com]. + + + +## Contributing + + +This project welcomes contributions from the community. Before submitting a pull +request, please [review our contribution guide](./CONTRIBUTING.md). + +## Security + +Please consult the [security guide](./SECURITY.md) for our responsible security +vulnerability disclosure process. + +## License +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](LICENSE.txt) for more details. + +ORACLE AND ITS AFFILIATES DO NOT PROVIDE ANY WARRANTY WHATSOEVER, EXPRESS OR IMPLIED, FOR ANY SOFTWARE, MATERIAL OR CONTENT OF ANY KIND CONTAINED OR PRODUCED WITHIN THIS REPOSITORY, AND IN PARTICULAR SPECIFICALLY DISCLAIM ANY AND ALL IMPLIED WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE. FURTHERMORE, ORACLE AND ITS AFFILIATES DO NOT REPRESENT THAT ANY CUSTOMARY SECURITY REVIEW HAS BEEN PERFORMED WITH RESPECT TO ANY SOFTWARE, MATERIAL OR CONTENT CONTAINED OR PRODUCED WITHIN THIS REPOSITORY. IN ADDITION, AND WITHOUT LIMITING THE FOREGOING, THIRD PARTIES MAY HAVE POSTED SOFTWARE, MATERIAL OR CONTENT TO THIS REPOSITORY WITHOUT ANY REVIEW. USE AT YOUR OWN RISK. \ No newline at end of file diff --git a/ai/generative-ai-service/image-to-text/SECURITY.md b/ai/generative-ai-service/image-to-text/SECURITY.md new file mode 100644 index 000000000..fb42c9453 --- /dev/null +++ b/ai/generative-ai-service/image-to-text/SECURITY.md @@ -0,0 +1,38 @@ +# Reporting security vulnerabilities + +Oracle values the independent security research community and believes that +responsible disclosure of security vulnerabilities helps us ensure the security +and privacy of all our users. + +Please do NOT raise a GitHub Issue to report a security vulnerability. If you +believe you have found a security vulnerability, please submit a report to +[secalert_us@oracle.com][1] preferably with a proof of concept. Please review +some additional information on [how to report security vulnerabilities to Oracle][2]. +We encourage people who contact Oracle Security to use email encryption using +[our encryption key][3]. + +We ask that you do not use other channels or contact the project maintainers +directly. + +Non-vulnerability related security issues including ideas for new or improved +security features are welcome on GitHub Issues. + +## Security updates, alerts and bulletins + +Security updates will be released on a regular cadence. Many of our projects +will typically release security fixes in conjunction with the +Oracle Critical Patch Update program. Additional +information, including past advisories, is available on our [security alerts][4] +page. + +## Security-related information + +We will provide security related information such as a threat model, considerations +for secure use, or any known security issues in our documentation. Please note +that labs and sample code are intended to demonstrate a concept and may not be +sufficiently hardened for production use. + +[1]: mailto:secalert_us@oracle.com +[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html +[3]: https://www.oracle.com/security-alerts/encryptionkey.html +[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file diff --git a/ai/generative-ai-service/image-to-text/THIRD_PARTY_LICENSES.txt b/ai/generative-ai-service/image-to-text/THIRD_PARTY_LICENSES.txt new file mode 100644 index 000000000..b480e5876 --- /dev/null +++ b/ai/generative-ai-service/image-to-text/THIRD_PARTY_LICENSES.txt @@ -0,0 +1,223 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + +The MIT License + +Copyright (c) Jerry Liu + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. \ No newline at end of file diff --git a/ai/generative-ai-service/image-to-text/preview.jpg b/ai/generative-ai-service/image-to-text/preview.jpg new file mode 100644 index 000000000..c1f23f048 Binary files /dev/null and b/ai/generative-ai-service/image-to-text/preview.jpg differ diff --git a/ai/generative-ai-service/mindmaps/LICENSE b/ai/generative-ai-service/mindmaps/LICENSE new file mode 100644 index 000000000..62c949c4e --- /dev/null +++ b/ai/generative-ai-service/mindmaps/LICENSE @@ -0,0 +1,35 @@ +Copyright (c) 2024 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/ai/generative-ai-service/mindmaps/README.md b/ai/generative-ai-service/mindmaps/README.md new file mode 100644 index 000000000..9501931b1 --- /dev/null +++ b/ai/generative-ai-service/mindmaps/README.md @@ -0,0 +1,75 @@ +# MindMapGenerator using OCI Generative AI & Visual Builder + +MindMapGenerator, can take a text, analyze it, classify it into levels and sublevels, and generate a structured mind map based on the content using Generative AI. It also works with PDF documents by leveraging Oracle Document Understanding to analyze and extract information from the file before creating the mind map. Additionally, the demo can capture information in real-time using the Speech service, processing spoken input to generate an insightful and organized mind map on the fly. Can be complementary to AI Meetings or other different use cases. + +Reviewed: 17.12.2024 + +# **1. Prepare your user** + + In Oracle Cloud Infrastructure (OCI), API keys are used for secure authentication when accessing OCI resources through REST APIs. OCI API keys consist of two parts: a Public key and a Private key. You use the OCI console to generate the Private/Public key pair. + Generate API Keys using OCI Console + To Generate the API Keys using OCI Console: + + - Login into your OCI Account. + ![alt text]![alt text](mindmaps-md/ak1.png) + - Click on the Profile icon at the top-right corner and select your Profile hyperlink. + ![alt text](mindmaps-md/ak2.png) + - Under Resources section at the bottom-left, select API Keys and then click Add API Key. + ![alt text](mindmaps-md/ak3.png) + - The Add API Key dialog is displayed. Select Generate API Key Pair to create a new key pair. + ![alt text](mindmaps-md/ak4.png) + - Click Download Private Key. A .pem file is saved to your local device. You do not need to download the public key and click Add button. + ![alt text](mindmaps-md/ak5.png) + + + +# **2.Pick you compartment** +Identify the compartment you're currently working within. Navigate to 'Identity' -> 'Compartments'. Locate your compartment and make a note of its OCID (Oracle Cloud Identifier) + +# **3.Open Visual Builder** +## Import Visual Builder project +* Open Visual Builder and click on the "Import" button. Choose "Application from file". +* Drop the zip project file +* Provide a name and an ID, for example "MindMaps_Generator". Click on Import button. + + +## Configure REST APIs authentication +* Open the recently created project. + +* Click on Services button (left side) and click on "Backends" + ![alt text](mindmaps-md/services.jpg) +* Now, click on GenAI, and Servers to edit server authentication. +* Click the pencil to provide the OCI Crendentials +* Provide the crendentials you got during the step 1. + ![alt text](mindmaps-md/signature.jpg) + +* Repeat the same process with the GenAI backend. + +## Provide your compartmentId and all the required variables +* Provide compartmentId default value in the project variable named "compartmentID" that you got during the step 2. + ![alt text](mindmaps-md/variables.jpg) +Repeat the process with bucketName, cohereModel,namespace & prefix + +## Provide JS libray to generate MindMaps +* I developed this sample using go.js but you can replace by any other. Please update the html page in the application providing your library + +# **4.Preview the application** +* Now can provide a topic in the text area and click "generate" button. + ![alt text](mindmaps-md/preview.jpg) + +* Automatically a mindmap will be created. + ![alt text](mindmaps-md/mindmap.jpg) + +## Notes +* It is important to note that, you can use any open source or licensed JS library to generate the MindMap. And you can do using Free Text & Documents Analysis. If you want to use Real Time Transcription you need to configure a Speech Bridge and modify the JS page. + +## I hope you liked it. +Author: Jesús Brasero + +# License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/ai/generative-ai-service/mindmaps/SECURITY.md b/ai/generative-ai-service/mindmaps/SECURITY.md new file mode 100644 index 000000000..fb42c9453 --- /dev/null +++ b/ai/generative-ai-service/mindmaps/SECURITY.md @@ -0,0 +1,38 @@ +# Reporting security vulnerabilities + +Oracle values the independent security research community and believes that +responsible disclosure of security vulnerabilities helps us ensure the security +and privacy of all our users. + +Please do NOT raise a GitHub Issue to report a security vulnerability. If you +believe you have found a security vulnerability, please submit a report to +[secalert_us@oracle.com][1] preferably with a proof of concept. Please review +some additional information on [how to report security vulnerabilities to Oracle][2]. +We encourage people who contact Oracle Security to use email encryption using +[our encryption key][3]. + +We ask that you do not use other channels or contact the project maintainers +directly. + +Non-vulnerability related security issues including ideas for new or improved +security features are welcome on GitHub Issues. + +## Security updates, alerts and bulletins + +Security updates will be released on a regular cadence. Many of our projects +will typically release security fixes in conjunction with the +Oracle Critical Patch Update program. Additional +information, including past advisories, is available on our [security alerts][4] +page. + +## Security-related information + +We will provide security related information such as a threat model, considerations +for secure use, or any known security issues in our documentation. Please note +that labs and sample code are intended to demonstrate a concept and may not be +sufficiently hardened for production use. + +[1]: mailto:secalert_us@oracle.com +[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html +[3]: https://www.oracle.com/security-alerts/encryptionkey.html +[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file diff --git a/ai/generative-ai-service/mindmaps/VBCS Project/MindMaps.zip b/ai/generative-ai-service/mindmaps/VBCS Project/MindMaps.zip new file mode 100644 index 000000000..82869a9d3 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/VBCS Project/MindMaps.zip differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/ak1.png b/ai/generative-ai-service/mindmaps/mindmaps-md/ak1.png new file mode 100644 index 000000000..4abc5db40 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/ak1.png differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/ak2.png b/ai/generative-ai-service/mindmaps/mindmaps-md/ak2.png new file mode 100644 index 000000000..3fb313126 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/ak2.png differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/ak3.png b/ai/generative-ai-service/mindmaps/mindmaps-md/ak3.png new file mode 100644 index 000000000..40f3ce03d Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/ak3.png differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/ak4.png b/ai/generative-ai-service/mindmaps/mindmaps-md/ak4.png new file mode 100644 index 000000000..576edb2c8 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/ak4.png differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/ak5.png b/ai/generative-ai-service/mindmaps/mindmaps-md/ak5.png new file mode 100644 index 000000000..3c8b57959 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/ak5.png differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/mindmap.jpg b/ai/generative-ai-service/mindmaps/mindmaps-md/mindmap.jpg new file mode 100644 index 000000000..3f28bf37b Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/mindmap.jpg differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/preview.jpg b/ai/generative-ai-service/mindmaps/mindmaps-md/preview.jpg new file mode 100644 index 000000000..c1f23f048 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/preview.jpg differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/services.jpg b/ai/generative-ai-service/mindmaps/mindmaps-md/services.jpg new file mode 100644 index 000000000..f6ef1e713 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/services.jpg differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/signature.jpg b/ai/generative-ai-service/mindmaps/mindmaps-md/signature.jpg new file mode 100644 index 000000000..079b74b02 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/signature.jpg differ diff --git a/ai/generative-ai-service/mindmaps/mindmaps-md/variables.jpg b/ai/generative-ai-service/mindmaps/mindmaps-md/variables.jpg new file mode 100644 index 000000000..2c99766e8 Binary files /dev/null and b/ai/generative-ai-service/mindmaps/mindmaps-md/variables.jpg differ diff --git a/ai/generative-ai-service/oci_langflow/LICENSE b/ai/generative-ai-service/oci_langflow/LICENSE new file mode 100644 index 000000000..62c949c4e --- /dev/null +++ b/ai/generative-ai-service/oci_langflow/LICENSE @@ -0,0 +1,35 @@ +Copyright (c) 2024 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/ai/generative-ai-service/oci_langflow/README.md b/ai/generative-ai-service/oci_langflow/README.md new file mode 100644 index 000000000..73d5d5dd6 --- /dev/null +++ b/ai/generative-ai-service/oci_langflow/README.md @@ -0,0 +1,17 @@ +# Integration of OCI Generative AI in Langflow + +This repository contains all the code for a prototype of the integration of OCI Generative AI in Langflow + +Reviewed: 23.01.2025 + +# **Link to code** + [Code](https://github.com/luigisaetta/oci_langflow/tree/main) + +# Owner Luigi Saeta +# License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/ai/generative-ai-service/oci_langflow/SECURITY.md b/ai/generative-ai-service/oci_langflow/SECURITY.md new file mode 100644 index 000000000..fb42c9453 --- /dev/null +++ b/ai/generative-ai-service/oci_langflow/SECURITY.md @@ -0,0 +1,38 @@ +# Reporting security vulnerabilities + +Oracle values the independent security research community and believes that +responsible disclosure of security vulnerabilities helps us ensure the security +and privacy of all our users. + +Please do NOT raise a GitHub Issue to report a security vulnerability. If you +believe you have found a security vulnerability, please submit a report to +[secalert_us@oracle.com][1] preferably with a proof of concept. Please review +some additional information on [how to report security vulnerabilities to Oracle][2]. +We encourage people who contact Oracle Security to use email encryption using +[our encryption key][3]. + +We ask that you do not use other channels or contact the project maintainers +directly. + +Non-vulnerability related security issues including ideas for new or improved +security features are welcome on GitHub Issues. + +## Security updates, alerts and bulletins + +Security updates will be released on a regular cadence. Many of our projects +will typically release security fixes in conjunction with the +Oracle Critical Patch Update program. Additional +information, including past advisories, is available on our [security alerts][4] +page. + +## Security-related information + +We will provide security related information such as a threat model, considerations +for secure use, or any known security issues in our documentation. Please note +that labs and sample code are intended to demonstrate a concept and may not be +sufficiently hardened for production use. + +[1]: mailto:secalert_us@oracle.com +[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html +[3]: https://www.oracle.com/security-alerts/encryptionkey.html +[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file diff --git a/ai/generative-ai-service/rag-genai/files/LangChainRAG.py b/ai/generative-ai-service/rag-genai/files/LangChainRAG.py index 1825824c4..fdc32b96c 100644 --- a/ai/generative-ai-service/rag-genai/files/LangChainRAG.py +++ b/ai/generative-ai-service/rag-genai/files/LangChainRAG.py @@ -4,7 +4,7 @@ from langchain_core.prompts import PromptTemplate from langchain_community.llms import OCIGenAI from langchain_community.document_loaders import UnstructuredURLLoader -compartment_id = "ocid1.compartment.oc1..aaaaaaaa7ggqkd4ptkeb7ugk6ipsl3gqjofhkr6yacluwj4fitf2ufrdm65q" +compartment_id = "" embeddings = OCIGenAIEmbeddings(model_id="cohere.embed-english-light-v3.0",service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",compartment_id= compartment_id,) testurls = ['https://docs.oracle.com/iaas/odsaz/odsa-rotate-wallet.html', 'https://docs.oracle.com/iaas/odsaz/odsa-change-password.html', 'https://docs.oracle.com/iaas/odsaz/odsa-database-actions.html'] loader = UnstructuredURLLoader(urls=testurls) diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/LICENSE b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/LICENSE new file mode 100644 index 000000000..9b47cfba2 --- /dev/null +++ b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/LICENSE @@ -0,0 +1,35 @@ +Copyright (c) 2024 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. \ No newline at end of file diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.md b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.md new file mode 100644 index 000000000..56ae2267c --- /dev/null +++ b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.md @@ -0,0 +1,26 @@ +# OIC Adapters clickthrough presentations + +Assets that contain oic adapters configuration and implementation practice for the +- EPM Adapter +- ERP Adapter +- HCM Adapter +- Kafka Adapter +- EBS Adapter + +Review Date: 28.11.2024 + +# When to use these assets? + +These assets should be used whenever needed to design solutions with mentioned applications and resources integration. + +# How to use these asset? + +The information is generic in nature and not specified for a particular customer. + +# License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.pdf new file mode 100644 index 000000000..7cfbf55bd Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/README.pdf differ diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Kafka Adapter click through presentation.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Kafka Adapter click through presentation.pdf new file mode 100644 index 000000000..75bb1dbd4 Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Kafka Adapter click through presentation.pdf differ diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EBS Adapter click through presentation.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EBS Adapter click through presentation.pdf new file mode 100644 index 000000000..98d122d6f Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EBS Adapter click through presentation.pdf differ diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EPM Cloud Adapter Sep single slider presentation.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EPM Cloud Adapter Sep single slider presentation.pdf new file mode 100644 index 000000000..40bc2250e Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle EPM Cloud Adapter Sep single slider presentation.pdf differ diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle ERP Cloud Adapter click through presentation.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle ERP Cloud Adapter click through presentation.pdf new file mode 100644 index 000000000..4f30d4166 Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle ERP Cloud Adapter click through presentation.pdf differ diff --git a/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle HCM Cloud Adapter click through presentation.pdf b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle HCM Cloud Adapter click through presentation.pdf new file mode 100644 index 000000000..641810cb5 Binary files /dev/null and b/app-dev/app-integration-and-automation/oracle-integration-cloud/05-oic-adapters-clickthrough/files/Oracle HCM Cloud Adapter click through presentation.pdf differ diff --git a/app-dev/devops-and-containers/devops/azure-devops-oke/README.md b/app-dev/devops-and-containers/devops/azure-devops-oke/README.md index 9b1d4b26a..096787083 100644 --- a/app-dev/devops-and-containers/devops/azure-devops-oke/README.md +++ b/app-dev/devops-and-containers/devops/azure-devops-oke/README.md @@ -149,7 +149,7 @@ Calling the LB endpoint with cURL or from the browser should show an echo:

-curl 144.200.51.195
+curl xxx.200.51.195
 {
   "path": "/",
   "headers": {
diff --git a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/README.md b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/README.md
index fabc38c16..0fe75dc84 100644
--- a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/README.md
+++ b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/README.md
@@ -64,7 +64,7 @@ IAM user option will work on both cases above, as local and as OCI Function.
 
 ## Build and test
 
-During following the steps of the Hello function example  adjust the line 93 to match your GenAI service OCID. 
+During following the steps of the Hello function example  adjust the line 131 to match your GenAI service model OCID. 
 
 

@@ -74,21 +74,56 @@ Testing with curl (or copy-pasting the API Gateway deployment url to a browser):

 curl https://n3yu.....ghhi.apigateway.eu-frankfurt-1.oci.customer-oci.com/
-What happened today 06/13/2024 100 years ago ?
 
-June 13, 2024 is June 13, 1924. Here are some events that occurred on this date:
+What happened today 01/17/2025 100 years ago ?
+On January 17th, 1925, 100 years ago, the following events took place:
+- In the US, President Calvin Coolidge delivered his annual State of the Union address to Congress. He discussed the thriving state of the national economy, emphasizing the record-high production of American industries and the growth of the country's merchant marine fleet. Coolidge also urged Congress to pass legislation facilitating world trade and improving diplomatic relations.
+- The first Winter Sports Week was held in Chamonix, France. This event eventually evolved into the prestigious Chamonix International Festival of Sports and Cinema.
+- The play "The New York Idea" by Langdon Mitchell premiered at the Ambassador Theatre on Broadway. It ran for 144 performances and received critical acclaim.
+- The silent film "The Gold Rush" directed by Charlie Chaplin was released in the United States. It's a classic comedy that tells the story of a prospector during the Klondike Gold Rush. Chaplin's unique brand of physical comedy and the film's innovative effects delighted audiences.
+- In Germany, the Weimar Republic experienced a political scandal known as the "German-Russian Trade and Credit Agreement." The agreement, which granted Germany a loan of 300 million marks from Russia, was signed secretly, leading to accusations of mismanagement and lack of transparency in the government. This incident further destabilized the already fragile Weimar Republic.
+
-1. The German airline Deutsche Lufthansa (DL) was officially founded on June 13, 1924. It is Germany's largest airline and is one of the world's largest airlines in terms of overall passengers carried. - -2. The British Broadcasting Corporation (BBC) aired its first radio broadcast. +# Native image using GraalVM + +GraalVM compiles your Java functions ahead of time into standalone binaries that start instantly, provide peak performance with no warmup, and use fewer resources. The key GraalVM benefits are: Low Resource Usage: Java applications compiled ahead-of-time by GraalVM require less memory and CPU to run. + +

+ +To do this a Docker multi-stage build is used. + +

+ +Before building the native image let's do a full maven build for the project to create the necessary libraries under target/lib: + +

+mvn clean install
+
-3. The Hollywood Sign was officially dedicated in California, marking the beginning of Hollywood's rise to prominence in the American film industry. +Then build the Docker container using multi-stage Docker file including the GraalVM native image compilation: -4. The Ford Motor Company introduced the firstassembly line to mass produce cars, making cars more accessible to the general public. +
+docker build -f Dockerfile.native -t fra.ocir.io/<YOUR OCI TENANCY NAMESPACE>/helloworldai-java:2 .
+
+ +The GraalVM compilation stage requires quite a bit resources from your localhost so in case for example using Rancher desktop +think of increasing the CPU and memory for it to make the build faster. + +

+ +In the Dockerfile.native two things are important: Including the reflection.json with the proper function class name and passing Fn FDK libraries with "-Djava.library.path=/lib" in the container CMD along with the "com.example.HelloAIFunction::handleRequest" function handler. -5. The German aerospace company Zeppelin began constructing the Hindenburg, a large passenger airship. +

+ +After the build push the container to OCIR repo: + +

+docker push fra.ocir.io/<YOUR OCI TENANCY NAMESPACE>/helloworldai-java:2
 
+Finally deploy the container to your OCI Function by replacing the container using the Cloud UI by editing the function and changing the container from helloworldai-java:1 to helloworldai-java:2. Then test it. + + # Useful Links - [OCI Functions](https://docs.oracle.com/en-us/iaas/Content/Functions/Concepts/functionsoverview.htm) @@ -99,6 +134,8 @@ June 13, 2024 is June 13, 1924. Here are some events that occurred on this date: - The Fn project is an open-source container-native serverless platform that you can run anywhere -- any cloud or on-premise. It’s easy to use, supports every programming language, and is extensible and performant - [OCI GenAI](https://www.oracle.com/artificial-intelligence/generative-ai/generative-ai-service/) - Discover the power of generative AI models equipped with advanced language comprehension for building the next generation of enterprise applications. Oracle Cloud Infrastructure (OCI) Generative AI is a fully managed service for seamlessly integrating these versatile language models into a wide range of use cases, including writing assistance, summarization, analysis, and chat +- [OCI Functions with GraalVM](https://github.com/shaunsmith/graalvm-fn-init-images) + - Discover GraalVM Native Image -based functions with this example GitHub repo - [Oracle](https://www.oracle.com/) - Oracle Website diff --git a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/Dockerfile.native b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/Dockerfile.native new file mode 100644 index 000000000..bb28b71f5 --- /dev/null +++ b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/Dockerfile.native @@ -0,0 +1,20 @@ +FROM container-registry.oracle.com/graalvm/native-image:23-ol8 AS native +WORKDIR /app + +COPY target . +ADD reflection.json . + +RUN native-image \ + -H:ReflectionConfigurationFiles=/app/reflection.json \ + -Ob \ + -H:Name=Hello \ + -cp "/app/Helloaifunc-1.0-SNAPSHOT.jar:/app/lib/*" \ + com.fnproject.fn.runtime.EntryPoint + +FROM fnproject/fn-java-fdk:jre17-1.0.198 as fdk + +FROM container-registry.oracle.com/os/oraclelinux:8-slim +COPY --from=native /app/Hello . +COPY --from=fdk /function/runtime/* . +ENTRYPOINT [ "./Hello" ] +CMD [ "com.example.HelloAIFunction::handleRequest", "-Djava.library.path=/lib"] \ No newline at end of file diff --git a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/pom.xml b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/pom.xml index 8fa8354c5..46e617e3d 100644 --- a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/pom.xml +++ b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/pom.xml @@ -8,6 +8,26 @@ Helloaifunc 1.0-SNAPSHOT + + + + + maven-dependency-plugin + + + install + + copy-dependencies + + + ${project.build.directory}/lib + + + + + + + 17 17 @@ -33,7 +53,7 @@ junit junit - 4.13.1 + 4.12 test @@ -67,6 +87,11 @@ runtime 3.37.2 + + org.bouncycastle + bcprov-jdk18on + 1.79 + \ No newline at end of file diff --git a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/reflection.json b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/reflection.json new file mode 100644 index 000000000..a3d8afc4e --- /dev/null +++ b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/reflection.json @@ -0,0 +1,10 @@ +[ + { + "name": "com.example.HelloAIFunction", + "allDeclaredMethods": true, + "methods": [ + { "name": "", "parameterTypes": [] } + ] + } +] + diff --git a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/src/main/java/com/example/HelloAIFunction.java b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/src/main/java/com/example/HelloAIFunction.java index 40d1bdc7c..1d96d5e60 100644 --- a/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/src/main/java/com/example/HelloAIFunction.java +++ b/app-dev/devops-and-containers/functions/java-helloworld-AI-with-local-dev-and-oci-functions/files/src/main/java/com/example/HelloAIFunction.java @@ -1,3 +1,41 @@ +/* +Copyright (c) 2021 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +*/ + package com.example; import com.oracle.bmc.ClientConfiguration; @@ -32,7 +70,7 @@ public class HelloAIFunction { // FILL IN PROPER VALUES FOR OCI GENAI SERVICE - private static final String ENDPOINT = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com"; + private static final String ENDPOINT = "https://inference.generativeai.eu-frankfurt-1.oci.oraclecloud.com"; private static final Region REGION = Region.EU_FRANKFURT_1; // FILL IN PROPER VALUES FOR IAM USER WHEN NOT USING INSTANCE_PRINCIPAL IN OCI FUNCTION diff --git a/app-dev/devops-and-containers/functions/java-helloworld-with-local-dev-and-oci-functions/README.md b/app-dev/devops-and-containers/functions/java-helloworld-with-local-dev-and-oci-functions/README.md index 44adde329..8807f6c66 100644 --- a/app-dev/devops-and-containers/functions/java-helloworld-with-local-dev-and-oci-functions/README.md +++ b/app-dev/devops-and-containers/functions/java-helloworld-with-local-dev-and-oci-functions/README.md @@ -130,8 +130,6 @@ docker login ams.ocir.io -u '<YOUR OCI TENANCY NAMESPACE>/oracleidentitycl docker push fra.ocir.io/<YOUR OCI TENANCY NAMESPACE>/helloworld-java:1
-

- The last step is to create the Function Application and the function deployment for it. This can be easily done using the Cloud UI.

diff --git a/cloud-architecture/oracle-apps-erp/19c-upgrade-dbcli/README.md b/cloud-architecture/oracle-apps-erp/19c-upgrade-dbcli/README.md index 4e53a1934..bd082d84f 100644 --- a/cloud-architecture/oracle-apps-erp/19c-upgrade-dbcli/README.md +++ b/cloud-architecture/oracle-apps-erp/19c-upgrade-dbcli/README.md @@ -2,6 +2,8 @@ By implementing the best practices mentioned in the document before commencing the migration and upgrade process, significant downtime savings can be achieved at various stages. The best practices provide recommendations for successfully migrating a 12c on-premises database to a VM DB system (DBCS) in OCI while simultaneously upgrading the database to 19c using the dbcli utility. +Reviewed:03.12.2024 + ## When to use this asset? This document is applicable if you are considering migration and upgrade of your on-premises Non-CDB 12c Database or database running on a compute VM in OCI to a 19c VM DB system (DBCS). diff --git a/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/rag-langchain-vllm-mistral/files/requirements.txt b/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/rag-langchain-vllm-mistral/files/requirements.txt index a20c1aec6..1bf79d032 100644 --- a/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/rag-langchain-vllm-mistral/files/requirements.txt +++ b/cloud-infrastructure/ai-infra-gpu/ai-infrastructure/rag-langchain-vllm-mistral/files/requirements.txt @@ -41,7 +41,7 @@ hyperframe==6.0.1 idna==3.7 interegular==0.3.3 jieba3k==0.35.1 -Jinja2==3.1.4 +Jinja2==3.1.5 joblib==1.4.0 jsonpatch==1.33 jsonpointer==2.4 diff --git a/cloud-infrastructure/networking/peering/README.md b/cloud-infrastructure/networking/peering/README.md index e46ac6abb..edb9c8b0b 100644 --- a/cloud-infrastructure/networking/peering/README.md +++ b/cloud-infrastructure/networking/peering/README.md @@ -18,7 +18,7 @@ Reviewed: 30.10.2024 - [LPG Part 2](https://blogs.oracle.com/cloud-infrastructure/post/easily-connect-isolated-networks-using-oracle-cloud-infrastructures-vcn-peering-solution-part-2) - [LPG inter tenancy](https://www.ateam-oracle.com/post/inter-tenancy-vcn-peering-using-remote-peering-connection) - [RPC 2 Regions](https://learnoci.cloud/how-to-connect-2-vcns-in-different-regions-using-remote-peering-connection-decac8b9e4de) - +- [RPC IAM Policy Tool](https://iwanhoogendoorn.nl/rpc-iam-policy-creator/) ## Videos & Podcasts - [LPG video tutorial](https://www.youtube.com/watch?v=kO1UlrwffgM) diff --git a/cloud-infrastructure/private-cloud-and-edge/README.md b/cloud-infrastructure/private-cloud-and-edge/README.md index 5ba015f3b..69762e2ae 100644 --- a/cloud-infrastructure/private-cloud-and-edge/README.md +++ b/cloud-infrastructure/private-cloud-and-edge/README.md @@ -1,11 +1,11 @@ # Private Cloud and Edge -Reviewed: 18.11.2024 +Reviewed: 02.12.2024 ## Useful Links -- [Oracle Compute Cloud@Customer](https://www.oracle.com/uk/cloud/compute/cloud-at-customer/) -- [Roving Edge Infrastructure](https://www.oracle.com/uk/cloud/roving-edge-infrastructure/) +- [Oracle Compute Cloud@Customer on oracle.com](https://www.oracle.com/cloud/compute/cloud-at-customer/) +- [Roving Edge Infrastructure on oracle.com](https://www.oracle.com/cloud/roving-edge-infrastructure/) ## License diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/README.md index 6b7546539..531162921 100644 --- a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/README.md +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/README.md @@ -4,7 +4,9 @@ Reviewed: 18.11.2024 ## Useful Links -- [Oracle Compute Cloud@Customer](https://www.oracle.com/uk/cloud/compute/cloud-at-customer/) +- [Oracle Compute Cloud@Customer on oracle.com](https://www.oracle.com/cloud/compute/cloud-at-customer/) +- [Oracle Compute Cloud@Customer documentation](https://docs.oracle.com/en-us/iaas/compute-cloud-at-customer/) +- [Oracle Compute Cloud@Customer X10 datasheet](https://www.oracle.com/a/ocom/docs/oracle-cloud-at-customer-datasheet.pdf) ## License diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/README.md index 8b1378917..418d41c9d 100644 --- a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/README.md +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/README.md @@ -1 +1,7 @@ +## License +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/iam-policies-for-isolation/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/iam-policies-for-isolation/README.md index 299fe81b9..47188621f 100644 --- a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/iam-policies-for-isolation/README.md +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/iam-policies-for-isolation/README.md @@ -96,4 +96,10 @@ HSP users on the C3): allow group CommonGroup to read all-resources in tenancy where target.compartment.name='root-compartment-name' ``` +## License +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/restrict-access-to-oci-console/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/restrict-access-to-oci-console/README.md index 2a27d3bd4..d90c41b00 100644 --- a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/restrict-access-to-oci-console/README.md +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/hsp/restrict-access-to-oci-console/README.md @@ -32,3 +32,11 @@ as per .  Ideally a policy should be implemented that restricts permissions **only** to the C3, see [this](../iam-policies-for-isolation/README.md) note. + +## License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/README.md new file mode 100644 index 000000000..6a83f51eb --- /dev/null +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/README.md @@ -0,0 +1,326 @@ +*Last Update: 27 November 2024* + +

Local LLM Inferencing and Interaction
Using the Ollama Open Source Tool

+

+ + + + +## Table of Content +1. [Introduction](#intro) +2. [Preparation](#prep) +3. [Security](#security) +4. [Ollama Installation](#install) +5. [Testing](#test) +6. [Useful References](#ref) + + +## Introduction + +[Ollama](https://github.com/ollama/ollama) is an open-source tool that runs large language models (LLMs) directly on a local machine. This makes it particularly appealing to AI developers, researchers, and businesses concerned with data control and privacy. It enables the loading and deployment of selected LLMs and provides access to them through APIs and freely obtainable chatbots. Ollama also contains a text-based server-side chatbot. + +By running models locally, you maintain full data ownership and avoid the potential security risks associated with cloud storage. Offline AI tools like Ollama also help reduce latency and reliance on external facilities, making them faster and more reliable. + +This article is intended to demonstrate and provide directions to install and create an Ollama LLM processing facility. Despite the fact that Ollama can be run on both personal servers and laptops, this installation is aimed at the Oracle Compute Cloud@Customer (C3) and Private Cloud Appliance (PCA) to capitalize on more readily available resources to increase performance and processing efficiency, especially if large models are used. + +Considerations: +* A firm grasp of C3/PCA/OCI concepts and administration is assumed. +* The creation and integration of a development environment is outside of the scope of this document. +* Oracle Linux 8 and macOS Sonoma 14.7.1 clients were used for testing but Windows is however widely supported. + +[Back to top](#toc)
+
+ + +## Preparation + +### System Requirements + +| Requirement | Specification | +|----------|----------| +| Operating system | Oracle Linux 8 or later
Ubuntu 22.04 or later
Windows
| +| RAM | 16 GB for running models up to 7B. "The rule of thumb" is to have at least 2x memory for the size of the LLM, also allowing for LLMs that will be loaded in memory simultaneously. | +| Disk space | 12 GB for installing Ollama and basic models. Additional space is required for storing model data depending on the used models. The LLM sizes can be obtained from the "trained models" link in the References section. For example the Llama 3.1 LLM with 405Bn parameters occupy 229GB of disk space | +| Processor | Recommended to use a modern CPU with at least 4 cores. For running models of approximately 15B, 8 cores (OCPUs) is recommended. Allocate accordingly | +| Graphics Processing Unit
(optional) | A GPU is not required for running Ollama, but can improve performance, especially when working with large models. If you have a GPU, you can use it to accelerate training of custom models. | + +>[!NOTE] +>The GPU options in the Compute Cloud@Customer will be available soon. + +### Create a Virtual Machine Instance + +[C3: Creating an Instance](https://docs.oracle.com/en-us/iaas/compute-cloud-at-customer/topics/compute/creating-an-instance.htm#creating-an-instance)
+[PCA 3.0: Working with Instances](https://docs.oracle.com/en/engineered-systems/private-cloud-appliance/3.0-latest/user/user-usr-instance-lifecycle.html) + +Create a VM in a public subnet following these guidelines: + +1. Hostname `llm-host` +2. Select the Oracle Linux 8 image +3. Start with 6x OCPUs and 96GB of RAM using an available "Flex" shape (resources can be adjusted later depending on workload) +5. Select the default boot volume size +6. Select a public subnet and allow a public IP address to be assigned +7. Configure the public key information +8. Select "Restore instance lifecycle state after infrastructure maintenance" +9. Apply appropriate tagging if required +10. Ensure that the VM is accessible via `ssh` +11. Configure the proxy setup if required (described below) +12. Update your local host's `/etc/hosts` file to reflect your public IP address for `llm-host` +13. Perform an OS update in `llm-host` before proceeding: + +``` +sudo dnf update +``` + +### Create a Block Storage Device for LLMs + +[C3: Creating and Attaching Block Volumes](https://docs.oracle.com/en-us/iaas/compute-cloud-at-customer/topics/block/creating-and-attaching-block-volumes.htm)
+[PCA 3.0: Creating and Attaching Block Volumes](https://docs.oracle.com/en/engineered-systems/private-cloud-appliance/3.0-latest/user/user-usr-blk-volume-create-attach.html) + +1. Create and attach a block volume to the VM +2. Volume name `llm-repo` +3. A block volume of at least 150GB (research the model sizes!) in size is recommended and should multiple standard LLMs be hosted or larger if advanced workloads are foreseen, e.g. copies of LLMs for development, collection and loading of RAG material, etc the recommendation is 1TB. +4. Select High Performance if available +5. Select your appropriate backup policy +6. Apply appropriate tagging if required +7. It is recommended to use the `xfs` filesystem for formatting the block volume +8. Configure a persistent mount point (to survive reboots). The entry in the `/etc/fstab` file will typically resemble the following: +`/dev/disk/by-id/scsi-3600144f096933b92000061b1129e0037 /mnt/llm-repo xfs _netdev,nofail 0 0` +9. To set initial unlimited access to the mounted filesystem perform the command on the mount point: +``` +sudo chmod 777 /mnt/llm-repo +``` +>[!IMPORTANT] +>Note the mount options in the `/etc/fstab` file + +### Proxy Settings + +In the event of a proxy'd network add the following to the `/etc/profile.d/proxy.sh` file to set the proxy environment variables system-wide: + +``` +http_proxy=http://:80 +https_proxy=http://:80 +no_proxy="127.0.0.1, localhost" +export http_proxy +export https_proxy +export no_proxy +``` + +>[!TIP] +>The `no_proxy` environment variable can be expanded to include your internal domains. It is not required to list IP addresses in internal subnets of the C3/PCA. + +Edit the `/etc/yum.conf` file to include the following line: +``` +proxy=http://:80 +``` + +[Back to top](#toc)
+
+ + +## Security + +### General + +Resource constraints are often experienced on personal computers and more compute, memory and disk resources are required to run LLM operations more efficiently. Hence the increasing deployment of Ollama in cloud- or corporate hardware environments in a 2-tier client-server architecture. The security architecture of Ollama for this deployment architecture exposes the installation and data to a number of vulnerabilities. + +Several API endpoint vulnerabilities have been identified in the client-server deployment model of Ollama and some have been addressed successfully by means of security patching. Collectively, the vulnerabilities could allow an attacker to carry out a wide-range of malicious actions with a single HTTP request, including denial-of-service (DoS) attacks, model poisoning, model theft, and more. + +*A future article will describe a secure (using reverse proxying and TLS) client-server deployment architecture that can be made available for secure corporate use that also ensures data usage privacy.* + +>[!NOTE] +>Refer to the article [Why You Should Trust Meta AI's Ollama for Data Security](https://myscale.com/blog/trust-meta-ai-ollama-data-security) for further information on the benefits of running LLMs locally. + +### Open the Firewall for the Ollama Listening Port + +``` +sudo firewall-cmd –-set-default-zone=public +``` +``` +sudo firewall-cmd –-add-port=11434/tcp --add-service=http –-zone=public +``` +``` +sudo firewall-cmd --runtime-to-permanent +``` +``` +sudo firewall-cmd –reload +``` +``` +sudo firewall-cmd –-info-zone=public +``` + +

+ +### Grant VCN Access through Security List + +Edit the VCN default security list to reflect the following: + +

+ +Should you want to limit the access to a specific IP address the source should be: + +

+ +>[!TIP] +>To avoid continuous changes to the security list obtain a reserved IP address for your client machine from the network administrator. + +[Back to top](#toc)
+
+ + +## Ollama Installation + +### General + +The installation comprises the following components: + +| Server | Client | +|----------|----------| +| Ollama | GUI Tools1
Character based tools
API Development kits2 | + +1 Examples of GUIs: [Msty](https://msty.app/), [OpenWebUI](https://openwebui.com/), [ollama-chats](https://github.com/drazdra/ollama-chats)
+2 See [Ollama documentation](https://github.com/ollama/ollama/tree/main/docs) + +>[!IMPORTANT] +>When GPU's become available the NVIDIA and CUDA drivers should be installed. This configuration will also be tested on the Roving Edge Device GPU model. + +### Installation + +``` +cd /tmp +curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz +sudo tar -C /usr -xzf ollama-linux-amd64.tgz +sudo chmod +x /usr/bin/ollama +sudo useradd -r -s /bin/false -m -d /usr/share/ollama ollama +``` +``` +sudo tee /usr/lib/systemd/system/ollama.service > /dev/null < +For Nvidia GPUs, add `Environment="OLLAMA_FLASH_ATTENTION=1"` to improve token generation speed. + +Enable and start Ollama: +``` +sudo systemctl daemon-reload +sudo systemctl enable ollama +sudo systemctl start ollama +``` + +Ollama will be accessible at http://127.0.0.1:11434 or http://:11434. + +Execute: +``` +sudo chown ollama:ollama /mnt/llm-repo +sudo chmod 755 /mnt/llm-repo +``` + +[Back to top](#toc)
+
+ + + +## Testing + +From the local host, test the accessibility of the port and the availability of the Ollama server: + +``` +nc -zv llm-host 11434 +curl http://llm-host:11434 +curl -I http://llm-host:11434 +``` +

+ +Login to `llm-host` and note the command line options that are available: + +``` +ollama +``` + +

+ +Also note the environment variable options that are available: + +``` +ollama help serve +``` + +

+ +Download and test your first LLM (and you will notice the population of `/mnt/llm-repo` with data by running `ls -lR /mnt/llm-repo`): + +

+ +Run some more tests from your client to test the APIs: + +``` +$ curl http://llm-host:11434/api/tags +$ curl http://llm-host:11434/api/ps +$ curl -X POST http://llm-host:11434/api/generate -d '{ + "model": "mistral", + "prompt":"Hello Mistral!", + "stream": false + }' +``` + +

+ +1. `curl http://llm-host:11434/api/tags` returns a list of installed LLMs +2. `curl http://llm-host:11434/api/ps` returns a list of LLMs already loaded into memory + +>[!TIP] +>The duration that the LLM can stay loaded into memory can be adjusted by changing the `OLLAMA_KEEP_ALIVE` environment parameter (default = 5 mins). + +Install any of the GUI clients mentioned previously and test the connectivity and accessibility. For using [the Msty app](https://msty.app/) you need to: + +1. Create a Remote Models Provider +2. Name it appropriately +3. The Service Endpoint is `http://llm-host:11434` +4. "Fetch Models" (that are already installed, in this case `mistral`) +5. This step can be repeated as new models are added from the Ollama model repository + +Example output as follows: + +

+ +[Back to top](#toc)
+
+ + +## Useful References + +* [Ollama documentation](https://github.com/ollama/ollama/tree/main/docs) +* [Pre-trained Ollama models](https://ollama.com/library) +* [Msty GUI client](https://msty.app/) +* [OpenWebUI](https://github.com/open-webui/open-webui) +* [Ollama-chats](https://github.com/drazdra/ollama-chats) +* [Ollama Python library](https://github.com/ollama/ollama-python) +* [Getting started with Ollama for Python](https://github.com/RamiKrispin/ollama-poc) +* [Ollama and Oracle Database 23ai vector search](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/generate-summary-using-ollama.html) + +[Back to top](#toc)
+
+ +## License +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](LICENSE) for more details. diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/README.md new file mode 100644 index 000000000..f2f777380 --- /dev/null +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/README.md @@ -0,0 +1,7 @@ +## License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/firewall-info.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/firewall-info.png new file mode 100644 index 000000000..ffda059d1 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/firewall-info.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/msty-example.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/msty-example.png new file mode 100644 index 000000000..9a09a2ecf Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/msty-example.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-curl-statements.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-curl-statements.png new file mode 100644 index 000000000..49bc74cf1 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-curl-statements.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-env-var.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-env-var.png new file mode 100644 index 000000000..1dbea63ee Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-env-var.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-logo.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-logo.png new file mode 100644 index 000000000..85dd87b7f Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-logo.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-and-test.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-and-test.png new file mode 100644 index 000000000..b8a409712 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-and-test.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-model.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-model.png new file mode 100644 index 000000000..8fe4bf223 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-pull-model.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-remote-test.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-remote-test.png new file mode 100644 index 000000000..3ef9537e1 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-remote-test.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-syntax.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-syntax.png new file mode 100644 index 000000000..1df8e31dd Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/ollama-syntax.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list-individual.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list-individual.png new file mode 100644 index 000000000..13839bac7 Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list-individual.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list.png b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list.png new file mode 100644 index 000000000..829f8c37e Binary files /dev/null and b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/local-llm/images/security-list.png differ diff --git a/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/mos-notes/README.md b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/mos-notes/README.md new file mode 100644 index 000000000..7ee689de3 --- /dev/null +++ b/cloud-infrastructure/private-cloud-and-edge/compute-cloud-at-customer/mos-notes/README.md @@ -0,0 +1,31 @@ +# Compute Cloud@Customer Support Notes + +Reviewed: 29.11.2024 + +Collection of My Cloud Oracle Support (MCOS) & My Oracle Support (MOS) notes relevant to Compute Cloud@Customer (CCATC) and Private Cloud Appliance (PCA) notes that are relevant to CCATC. The collection will continue to be updated as needed and is intended for use by CCATC administrators or anyone working on the CCATC service. + +# My Oracle Cloud Support + +- [[PCA 3.x] Support Process for Private Cloud Appliace and Compute Cloud@Customer Technical Briefs and 3rd Party Solutions](https://support.oracle.com/ic/builder/rt/customer_portal/live/webApps/customer-portal/?kmExternalId=KB159007) +- [[CCATC] How to Re-bootstrap a Compute Cloud @ Customer Rack (Doc ID 3010344.1)](https://support.oracle.com/ic/builder/rt/customer_portal/live/webApps/customer-portal/?anchorId=&documentId=KB161124) + +# My Oracle Support + +- [Information Center: [CCATC] Oracle Compute Cloud at Customer (Doc ID 2936731.2)]() +- [[CCATC] Application Verification Matrix (Doc ID 2966540.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2966540.1) +- [[CCATC] Network Validation Tool (Doc ID 2974831.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2974831.1) +- [[CCATC] OCI Compute Cloud at Customer - Service Request (SR) Creation (Doc ID 2969961.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2969961.1) +- [[PCA 3.x] How to Setup and Gather Periscope Data (Doc ID 3023200.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=3023200.1) +- [[CCATC] Oracle Compute Cloud at Customer Federation (Doc ID 3004017.2)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=3004017.2) +- [[CCATC] How to Setup IDP as a Provider for the Compute Cloud at Customer (Doc ID 3006544.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=3006544.1) +- [[CCATC] How to Update a Compute Cloud@Customer x509 Certificate in OCI's IDP (Doc ID 3056675.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=3056675.1) +- [[CCATC] How to Re-bootstrap a Compute Cloud @ Customer Rack (Doc ID 3010344.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=3010344.1) +- [[PCA 3.x] Private Cloud Appliance: Software Updates (Doc ID 2906831.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2906831.1) + +## License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/README.md b/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/README.md new file mode 100644 index 000000000..9abc659f1 --- /dev/null +++ b/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/README.md @@ -0,0 +1,19 @@ +# Roving Edge Infrastructure + +Reviewed: 2.12.2024 + +## Useful Links + +- [Roving Edge Infrastructure on oracle.com](https://www.oracle.com/cloud/roving-edge-infrastructure/) +- [Roving Edge Infrastructure documentation](https://docs.oracle.com/en-us/iaas/Content/Rover/) +- [Roving Edge Device (2nd generation) datasheet](https://www.oracle.com/a/ocom/docs/data-sheet-roving-edge-device.pdf) +- [Roving Edge Ultra datasheet](https://www.oracle.com/a/ocom/docs/data-sheet-roving-edge-ultra.pdf) +- [Roving Edge Station datasheet](https://www.oracle.com/a/ocom/docs/data-sheet-roving-edge-station.pdf) + +## License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/blogs/README.md b/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/blogs/README.md new file mode 100644 index 000000000..0096192c3 --- /dev/null +++ b/cloud-infrastructure/private-cloud-and-edge/roving-edge-infrastructure/blogs/README.md @@ -0,0 +1,53 @@ +# Roving Edge Infrastructure + +Reviewed: 02.12.2024 + +## Blogs, Press Releases, News Articles, Videos & Podcasts + +Blogs / Press Releases +- https://blogs.oracle.com/cloud-infrastructure/post/performance-enhancements-next-gen-roving-edge-red (10/Oct/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/2024-gartner-mq-distributed-hybrid-infrastructure (10/Oct/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/new-capabilities-at-the-edge-key-takeaways-from-oracle-cloudworld-2024 (30/Sep/2024) +- https://www.oracle.com/news/announcement/ocw24-oracle-oracle-expands-its-distributed-cloud-capabilities-to-help-organizations-innovate-with-ai-2024-09-11/ (11/Sep/2024) +- Accelerating AI at the Edge with Oracle Roving Edge Infrastructure | Intel: https://www.youtube.com/watch?v=sd0woibrAVo (11/Sep/2024) +- Intel on AI: Accelerating AI at the Edge with Oracle Roving Edge Infrastructure https://open.spotify.com/episode/3Vga6QbXQC5wcDTRWTtPZL?si=BImzgCq8RUuuaCfea9vQcA (11/Sep/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/empowering-edge-oracle-roving-edge-device-2nd-gen (10/Sep/2024) +- https://www.prnewswire.com/news-releases/oracle-and-rafael-to-provide-cloud-based-ai-solutions-for-defense-missions-302242995.html (10/Sep/2024) +- https://www.prnewswire.com/news-releases/oracle-and-anduril-industries-partner-to-deliver-ai-powered-defense-solutions-from-the-datacenter-to-the-tactical-edge-302242992.html (10/Sep/2024) +- https://blogs.oracle.com/developers/post/empower-your-analysts-with-natural-language-querying-at-the-edge (13/Jun/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/reimagining-edge-data-analysis-llms-opensource (20/May/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/practical-inferencing-open-source-models (30/Apr/2024) +- https://blogs.oracle.com/dataintegration/post/oci-goldengate-stream-analytics-is-now-generally-available (12/Feb/2024) +- https://blogs.oracle.com/cloud-infrastructure/post/zerotrust-interoperability-defence-alliances (8/Dec/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/oracle-edge-tactical-solutions-nato-exercise (18/Oct/2023) +- https://blogs.oracle.com/dataintegration/post/oci-goldengate-stream-analytics-now-available-in-limited-availability (6/Oct/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/govt-grade-kubernetes-rancher-govt-solutions-oci (14/Sep/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/oracle-roving-edge-infrastructure-available-uk (6/Sep/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/oracle-roving-edge-infrastructure-expands-to-eu (6/Jun/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/druid-software-3gpp-compliant-enterprise-core (17/Apr/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/hands-on-oci-roving-edge (4/Apr/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/enabling-tactical-5g-for-military-operations (23/Feb/2023) +- https://blogs.oracle.com/cloud-infrastructure/post/government-offerings-oracle-cloud-marketplace (23/Dec/2022) +- https://blogs.oracle.com/cloud-infrastructure/post/fall-2022-compliance-attestations (2/Dec/2022) +- https://blogs.oracle.com/cloud-infrastructure/post/falkonry-oracle (29/Nov/2022) +- https://blogs.oracle.com/cloud-infrastructure/post/unlocking-5g-with-end-to-end-distributed-cloud (21/Jun/2022) +- https://blogs.oracle.com/cloud-infrastructure/post/oracle-cloud-marketplace-supports-compatible-images-for-the-oracle-roving-edge-infrastructure (1/Feb/2022) +- https://blogs.oracle.com/cloud-infrastructure/post/introducing-oracle-cloud-for-telcos-redefining-the-telco-cloud-market (11/Jan/2022) +- https://blogs.oracle.com/government-education/post/oracle-successfully-demonstrates-tactical-cloud-computing-in-key-dod-exercise (3/Dec/2021) +- https://blogs.oracle.com/government-education/post/oracle-successfully-demonstrates-tactical-cloud-computing-in-key-dod-exercise-army-cyber-quest (16/Nov/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/high-powered-compute-in-the-most-remote-places-in-the-world (15/Nov/2021) +- https://blogs.oracle.com/government-education/post/oracle-cracks-the-code-to-deliver-devsecops-and-tactical-edge-devices-to-military-intelligence (29/Oct/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/big-data-analytics-edge-computing-and-more-new-oracle-cloud-marketplace-solutions-for-government (12/Oct/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/oracles-distinct-approach-on-hybrid-and-multicloud (30/Apr/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/oracle-cloud-delivers-innovations-at-nvidias-gtc21 (12/Apr/2021) +- https://blogs.oracle.com/dataintegration/post/goldengate-stream-analytics-powers-oracle-reds-for-prognostic-ml-solutions (14/Feb/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/resetting-the-boundaries-of-hybrid-cloud-flexibility-and-control (9/Feb/2021) +- https://blogs.oracle.com/cloud-infrastructure/post/announcing-availability-of-the-oracle-roving-edge-infrastructure-for-us-government-customers (9/Feb/2021) + +## License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/cloud-infrastructure/storage/README.md b/cloud-infrastructure/storage/README.md index c07b99c6b..cdf51b8bf 100644 --- a/cloud-infrastructure/storage/README.md +++ b/cloud-infrastructure/storage/README.md @@ -2,7 +2,7 @@ Oracle Cloud Infrastructure provides customers with high-performance computing and low-cost cloud storage options. Through on-demand local, object, file, block, and archive storage, Oracle Cloud addresses key storage workload requirements and use cases. Customers can use the storage gateway and data transfer service to safely and securely move their data to the cloud. -Reviewed: 18.11.2024 +Reviewed: 03.12.2024 # Useful Links diff --git a/cloud-infrastructure/storage/block-storage/README.md b/cloud-infrastructure/storage/block-storage/README.md index 193b81728..15b2baa3b 100644 --- a/cloud-infrastructure/storage/block-storage/README.md +++ b/cloud-infrastructure/storage/block-storage/README.md @@ -4,7 +4,7 @@ High-performance block storage at any scale. We've designed our storage platform as an ideal complement to Oracle compute and networking services to support the highest performance requirements. OCI Block Volume uses the latest NVMe SSDs and provides nonblocking network connectivity to every host. Oracle delivers a consistent, low-latency performance of up to 225 IOPS/GB to a maximum of 300,000 IOPS and 2,680 MB/sec of throughput per volume. -Reviewed: 18.11.2024 +Reviewed: 03.12.2024 # Table of Contents diff --git a/cloud-infrastructure/storage/block-storage/showcase-2/README.md b/cloud-infrastructure/storage/block-storage/showcase-2/README.md index f9830d5d3..b9b3b21a2 100644 --- a/cloud-infrastructure/storage/block-storage/showcase-2/README.md +++ b/cloud-infrastructure/storage/block-storage/showcase-2/README.md @@ -9,7 +9,7 @@ Assets that contain a great opportunity to learn about Cloud Resilience or "Keep - A list of Oracle Architectural Best Practices -Reviewed: 18.11.2024 +Reviewed: 03.12.2024 # When to use this asset? diff --git a/cloud-infrastructure/storage/block-storage/showcase-3/README.md b/cloud-infrastructure/storage/block-storage/showcase-3/README.md index 31fe3d3ca..80b8a93c1 100644 --- a/cloud-infrastructure/storage/block-storage/showcase-3/README.md +++ b/cloud-infrastructure/storage/block-storage/showcase-3/README.md @@ -5,7 +5,7 @@ Assets that contain a bash script great opportunity to learn about Customer managed (boot and block) volume backups and how to copy them to a different region. -Reviewed: 18.11.2024 +Reviewed: 03.12.2024 # When to use this asset? diff --git a/cloud-infrastructure/storage/block-storage/showcase-4/README.md b/cloud-infrastructure/storage/block-storage/showcase-4/README.md index 44a35cc82..30236a78a 100644 --- a/cloud-infrastructure/storage/block-storage/showcase-4/README.md +++ b/cloud-infrastructure/storage/block-storage/showcase-4/README.md @@ -5,7 +5,7 @@ Assets that contain a process incl. a script how to backup and restore an individual boot or block volume to a customer-managed bucket. -Reviewed: 18.11.2024 +Reviewed: 03.12.2024 # When to use this asset? diff --git a/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-19c/README.md b/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-19c/README.md index f4fc35b5e..beca972f6 100644 --- a/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-19c/README.md +++ b/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-19c/README.md @@ -1,6 +1,7 @@ # MongoDB compatibility advisor for Autonomous MongoDB API This tool helps you to identify which MongoDB queries are supported when using the Oracle Autonomous MongoDB API. +**IMPORANT NOTE:** This tool is not an official product and is provided as a community-driven resource to assist in expediting the process of checking compatibility. While it aims to be helpful, it is not guaranteed to cover all scenarios or provide complete accuracy. Users are strongly encouraged to consult the official documentation for definitive guidance and support. Reviewed: 24.10.2024 diff --git a/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-23ai/README.md b/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-23ai/README.md index 873df4381..6a6557e21 100644 --- a/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-23ai/README.md +++ b/data-platform/autonomous-database/autonomous-json/mongodb-compatibility-advisor-23ai/README.md @@ -1,7 +1,7 @@ # MongoDB compatibility advisor for Autonomous MongoDB API This tool helps you to identify which MongoDB queries are supported when using the Oracle Autonomous MongoDB API. - +**IMPORANT NOTE:** This tool is not an official product and is provided as a community-driven resource to assist in expediting the process of checking compatibility. While it aims to be helpful, it is not guaranteed to cover all scenarios or provide complete accuracy. Users are strongly encouraged to consult the official documentation for definitive guidance and support. Reviewed: 04.11.2024 # When to use this asset? diff --git a/data-platform/autonomous-database/autonomous-serverless/README.md b/data-platform/autonomous-database/autonomous-serverless/README.md index 72e866459..4403b6b21 100644 --- a/data-platform/autonomous-database/autonomous-serverless/README.md +++ b/data-platform/autonomous-database/autonomous-serverless/README.md @@ -48,6 +48,9 @@ Reviewed: 23.10.2024 - [Capture-Replay Workloads between non-Autonomous and Autonomous Databases](https://www.youtube.com/watch?v=cWZ9MPBZemc) - How to Capture and Replay Workloads between non-Autonomous and Autonomous Databases. +- [Capture-Replay Workloads between Autonomous Databases](https://www.youtube.com/watch?v=JXpQe7zUFs8) + - How to Capture and Replay Workloads between Autonomous Databases. + - [Enable SQL Trace on Autonomous Database](https://blogs.oracle.com/datawarehousing/post/enable-sql-trace-on-autonomous-database) - Step-by-step blog article explaining how to collect a sql trace from Autonomous Database. diff --git a/data-platform/core-converged-db/ai-vector-search/README.md b/data-platform/core-converged-db/ai-vector-search/README.md index 51abda565..a89e23cbf 100644 --- a/data-platform/core-converged-db/ai-vector-search/README.md +++ b/data-platform/core-converged-db/ai-vector-search/README.md @@ -4,16 +4,19 @@ Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads a If you are interested in Database 23ai , [please see our page here](https://github.com/oracle-devrel/technology-engineering/tree/main/data-platform/core-converged-db/database-23ai). -Reviewed Date: 20.08.2024 +Reviewed Date: 21.01.2025 # Useful Links ## Documentation +- [What is a Vector Database](https://www.oracle.com/database/vector-database/) - [Oracle.com](https://www.oracle.com/database/ai-vector-search/) +- [Oracle AI Vector Search Technical Architecture (interactive diagram)](https://docs.oracle.com/en/database/oracle/oracle-database/23/vsiad/aivs_genarch.html) - [Oracle AI Vector Search User's Guide](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/overview-ai-vector-search.html) - [PL/SQL Packages and Types Reference: DBMS_VECTOR](https://docs.oracle.com/en/database/oracle/oracle-database/23/arpls/dbms_vector1.html#GUID-F9FCB225-821A-4CCA-92B5-58B9927234FA) - [PL/SQL Packages and Types Reference: DBMS_VECTOR_CHAIN](https://docs.oracle.com/en/database/oracle/oracle-database/23/arpls/dbms_vector_chain1.html#GUID-D80DDBEF-F1A9-4267-9D3C-A54D237D95C1) +- [DBMS_HYBRID_VECTOR.SEARCH PL/SQL function](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/search.html#GUID-A386BDB0-35D0-41E1-8F41-49AEBEC13BFC) - [Oracle AI Vector Search FAQ](https://www.oracle.com/database/ai-vector-search/faq/) ## Blogs & Videos @@ -36,10 +39,11 @@ Reviewed Date: 20.08.2024 ## Blogs - [Getting started with vectors in 23ai](https://blogs.oracle.com/coretec/post/getting-started-with-vectors-in-23ai) +- [Hybrid Vector Index - a combination of AI Vector Search with Text Search](https://blogs.oracle.com/coretec/post/hybrid-vector-index-the-combination-of-full-text-and-semantic-vector-search) ## Videos -- [Ai Vector Search](https://youtu.be/_tOGB9l8cvg) +- [AI Vector Search](https://youtu.be/_tOGB9l8cvg) # License diff --git a/data-platform/core-converged-db/database-23ai/README.md b/data-platform/core-converged-db/database-23ai/README.md index 917b001bc..f6c851dff 100644 --- a/data-platform/core-converged-db/database-23ai/README.md +++ b/data-platform/core-converged-db/database-23ai/README.md @@ -6,7 +6,7 @@ Oracle Database 23ai is available cloud-first, in OCI on Oracle Exadata Database If you are interested in AI Vector Search, [please see our page here](https://github.com/oracle-devrel/technology-engineering/tree/main/data-platform/core-converged-db/ai-vector-search). -Reviewed: 15.11.2024 +Reviewed: 21.1.2025 # Useful Links @@ -75,7 +75,7 @@ Reviewed: 15.11.2024 ## Blogs - [Oracle Database Release 23ai is available now!](https://blogs.oracle.com/coretec/post/oracle-database-release-23ai) -- [Oracle Database 23c Free Developer Release - 10 features you should know](https://blogs.oracle.com/coretec/post/oracle-database-23c-free-developer-sql) +- [Oracle Database 23 Free Developer Release - 10 features you should know](https://blogs.oracle.com/coretec/post/oracle-database-23c-free-developer-sql) - [Sample data for Oracle Database](https://blogs.oracle.com/coretec/post/sample-data-for-oracle-database) - [Less coding using Domains](https://blogs.oracle.com/coretec/post/less-coding-with-sql-domains-in-23c) - [New value constructor in 23ai](https://blogs.oracle.com/coretec/post/new-value-constructor-in-23c) @@ -88,6 +88,7 @@ Reviewed: 15.11.2024 - [SQL Transpiler in 23ai - automatic PL/SQL conversion into SQL](https://blogs.oracle.com/coretec/post/sql-transpiler-in-23c) - [New full-text search in 23ai: Ubiquitous Database Search](https://blogs.oracle.com/coretec/post/sql-transpiler-in-23c) - [Easy Database Search with APEX in 23ai](https://blogs.oracle.com/coretec/post/easy-database-search-with-apex-in-23c) +- [Text Index with Automatic Maintenance](https://blogs.oracle.com/coretec/post/text-index-with-automatic-maintenance) - [SQL statement tracking in 23ai with SQL History](https://blogs.oracle.com/coretec/post/easy-sql-statement-tracking-in23c) - [Developing a Java application using Oracle Graph Server](https://blogs.oracle.com/coretec/post/developing-a-java-application-working-on-top-of-oracle-property-graph-server) - [ETags with Oracle Database tables](https://blogs.oracle.com/coretec/post/valuebwith-table-data) @@ -98,7 +99,9 @@ Reviewed: 15.11.2024 - [Bigfile Tablespace Shrink](https://blogs.oracle.com/coretec/post/bigfile-tablespace-shrink) - [Real-Time SQL Monitoring: a MUST for SQL Tuning](https://blogs.oracle.com/coretec/post/oracle-database-real-time-sql-monitoring-one-of-the-most-important-tools) - [New Features and Release Updates](https://blogs.oracle.com/coretec/post/where-to-find-features-in-release-updates) -- [23ai blogs on blogs.oracle.com/coretec](https://blogs.oracle.com/coretec/category/cased-concurrency-control-rt-23ai) +- [Diagnose SQL performance with DBMS_SQLDIAG](https://blogs.oracle.com/coretec/post/diagnose-sql-performance-with-dbmssqldiag) +- [Hybrid Vector Index - a combination of AI Vector Search with Text Search](https://blogs.oracle.com/coretec/post/hybrid-vector-index-the-combination-of-full-text-and-semantic-vector-search) +- [all 23ai blogs on blogs.oracle.com/coretec](https://blogs.oracle.com/coretec/category/crt-23ai) ## Videos diff --git a/data-platform/core-converged-db/db-graph/README.md b/data-platform/core-converged-db/db-graph/README.md new file mode 100644 index 000000000..8729351fa --- /dev/null +++ b/data-platform/core-converged-db/db-graph/README.md @@ -0,0 +1,57 @@ +# Oracle Database Graph + +Graphs enable you to find connections and explore relationships in your data. Oracle Graph is an AI-ready, integrated feature of Oracle's converged database that eliminates the need for a separate graph database and data movement. Analysts and developers can address various use cases, including financial fraud detection and manufacturing traceability, while gaining enterprise-grade security, ease of data ingestion, and strong support for operational workloads.Oracle Graph supports both RDF (resource description framework) and property graphs on operational data. +Developers can now easily build graph applications with SQL using existing SQL development tools and frameworks through SQL property graphs. The combination of AI Vector Search and RDF knowledge graph capabilities in Oracle Database makes it simple to extend retrieval-augmented generation (RAG) workflows with graph analysis, enabling improved search response accuracy—also known as GraphRAG. + +Reviewed: 22.01.2025 + +# Useful Links + +## Documentation + +- [What is a Graph?](https://www.oracle.com/autonomous-database/what-is-graph-database/#graph-database-defined) +- [Integrated Graph Database on oracle.com](https://www.oracle.com/database/graph/) +- [Graph Developer's Guide for Property Graph](https://docs.oracle.com/pls/topic/lookup?ctx=property-graph-latest&id=SPGDG) +- [Graph Developer's Guide for RDF Graph](https://docs.oracle.com/en/database/oracle/oracle-database/23/rdfrm/index.html) +- [Graph Java API Reference for Property Graph (Javadoc)](https://docs.oracle.com/pls/topic/lookup?ctx=property-graph-latest&id=SPGJV) +- [Graph JavaScript API Reference for Property Graph Visualization](https://docs.oracle.com/pls/topic/lookup?ctx=property-graph-latest&id=PGJSD) +- [Graph Python API Reference for Property Graph](https://docs.oracle.com/pls/topic/lookup?ctx=property-graph-latest&id=python_doc) +- [data sets for property graph (Karin Patenge)](https://github.com/karinpatenge/property-graph) + +## Blogs and Technical Briefs + +- [17 Use Cases for Graph Databases and Graph Analytics](https://www.oracle.com/a/ocom/docs/graph-database-use-cases-ebook.pdf) +- [See your data through new eyes: Everything can be a graph](https://blogs.oracle.com/coretec/post/see-your-data-through-new-eyes-everything-can-be-a-graph) +- [Provisioning analytic environments easily - like Oracle Graph)](https://blogs.oracle.com/coretec/post/provisioning-analytic-environments-easily-like-oracle-graph) +- [Get Started with Oracle Graph using SQL Developer](https://medium.com/oracledevs/get-started-with-oracle-graph-using-sql-developer-e5ccbd61c9fb) +- [Visualizing Graphs in Oracle Graph](https://medium.com/oracledevs/visualizing-graphs-in-oracle-graph-6159c8ce2491) +- [Run Graph Algorithms in Jupyter Notebook](https://medium.com/oracledevs/run-graph-algorithms-in-jupyter-notebook-4a612c870652) + + +## LiveLabs + +- [all Graph LiveLabs](https://apexapps.oracle.com/pls/apex/f?p=133:100:117224653754233::::SEARCH:graph) + +## Videos + +- [Oracle Database 23c – Property Graphs](https://www.youtube.com/watch?v=4uknPkJkUIk) +- [When Property Graphs Meet SQL](https://www.youtube.com/watch?v=S1PJLmDiRMY) + +# Team Publications + +## Blogs + +- [GraphQL, SQL/PGQ or SPARQL?](https://blogs.oracle.com/coretec/post/graphql-pgqql-or-sparql) +- [Developing a Java application using Oracle Graph Server](https://blogs.oracle.com/coretec/post/developing-a-java-application-working-on-top-of-oracle-property-graph-server) + +## Videos + +- [SQL/PGQ and SQL Property Graph](https://youtu.be/cezMbCP6hvc) + +# License + +Copyright (c) 2025 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/property-graph/oracle-graph-presentations/PROPERTY-GRAPH-23.pdf b/data-platform/core-converged-db/db-graph/db-graph-presentations/PROPERTY-GRAPH-23.pdf similarity index 100% rename from data-platform/core-converged-db/property-graph/oracle-graph-presentations/PROPERTY-GRAPH-23.pdf rename to data-platform/core-converged-db/db-graph/db-graph-presentations/PROPERTY-GRAPH-23.pdf diff --git a/data-platform/core-converged-db/db-graph/db-graph-presentations/README.md b/data-platform/core-converged-db/db-graph/db-graph-presentations/README.md new file mode 100644 index 000000000..d495b484e --- /dev/null +++ b/data-platform/core-converged-db/db-graph/db-graph-presentations/README.md @@ -0,0 +1,20 @@ +# Oracle Graph Presentations +This presentations provide information about general Oracle Graph functions. + +Reviewed: 22.01.2025 + +# When to use this asset? + +To present or to learn about Oracle Graph. + +# How to use this asset? + +Use PDF as a presentation. + +# License + +Copyright (c) 2025 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_in_db.pdf b/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_in_db.pdf new file mode 100644 index 000000000..2ded6d279 Binary files /dev/null and b/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_in_db.pdf differ diff --git a/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_introduction.pdf b/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_introduction.pdf new file mode 100644 index 000000000..2dd17b62e Binary files /dev/null and b/data-platform/core-converged-db/db-graph/db-graph-presentations/graph_rag_introduction.pdf differ diff --git a/data-platform/core-converged-db/db-in-memory/README.md b/data-platform/core-converged-db/db-in-memory/README.md index edcbeefc8..8bd04ffbc 100644 --- a/data-platform/core-converged-db/db-in-memory/README.md +++ b/data-platform/core-converged-db/db-in-memory/README.md @@ -2,7 +2,7 @@ Database In-Memory features a highly optimized In-Memory Column Store (IM column store) maintained alongside the existing buffer cache. The primary purpose of the IM column store is to accelerate columnoriented data accesses made by analytic operations. It is similar in spirit to having a conventional index (for analytics) on every column in a table. However, it is much more lightweight than a conventional index, requiring no logging, or any writes to the database. Just as the performance benefit to an application from conventional indexes depends on the amount of time the application spends accessing data in the tables that are indexed, the benefit from the IM column store also depends on the amount of time the application spends on data access for analytic operations. It is therefore important to understand the basic characteristics of your application to determine the potential benefits from Database In-Memory. -Review Date: 03.06.2024 +Review Date: 20.11.2024 ## Useful Links @@ -10,7 +10,9 @@ Review Date: 03.06.2024 - [Database In-Memory](https://www.oracle.com/database/in-memory/) - [Database In-Memory Guide](https://docs.oracle.com/en/database/oracle/oracle-database/23/inmem/index.html#Oracle%C2%AE-Database) -- +- [Oracle Database In-Memory Option (DBIM) Basics and Interaction with Data Warehousing Features (Doc ID 1903683.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=286800017302956&id=1903683.1&_afrWindowMode=0&_adf.ctrl-state=k8hteq5ab_4) +- [Oracle Database In-Memory Data Sheet](https://www.oracle.com/a/otn/docs/database/database-in-memory-datasheet.pdf) + ### Team Publications - [When and How to use Oracle Database In-Memory Advisor](https://blogs.oracle.com/coretec/post/how-to-use-oracle-database-in-memory-advisor) @@ -19,8 +21,10 @@ Review Date: 03.06.2024 ### Blogs and technical briefs +- [New Database In-Memory Features in Oracle Database 23ai](https://blogs.oracle.com/in-memory/post/dbim-new-features-23ai) - [DBIM Resources](https://blogs.oracle.com/in-memory/post/dbim-resources) -- [Oracle Database In-Memory](https://blogs.oracle.com/in-memory/) +- [Oracle Database In-Memory postings](https://blogs.oracle.com/in-memory/) +- [New In-Memory Eligibility Test](https://blogs.oracle.com/in-memory/post/inmemory-eligibility-test#:~:text=The%20purpose%20of%20the%20eligibility%20test%20is%20to,to%20get%20more%20information%20about%20your%20analytic%20workload.) - [When to Use Oracle Database In-Memory](https://www.oracle.com/docs/tech/when-to-use-oracle-database-in-memory.pdf) - [Oracle Database In-Memory Implementation Guidelines](https://www.oracle.com/technetwork/database/in-memory/learnmore/twp-oracle-dbim-implementation-3863029.pdf) diff --git a/data-platform/core-converged-db/db-json/README.md b/data-platform/core-converged-db/db-json/README.md index 68e0ff568..bbe47cd65 100644 --- a/data-platform/core-converged-db/db-json/README.md +++ b/data-platform/core-converged-db/db-json/README.md @@ -3,7 +3,7 @@ The Oracle Database provides various perspectives for viewing and accessing JSON data. One can view it from the SQL database point of view or from the document or collection view that comes with tools and APIs like SODA (short for Simple Oracle Document Access). With SODA you can create and store collections of documents in the Oracle Database, and retrieve, without needing to know SQL or how the documents are stored in the database. Documents can be fetched from the database by key lookup or by using query-by-example (QBE) pattern-matching. Regardless of the chosen perspective, the benefits of using Oracle Database as a Converged Database are always included. In release 23, JSON Relational Duality, one of the highlights for JSON developers delivers a solution that provides the benefits of both relational tables and JSON documents, without the tradeoffs of either model. Using Duality Views, data is still stored in relational tables in a highly efficient normalized format but is accessed by apps in the form of JSON documents. -Reviewed Date: 30.10.2024 +Reviewed Date: 07.01.2025 # Useful Links @@ -19,6 +19,7 @@ Reviewed Date: 30.10.2024 - [JSON in Oracle Database Insider](https://blogs.oracle.com/database/category/db-json) - [Simplify APEX App REST APIs with JSON Duality Views](https://diveintoapex.com/2024/03/05/simplify-apex-app-rest-apis-with-json-duality-views/) +- [Search indexes for JSON](https://blogs.oracle.com/database/post/search-indexes-for-json) ## LiveLabs and Live SQL Workshops diff --git a/data-platform/core-converged-db/db-performance/presentations/OracleDatabaseMemory&Caching.pdf b/data-platform/core-converged-db/db-performance/presentations/OracleDatabaseMemory&Caching.pdf new file mode 100644 index 000000000..d3a56d9fd Binary files /dev/null and b/data-platform/core-converged-db/db-performance/presentations/OracleDatabaseMemory&Caching.pdf differ diff --git a/data-platform/core-converged-db/db-performance/presentations/README.md b/data-platform/core-converged-db/db-performance/presentations/README.md new file mode 100644 index 000000000..b9462777e --- /dev/null +++ b/data-platform/core-converged-db/db-performance/presentations/README.md @@ -0,0 +1,22 @@ +# Presentations +Here you will find presentations about database performance topics e.g. an overview presentation about memory and caching technologies in Oracle Database. + +For more information on additional advanced technologies, use [True Cache](https://github.com/oracle-devrel/technology-engineering/tree/main/data-platform/core-converged-db/true-cache) or [Oracle Database In-Memory](https://github.com/oracle-devrel/technology-engineering/tree/main/data-platform/core-converged-db/db-in-memory). + +Reviewed: 20.11.2024 + +# When to use this asset? + +To present or to learn about Oracle Database performance technologies. + +# How to use this asset? + +Use PDF as a presentation. + +# License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/property-graph/OracleSupportforPropertyGraph_overview.pdf b/data-platform/core-converged-db/property-graph/OracleSupportforPropertyGraph_overview.pdf deleted file mode 100644 index c003b6daf..000000000 Binary files a/data-platform/core-converged-db/property-graph/OracleSupportforPropertyGraph_overview.pdf and /dev/null differ diff --git a/data-platform/core-converged-db/property-graph/README.md b/data-platform/core-converged-db/property-graph/README.md deleted file mode 100644 index 14cb02b59..000000000 --- a/data-platform/core-converged-db/property-graph/README.md +++ /dev/null @@ -1,64 +0,0 @@ -# Property Graph - -The Property Graph feature of Oracle Database offers powerful graph support for Oracle database, to explore and discover complex relationships in data sets such as customer data, social networks, and financial transactions, for applications in product recommendation, fraud detection, risk analysis, targeted marketing, and more. This high-performance Graph feature delivers advanced graph query language support, developer APIs, and analytics capabilities with nearly 60 pre-built algorithms. The Property Graph features including the PGQL (Property Graph Query Language) support is available from Oracle Database 12.2 onward. Starting from Oracle Database 23, you can also create and query property graphs using SQL, as defined in the SQL:2023 standard. - -Oracle Graph Server and Client is a software package that works with the Property Graph feature in the Oracle Database. It includes graph tools, client libraries, and Oracle Graph Server for running graph analytics algorithms. Oracle Graph Server and Client releases four times a year and its release version number is not related to the Oracle Database Release version number. - -Review Date: 26.09.2024 - -# Useful Links - -## Documentation - -- [What is a Graph?](https://www.oracle.com/autonomous-database/what-is-graph-database/#graph-database-defined) -- [Integrated Graph Database on oracle.com](https://www.oracle.com/database/graph/) -- [Graph Developer's Guide for Property Graph](https://docs.oracle.com/en/database/oracle/property-graph/24.2/spgdg/sql-property-graphs.html#GUID-B82E61FF-3601-4D9C-99F9-6A5D4FF961E6) -- [SQL Language Reference](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/create-property-graph.html#SQLRF-GUID-37364ADB-E89C-4D92-A431-F2544FEDB218) -- [Oracle Graph Server and Client Releases](https://docs.oracle.com/en/database/oracle/property-graph/) -- [Oracle Graph JavaScript API Reference for Property Graph Visualization](https://docs.oracle.com/en/database/oracle/property-graph/24.2/pgjsd/index.html) -- [Graph Java API Reference for Property Graph (Javadoc)](https://docs.oracle.com/en/database/oracle/property-graph/24.2/spgjv/index.html) -- [Graph Developer's Guide for RDF Graph](https://docs.oracle.com/en/database/oracle/oracle-database/23/rdfrm/index.html) -- [Graph Python API Reference for Property Graph](https://docs.oracle.com/en/database/oracle/property-graph/24.2/books.html) -- [Visualizing SQL Graph Queries Using the APEX Graph Visualization Plug-in](https://docs.oracle.com/en/database/oracle/property-graph/23.4/spgdg/visualizing-sql-graph-queries-using-apex-graph-visualization-plug.html) - -## Blogs and Technical Briefs - -- [17 Use Cases for Graph Databases and Graph Analytics](https://www.oracle.com/a/ocom/docs/graph-database-use-cases-ebook.pdf) -- [Operational Property Graphs with SQL in Oracle Database 23c](https://www.oracle.com/a/ocom/docs/database/operational_property_graph_with_23c.pdf) -- [Second Quarterly Update on Oracle Graph (2024)](https://blogs.oracle.com/database/post/second-quarterly-update-on-oracle-graph-2024) -- [See your data through new eyes: Everything can be a graph](https://blogs.oracle.com/coretec/post/see-your-data-through-new-eyes-everything-can-be-a-graph) -- [Oracle Graph Blog](https://blogs.oracle.com/database/category/db-graph) -- [Oracle Graph Learning Path](https://blogs.oracle.com/database/post/oracle-graph-learning-path) -- [Oracle Graph on medium](https://medium.com/tag/oracle-graph) - - -## LiveLabs - -- [Exploring Operational Property Graphs in 23c Free](https://apexapps.oracle.com/pls/apex/r/dbpm/livelabs/view-workshop?wid=3659) -- [Break down the Monolith - the Oracle Property Graph in Software re-engineering](https://apexapps.oracle.com/pls/apex/r/dbpm/livelabs/view-workshop?wid=3788&clear=RR,180&session=3234280003655) -- [Graph Studio: Find Circular Payment Chains with Graph Queries in Autonomous Database](https://apexapps.oracle.com/pls/apex/r/dbpm/livelabs/view-workshop?wid=770) -- [all Graph LiveLabs](https://apexapps.oracle.com/pls/apex/f?p=133:100:117224653754233::::SEARCH:graph) - -## Videos - -- [Oracle Database 23c – Property Graphs](https://www.youtube.com/watch?v=4uknPkJkUIk) -- [When Property Graphs Meet SQL](https://www.youtube.com/watch?v=S1PJLmDiRMY) - -# Team Publications - -## Blogs - -- [GraphQL, SQL/PGQ or SPARQL?](https://blogs.oracle.com/coretec/post/graphql-pgqql-or-sparql) -- [Developing a Java application using Oracle Graph Server](https://blogs.oracle.com/coretec/post/developing-a-java-application-working-on-top-of-oracle-property-graph-server) - -## Videos - -- [SQL/PGQ and SQL Property Graph](https://youtu.be/cezMbCP6hvc) - -# License - -Copyright (c) 2024 Oracle and/or its affiliates. - -Licensed under the Universal Permissive License (UPL), Version 1.0. - -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/README.md b/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/README.md deleted file mode 100644 index 48ab49f3d..000000000 --- a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/README.md +++ /dev/null @@ -1,29 +0,0 @@ -# Property Graph Demo - -This code demonstrates: - 1. The integration of Property Graph with SQL in Oracle Database 23 - 2. Java API for Property Graph Server - -It uses demonstration tables described in the following LiveLabs Workshop [Analyze, Query, and Visualize Property Graphs with Oracle Database](https://apexapps.oracle.com/pls/apex/r/dbpm/livelabs/run-workshop?p210_wid=686&p210_wec=&session=108772142407826) - -Reviewed: 30.10.2024 - -## When to use this asset? - -To demo Property Graph functionalities. - -## How to use this asset? - -Requirements: - 1. Oracle Database 23 - 2. Database user account with the privileges described in the mentioned LiveLabs Workshop - 3. Tables used in this workshop created in the mentioned schema - 4. Oracle Property Graph Server installation using a user account - -# License - -Copyright (c) 2024 Oracle and/or its affiliates. - -Licensed under the Universal Permissive License (UPL), Version 1.0. - -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/pom.xml b/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/pom.xml deleted file mode 100644 index e50401b8e..000000000 --- a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/pom.xml +++ /dev/null @@ -1,49 +0,0 @@ - - - 4.0.0 - - pg.oracle - PGJava - 1.0-SNAPSHOT - - - 18 - 18 - UTF-8 - - - - spoofax - https://artifacts.metaborg.org/content/repositories/releases - - - - - com.oracle.database.jdbc - ojdbc11 - 23.3.0.23.09 - - - com.oracle.database.graph - opg-client - 24.1.0 - - - - com.oracle.database.security - oraclepki - 23.3.0.23.09 - - - com.oracle.database.graph - pgx-api - 24.1.0 - - - com.oracle.database.graph - graph-query-ir - 2.0.8.2 - - diff --git a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/src/main/java/pg/oracle/Main.java b/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/src/main/java/pg/oracle/Main.java deleted file mode 100644 index 1382b9cc0..000000000 --- a/data-platform/core-converged-db/property-graph/java-api-for-property-graphs/files/src/main/java/pg/oracle/Main.java +++ /dev/null @@ -1,102 +0,0 @@ -package pg.oracle; - -/* property graph demo - this code demonstrates: - 1. integration of Property Graph with SQL in Oracle Database 23c - 2. Java API for Property Graph Server - It uses demonstration tables described in the following LiveLabs Workshop: - "Analyze, Query, and Visualize Property Graphs with Oracle Database" - https://apexapps.oracle.com/pls/apex/r/dbpm/livelabs/run-workshop?p210_wid=686&p210_wec=&session=108772142407826 - Requirements: - 1. Oracle Database 23c - 2. Database user account ( ) with the privileges described in the mentioned LiveLabs Workshop - 3. Tables used in this workshop created in the mentioned schema - 4. Oracle Property Graph Server installation using user account - */ - - -import java.sql.*; -import oracle.jdbc.pool.OracleDataSource; -import oracle.pg.rdbms.GraphServer; -import oracle.pgx.api.*; - -public class Main { - private static String dbHost = ""; - private static String pgxHost = ""; - private static int dbPort = ; - private static int pgxPort = ; - private static String dbService = ""; - private static String username = ""; - private static String password = ""; - private static String pgName = "CUSTOMER_GRAPH"; - public static void SQLPGDemo() { - try { - OracleDataSource ds = new OracleDataSource(); - ds.setDriverType("thin"); - ds.setServerName(dbHost); - ds.setServiceName(dbService); - ds.setPortNumber(dbPort); - ds.setUser(username); - ds.setPassword(password); - Connection con = ds.getConnection(); - System.out.println("Connected to the database"); - System.out.println("(Re)creating property graph "+pgName); - Statement stmt = con.createStatement(); - stmt.execute("CREATE OR REPLACE PROPERTY GRAPH "+pgName+" VERTEX TABLES (\n" + - " customer\n" + - " , account\n" + - " , merchant\n" + - " )\n" + - " EDGE TABLES (\n" + - " account as account_edge\n" + - " SOURCE KEY(id) REFERENCES account (id)\n" + - " DESTINATION KEY(customer_id) REFERENCES customer (id)\n" + - " LABEL owned_by PROPERTIES (id)\n" + - " , parent_of as parent_of_edge \n" + - " SOURCE KEY(customer_id_parent) REFERENCES customer (id)\n" + - " DESTINATION KEY(customer_id_child) REFERENCES customer (id)\n" + - " , purchased as puchased_edge \n" + - " SOURCE KEY(account_id) REFERENCES account (id)\n" + - " DESTINATION KEY(merchant_id) REFERENCES merchant (id)\n" + - " , transfer as transfer_edge \n" + - " SOURCE KEY(account_id_from) REFERENCES account (id)\n" + - " DESTINATION KEY(account_id_to) REFERENCES account (id)\n" + - " ) "); - System.out.println("Graph (re)created succesfully."); - ResultSet rset = stmt.executeQuery("SELECT account_no\n" + - "FROM GRAPH_TABLE ( CUSTOMER_GRAPH MATCH (v1)-[transfer_edge]->{1,2}(v1)\n" + - "columns (v1.account_no as account_no))"); - while (rset.next()) { - System.out.println(rset.getString(1)); - } - con.close(); - } - catch (Exception e) {e.printStackTrace();} - } - - public static void PGXDemo() { - try { - ServerInstance si = GraphServer.getInstance("https://"+pgxHost+":"+pgxPort,username,password.toCharArray()); - PgxSession ses = si.createSession("my-session"); - System.out.println("Connected to graph server"); - PgxGraph graph = ses.readGraphByName(username.toUpperCase(), pgName, GraphSource.PG_SQL); - System.out.println("Graph loaded into Property Graph Server"); - PgqlResultSet rset = graph.queryPgql("SELECT a1.account_no AS a1_account\n" + - " , t1.transfer_date AS t1_date\n" + - " , t1.amount AS t1_amount\n" + - " , a2.account_no AS a2_account\n" + - " , t2.transfer_date AS t2_date\n" + - " , t2.amount AS t2_amount\n" + - "FROM MATCH (a1)-[t1:transfer_edge]->(a2)-[t2:transfer_edge]->(a1)\n" + - "WHERE t1.transfer_date < t2.transfer_date"); - while (rset.next()) { - System.out.println(rset.getString(1)); - } - } - catch (Exception e) {e.printStackTrace();} - } - public static void main(String[] args) { - SQLPGDemo(); - PGXDemo(); - } -} diff --git a/data-platform/core-converged-db/property-graph/oracle-graph-presentations/README.md b/data-platform/core-converged-db/property-graph/oracle-graph-presentations/README.md deleted file mode 100644 index b2526efbd..000000000 --- a/data-platform/core-converged-db/property-graph/oracle-graph-presentations/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# Oracle Property Graph Presentations -This presentation provides information about general Oracle Property Graph functions and highlights Oracle Database 23 features. - -Reviewed: 30.04.2024 - -# When to use this asset? - -To present or to learn about Oracle Property Graph. - -# How to use this asset? - -Use PDF as a presentation. - -# License - -Copyright (c) 2024 Oracle and/or its affiliates. - -Licensed under the Universal Permissive License (UPL), Version 1.0. - -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/core-converged-db/spatial/README.md b/data-platform/core-converged-db/spatial/README.md index f6a9566de..6b09b84e1 100644 --- a/data-platform/core-converged-db/spatial/README.md +++ b/data-platform/core-converged-db/spatial/README.md @@ -7,7 +7,7 @@ With spatial features included for free with all editions and services of Oracle You can use the free tool Spatial Studio, to a visualize, explore, and analyze geospatial data stored in and managed by Oracle Spatial. Spatial Studio is a multiuser Java EE application that can be used as a standalone tool (Quick Start) or deployed to WebLogic Server. -Review Date: 26.09.2024 +Review Date: 21.01.2025 # Useful Links @@ -22,6 +22,7 @@ Review Date: 26.09.2024 - [Technical brief: Network Data Model in Oracle Database](https://www.oracle.com/a/tech/docs/ndm21c.pdf) - [Optimize Oracle Spatial Performance (Best Practices, Tips,and Tricks)](https://www.oracle.com/de/a/ocom/docs/database/spatial-performance-and-best-practices.pdf) - [Oracle Spatial Product Portal (Oracle.com)](https://www.oracle.com/database/spatial/) + - [Oracle Spatial partners: Data downloads](https://www.oracle.com/database/technologies/spatial-partners-data/) ## Blogs diff --git a/data-platform/data-development/apex/README.md b/data-platform/data-development/apex/README.md index 20f2548f3..3713be8b1 100644 --- a/data-platform/data-development/apex/README.md +++ b/data-platform/data-development/apex/README.md @@ -4,7 +4,7 @@ Using APEX, developers don't need to be experts in a vast array of technologies to deliver sophisticated solutions. Just focus on solving the problem and let APEX take care of the rest. -Reviewed: 14.02.2024 +Reviewed: 20.11.2024 # Table of Contents 1. [Team Publications](#team-publications) @@ -17,17 +17,13 @@ Reviewed: 14.02.2024 - [APEX with AI - a chat about the new landscape for Application Development](https://youtu.be/f1cBdLEGY00) - [Cloud Coaching - Unlock the Power of APEX](https://www.youtube.com/watch?v=6rG_tX8ScBc) - [APEX on Autonomous Rockin' Experience](https://medium.com/@paulbrad71/apex-on-autonomous-database-rockin-experience-e57c4c4dc4f3) - -## Tutorials/ How-To's - -- [APEX Integration with Azure AD](https://medium.com/@rana.saeed/can-your-external-users-authenticate-into-your-apex-apps-606e48f9417b) -- [APEX Integration with Real Application Security(RAS)](https://medium.com/@rana.saeed/integrating-real-application-security-ras-with-oracle-apex-33892967beff) -- [Uploading files to OCI Object Storage via APEX](https://medium.com/@devpiotrekk/uploading-files-to-oci-object-storage-via-apex-42ad396ec55d) -- [APEX Devops with Liquibase](https://medium.com/@devpiotrekk/start-your-apex-devops-adventure-with-liquibase-f8e45c3d1e6a) -- [APEX Service DevOps with DBMS_CLOUD](https://medium.com/oracledevs/apex-service-can-devops-too-dbms-cloud-on-autonomous-72be9842d2f88) -- [Integrating Data Science with APEX](https://blogs.oracle.com/ai-and-datascience/post/yolov5-models-in-apex-using-oracle-data-science) -- [Other APEX Blogs](https://blogs.oracle.com/apex/) -- [APEX YouTube channel](https://www.youtube.com/channel/UCEpIXFjcQIztReQNLymvYrQ/playlists?view=1&sort=da) +- [Embedding Oracle Analytics Cloud in Oracle APEX](https://medium.com/@cristina.varas98/embedding-oracle-analytics-cloud-in-oracle-application-express-apex-e973402cb9ff) +- [Enhancing Data Analysis with Select AI in Oracle APEX](https://medium.com/@cristina.varas98/enhancing-data-analysis-with-select-ai-in-oracle-apex-9d00b070a073) +- [How to Modify Your Interactive Grid Using JavaScript in Oracle APEX](https://medium.com/@cristina.varas98/how-to-modify-your-interactive-grid-using-javascript-in-oracle-apex-d3c9f31a06ab) +- [Unlock the Power of AI in Oracle APEX 24.1: Set Up OpenAI Credentials for APEX Assistant](https://medium.com/@cristina.varas98/leveraging-machine-learning-for-image-clustering-a-case-study-with-a-museums-digital-archive-a6b6e1242https://medium.com/@cristina.varas98/unlock-the-power-of-ai-in-oracle-apex-24-1-set-up-openai-credentials-for-apex-assistant-2e840d8506f5) +- [Configuring Email in Oracle APEX on an Autonomous Database (ADB)](https://medium.com/@cristina.varas98/configuring-email-in-oracle-apex-on-an-autonomous-database-adb-d46fa015be95) +- [Creating my own specialized ChatGPT for Oracle APEX Development using Fine-Tuned OpenAI model](https://medium.com/@cristina.varas98/creating-my-own-specialized-chatgpt-for-oracle-apex-development-using-fine-tuned-openai-models-78263062aaf4) +- [Build a ChatGPT-like Oracle APEX Application Using OpenAI’s Fine-tuned Model](https://medium.com/@cristina.varas98/build-a-chatgpt-like-oracle-apex-application-using-openais-fine-tuned-model-acb8bac4b7c8) # Useful Links @@ -36,21 +32,32 @@ Reviewed: 14.02.2024 - [APEX EVENTS](https://apex.oracle.com/pls/apex/r/apex_pm/apex-events/events) - [APEX Office Hours](https://apex.oracle.com/en/community/office-hours/) - [Official APEX Landing Page](https://apex.oracle.com/en/) -- [What's New in the latest Release APEX 23.2](https://docs.oracle.com/en/database/oracle/apex/23.2/htmrn/new-features.html) +- [What's New in the latest Release APEX 24.1](https://docs.oracle.com/en/database/oracle/apex/24.1/htmrn/new-features.html) - [APEX best practice architecture](https://docs.oracle.com/solutions/?q=apex&cType=reference-architectures&sort=date-desc&lang=en) -- [APEX Managing Application Performance](https://docs.oracle.com/en/database/oracle/apex/23.2/htmdb/managing-application-performance.html#GUID-1684F55B-7782-4B1F-96AB-0D41BCBDB1BA) -- [APEX Managing Application Security](https://docs.oracle.com/en/database/oracle/apex/23.2/htmdb/managing-application-security.html#GUID-417EAE67-7A36-4AE0-816B-D28716829F96) +- [APEX Managing Application Performance](https://docs.oracle.com/en/database/oracle/apex/24.1/htmdb/managing-application-performance.html) +- [APEX Managing Application Security](https://docs.oracle.com/en/database/oracle/apex/24.1/htmdb/managing-application-security.html) - [Training and Certification](https://apex.oracle.com/en/learn/training/) - [Tutorials](https://apex.oracle.com/en/learn/tutorials/) ## Documentation -- [APEX Documentation](https://docs.oracle.com/en/database/oracle/apex/23.2/index.html) -- [APEX API Reference](https://docs.oracle.com/en/database/oracle/apex/23.2/aeapi/#Oracle%C2%AE-APEX) +- [APEX Documentation]https://docs.oracle.com/en/database/oracle/apex/24.1/index.html) +- [APEX API Reference](https://docs.oracle.com/en/database/oracle/apex/24.1/api-references.html) - [APEX Universal Theme](https://apex.oracle.com/pls/apex/r/apex_pm/ut/getting-started) -- [APEX Managing Application Backups](https://docs.oracle.com/en/database/oracle/apex/22.1/htmdb/managing-application-backups.html) +- [APEX Managing Application Backups](https://docs.oracle.com/en/database/oracle/apex/24.1/htmdb/managing-application-backups.html) - [APEX Application Development Lifecycle Technical Paper](https://www.oracle.com/a/tech/docs/apex-lifecycle-management-v3.pdf) +## Tutorials/ How-To's + +- [APEX Integration with Azure AD](https://medium.com/@rana.saeed/can-your-external-users-authenticate-into-your-apex-apps-606e48f9417b) +- [APEX Integration with Real Application Security(RAS)](https://medium.com/@rana.saeed/integrating-real-application-security-ras-with-oracle-apex-33892967beff) +- [Uploading files to OCI Object Storage via APEX](https://medium.com/@devpiotrekk/uploading-files-to-oci-object-storage-via-apex-42ad396ec55d) +- [APEX Devops with Liquibase](https://medium.com/@devpiotrekk/start-your-apex-devops-adventure-with-liquibase-f8e45c3d1e6a) +- [APEX Service DevOps with DBMS_CLOUD](https://medium.com/oracledevs/apex-service-can-devops-too-dbms-cloud-on-autonomous-72be9842d2f88) +- [Integrating Data Science with APEX](https://blogs.oracle.com/ai-and-datascience/post/yolov5-models-in-apex-using-oracle-data-science) +- [Other APEX Blogs](https://blogs.oracle.com/apex/) +- [APEX YouTube channel](https://www.youtube.com/channel/UCEpIXFjcQIztReQNLymvYrQ/playlists?view=1&sort=da) + # License Copyright (c) 2024 Oracle and/or its affiliates. diff --git a/data-platform/data-development/devops-database/README.md b/data-platform/data-development/devops-database/README.md index 2e23b440f..078de7206 100644 --- a/data-platform/data-development/devops-database/README.md +++ b/data-platform/data-development/devops-database/README.md @@ -16,7 +16,6 @@ Reviewed: 18.11.2024 1. [Team Publications](#team-publications) 2. [Useful Links](#useful-links) -3. [Reusable Assets Overview](#reusable-assets-overview) # Team Publications diff --git a/data-platform/data-development/devops-database/continuous-deployment/ora-containers/README.md b/data-platform/data-development/devops-database/continuous-deployment/ora-containers/README.md index 4d7fc282d..17097a075 100644 --- a/data-platform/data-development/devops-database/continuous-deployment/ora-containers/README.md +++ b/data-platform/data-development/devops-database/continuous-deployment/ora-containers/README.md @@ -15,7 +15,7 @@ Autonomous Database Free Container Image is now available on [Oracle Container R If you are looking about information about Oracle Database running on Kubernetes check this content [here](../../../ora-database-microservices/ora-kubernetes/README.md) -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 # Table of Contents diff --git a/data-platform/data-development/devops-database/continuous-deployment/pipelines/README.md b/data-platform/data-development/devops-database/continuous-deployment/pipelines/README.md index 00dbd2f64..f86466fce 100644 --- a/data-platform/data-development/devops-database/continuous-deployment/pipelines/README.md +++ b/data-platform/data-development/devops-database/continuous-deployment/pipelines/README.md @@ -15,10 +15,11 @@ Ansible is an open source IT automation engine that automates provisioning, conf - [Chef](https://www.chef.io/) Chef has a suite of automation solutions for both infrastructure as code, and deploy applications. --[Puppet](https://www.puppet.com/) +- [Puppet](https://www.puppet.com/) Puppet enables you to deploy and manage infrastructure. -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 + # Table of Contents 1. [Useful Links](#useful-links) diff --git a/data-platform/data-development/devops-database/continuous-deployment/unit-testing/README.md b/data-platform/data-development/devops-database/continuous-deployment/unit-testing/README.md index fc6d8018e..1ae54ac24 100644 --- a/data-platform/data-development/devops-database/continuous-deployment/unit-testing/README.md +++ b/data-platform/data-development/devops-database/continuous-deployment/unit-testing/README.md @@ -7,7 +7,7 @@ It allows for automated testing of Packages, Functions, Procedures, Triggers, Views, and anything else that can be executed and observed from PL/SQL. -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 # Table of Contents diff --git a/data-platform/data-development/devops-database/continuous-integration/db-supporting-features/README.md b/data-platform/data-development/devops-database/continuous-integration/db-supporting-features/README.md index f9e685e73..3b762a6d3 100644 --- a/data-platform/data-development/devops-database/continuous-integration/db-supporting-features/README.md +++ b/data-platform/data-development/devops-database/continuous-integration/db-supporting-features/README.md @@ -12,19 +12,19 @@ The primary PL/SQL packages are: [DBMS_METADATA](https://docs.oracle.com/en/database/oracle/oracle-database/19/arpls/DBMS_METADATA.html#GUID-F72B5833-C14E-4713-A588-6BDF4D4CBA2A) - The DBMS_METADATA package provides a way for you to retrieve metadata from the database dictionary as XML or creation DDL and to submit the XML to re-create the object. Use this to generate the data definition of schema objects. -## [APEX_EXPORT](https://docs.oracle.com/en/database/oracle/apex/23.2/aeapi/APEX_EXPORT.html#GUID-6A4628A6-9F86-4394-9938-87A7FFFC7BC8) +## [APEX_EXPORT](https://docs.oracle.com/en/database/oracle/apex/24.1/aeapi/APEX_EXPORT.html) - The APEX_EXPORT package provides APIs to export the definitions of applications, files, feedback, and workspaces to text files. ## [DBMS_CLOUD_REPO](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/dbms-cloud-repo-package.html#GUID-F8F0037B-6451-4742-9144-9FCE44459F64) - The DBMS_CLOUD_REPO package provides for the use of and management of cloud-hosted code repositories from Oracle Database. Supported cloud code repositories include GitHub, AWS CodeCommit, and Azure Repos. This package is only available in OCI Autonomous Database deployments and along with integration with repositories, it provides a wrapper to DBMS_METADATA for Code Commit & Install from the repository. ## ORDS -- [ORDS REST APIs for APEX](https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/23.4/orrst/api-oracle-apex.html) A set of ORDS REST APIs to retrieve APEX Workspace & Application statistics, details with support to export workspaces, applications and application components. +- [ORDS REST APIs for APEX](https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/24.3/orrst/api-oracle-apex.html) A set of ORDS REST APIs to retrieve APEX Workspace & Application statistics, details with support to export workspaces, applications and application components. - ORDS_MODULE provides PL/SQL APIs to manage ORDS objects, OAuth clients, privileges, roles and modules. Currently, ORDS_MODULE is only documented in the code of the various packages, functions and views. The simplest deployment and set of APIS for Continuous Integration are available for APEX on Autonomous Database services, with Cloud Repository integration & Schema export via DBMS_CLOUD_REPO, and APEX applications via APEX_EXPORT. These two packages provide extensive capabilities to commit code changes into the repository with little work required to manage connectivity, generation upload and commit operations. -Reviewed: 18.11.2024 +Reviewed: 20.11.2024 # Table of Contents 1. [Team Publications](#team-publications) @@ -40,7 +40,7 @@ Reviewed: 18.11.2024 ## Documentation - [DBMS_METADATA](https://docs.oracle.com/en/database/oracle/oracle-database/19/arpls/DBMS_METADATA.html#GUID-F72B5833-C14E-4713-A588-6BDF4D4CBA2A "Oracle Database 19c PL/SQL Packages and Types Reference") - [DBMS_METADATA](https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/using-oracle-dbms_metadata-api.html#GUID-D9B1300F-B21D-416E-8B9B-C542195EF249 "Oracle Database 19c Using the Metadata APIs") -- [APEX_EXPORT](https://docs.oracle.com/en/database/oracle/apex/23.2/aeapi/APEX_EXPORT.html#GUID-6A4628A6-9F86-4394-9938-87A7FFFC7BC8 "APEX 23.2 API Reference") +- [APEX_EXPORT](https://docs.oracle.com/en/database/oracle/apex/24.1/aeapi/APEX_EXPORT.html "APEX 24.1 API Reference") - [DBMS_CLOUD_REPO](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/dbms-cloud-repo-package.html#GUID-F8F0037B-6451-4742-9144-9FCE44459F64 "Autonomous Database Supplied Package Reference") ## Blogs diff --git a/data-platform/data-development/devops-database/continuous-integration/sqlcl-liquibase/README.md b/data-platform/data-development/devops-database/continuous-integration/sqlcl-liquibase/README.md index ac6974651..ea258c449 100644 --- a/data-platform/data-development/devops-database/continuous-integration/sqlcl-liquibase/README.md +++ b/data-platform/data-development/devops-database/continuous-integration/sqlcl-liquibase/README.md @@ -13,7 +13,7 @@ You can add the functionality to read these specialized changelogs to the Liquib # The Liquibase Feature in SQLcl SQLcl Liquibase with Oracle Database provides extended functionality to the Liquibase experience compared to the open-source Liquibase client. The Liquibase feature in SQLcl enables you to execute commands to generate a changelog for a single object or for a full schema in specialized changelogs and changesets. -Reviewed: 18.11.2024 +Reviewed: 20.11.2024 # Table of Contents @@ -26,8 +26,8 @@ Reviewed: 18.11.2024 # Useful Links -- [Oracle Documentation - Oracle SQLcl Release 23.3](https://docs.oracle.com/en/database/oracle/sql-developer-command-line/23.3/sqcug/using-liquibase.html#GUID-4CA25386-E442-4D9D-B119-C1ACE6B79539 "Using Liquibase") -- [SQLcl 23.3 Downloads](https://www.oracle.com/database/sqldeveloper/technologies/sqlcl/download/) +- [Oracle Documentation - Oracle SQLcl Release 24.3.1](https://docs.oracle.com/en/database/oracle/sql-developer-command-line/24.3/sqcug/using-liquibase.html "Using Liquibase") +- [SQLcl Downloads](https://www.oracle.com/database/sqldeveloper/technologies/sqlcl/download/) - [Liquibase Community](https://www.liquibase.org/) - [Liquibase Documentation](https://docs.liquibase.com/home.html "What is Liquibase?") - [Liquibase Documentation](https://docs.liquibase.com/start/release-notes/home.html "Release Notes") diff --git a/data-platform/data-development/ora-database-microservices/backend-springboot/README.md b/data-platform/data-development/ora-database-microservices/backend-springboot/README.md index 4ca345ee6..67fbe0f53 100644 --- a/data-platform/data-development/ora-database-microservices/backend-springboot/README.md +++ b/data-platform/data-development/ora-database-microservices/backend-springboot/README.md @@ -23,7 +23,7 @@ Oracle Backend for Spring Boot is a powerful platform designed to streamline the Oracle Backend for Spring Boot empowers developers to build, deploy, and manage modern applications with enhanced performance, reliability, and scalability, utilizing the comprehensive capabilities of Oracle's ecosystem. -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 # Table of Contents diff --git a/data-platform/data-development/ora-database-microservices/microtx/README.md b/data-platform/data-development/ora-database-microservices/microtx/README.md index 495b3a46a..4e6ba29f2 100644 --- a/data-platform/data-development/ora-database-microservices/microtx/README.md +++ b/data-platform/data-development/ora-database-microservices/microtx/README.md @@ -29,7 +29,7 @@ MicroTx Enterprise can handle a large number of transactions and scale horizonta MicroTx Enterprise includes dedicated support from Oracle, offering expert assistance and quicker resolution of issues to ensure smooth operation of mission-critical applications. Oracle Transaction Manager for Microservices provides a robust, high-performance solution for managing distributed transactions in a microservices architecture. By ensuring transactional integrity and offering flexible transaction models, MicroTx simplifies the development and operation of complex business processes, making it a powerful tool for modern applications. -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 # Table of Contents diff --git a/data-platform/data-development/ora-database-microservices/ora-kubernetes/README.md b/data-platform/data-development/ora-database-microservices/ora-kubernetes/README.md index cf5dcb33f..4cf2435cf 100644 --- a/data-platform/data-development/ora-database-microservices/ora-kubernetes/README.md +++ b/data-platform/data-development/ora-database-microservices/ora-kubernetes/README.md @@ -8,7 +8,7 @@ Oracle Supports Oracle Database running in Kubernetes, Furthermore, Oracle offer In this way, developers can operate our database in the same manner as any other component of their applications, and utilize our best solutions (Exadata, Autonomous, etc.) to run these databases, and DBAs maintain control of such important topics as security, high availability, disaster recovery... -Reviewed: 18.11.2024 +Reviewed: 19.11.2024 # Table of Contents diff --git a/data-platform/data-development/ords/README.md b/data-platform/data-development/ords/README.md index 659384aac..94c310203 100644 --- a/data-platform/data-development/ords/README.md +++ b/data-platform/data-development/ords/README.md @@ -4,16 +4,16 @@ ORDS is a Java application that enables developers with SQL and database skills Oracle Database 12c JSON Document store and higher and the Oracle NoSQL Database Reviewed: 28.01.2025 - + # Table of Contents 1. [Team Publications](#team-publications) 2. [Useful Links](#useful-links) -3. [Reusable Assets Overview](#reusable-assets-overview) # Team Publications - [ORDS version 24.4 is now available, time to upgrade!](https://www.oracle.com/database/sqldeveloper/technologies/db-actions/download/) + - [JSON & Customer Managed ORDS with ADB](https://medium.com/@devpiotrekk/cloud-scalability-using-customer-managed-oracle-rest-data-service-with-autonomous-json-275fa06e8d22) - Article demonstrating the performance of Customer Managed ORDS combined with the Autonomous Database - [How ORDS help with DevOps](https://www.youtube.com/watch?v=GIRdoGlqIDE) diff --git a/data-platform/data-development/sql/README.md b/data-platform/data-development/sql/README.md index 9de17d169..ec4f4fbc3 100644 --- a/data-platform/data-development/sql/README.md +++ b/data-platform/data-development/sql/README.md @@ -1,37 +1,51 @@ # SQL Repository -Welcome to the SQL repository of the Data Development team at Oracle. This repository is designed to provide comprehensive resources, tips, and tools to help you master SQL and leverage the latest features of Oracle23ai. Below you'll find an index to all the sections available in this repository, along with a brief introduction to what you can expect in each section. +Welcome to the SQL repository of the Data Development team at Oracle. This repository is designed to provide comprehensive resources, tips, and tools to help you master SQL and leverage the latest features of Oracle23ai. Structured Query Language (SQL) is the standard programming language used for managing and manipulating relational databases. SQL allows you to perform a variety of operations such as querying data, updating records, creating and modifying schema, and controlling access to the database. It is a crucial skill for data developers and analysts, enabling efficient data retrieval and manipulation to support data-driven decision-making. -PL/SQL, the Oracle procedural extension of SQL, is a portable, high-performance transaction-processing language. PL/SQL offers several advantages over other programming languages: from being tightly integrated with SQL to leveraging High Performance and High Productivity. PL/SQL is a portable and scalable Procedural Language which also increase Manageability and support for Object Oriented Programming. +Below you'll find an index to all the sections available in this repository, along with a brief introduction to what you can expect in each section. -Reviewed: 18.11.2024 +Reviewed: 20.11.2024 -# Table of Contents -- [Team Publications](#team-publications); -- [Useful Links](#useful-links); - - -# Team Publications -- [23ai New Features Series - Part 1](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart1-developer-activity-7196221427056889856-3HF2?utm_source=share&utm_medium=member_desktop) - 23ai Database New Features on Linkedin Post - I -- [23ai New Features Series - Part 2](https://www.linkedin.com/posts/cristina-varas-menadas-591825119_oracle-oracle23ai-sql-activity-7198945654272864256-wduQ/?utm_source=share&utm_medium=member_ios) - 23ai Database New Features on Linkedin Post - II -- [23ai New Features Series - Part 3](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart3-developer-activity-7201958687005982721-Oo3B?utm_source=share&utm_medium=member_desktop) - 23ai Database New Features on Linkedin Post - III -- [23ai New Features Series - Part 4](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart4-teammates-activity-7204946957843267584-x664?utm_source=share&utm_medium=member_desktop) - 23ai Database New Features on Linkedin Post - IV -- [23ai New Features Series - Part 5](https://www.linkedin.com/posts/ppaolucci_cristinavarasmenadas-sonjameyer-activity-7207374755878887425-a--C?utm_source=share&utm_medium=member_desktop) - 23ai Database New Features on Linkedin Post - V +# Table of contents +1. [SQL Tools](#sql-tools) +2. [SQL Tips](#sql-tips) +3. [SQL Oracle23ai](#sql-oracle23ai) +4. [SQL Do It Yourself](#sql-do-it-yourself) ## SQL Tools -[SQL Tools](sql-tools/README.md): In this section, you'll find a collection of essential tools and utilities that can help streamline your SQL development process. This includes: +In the [SQL Tools](SQL_Tools/README.md) section, you'll find a collection of essential tools and utilities that can help streamline your SQL development process. This includes: - SQL client software recommendations - Database management tools - Productivity-enhancing plugins and extensions - -## SQL Do it yourself -[SQL Do it yourself](sql-do-ityourself/images/SQL_Cheat_Sheet_Version1.1.png) : here you can reach the SQL Cheat Sheet containing all the SQL principles a Developer should know. +## SQL Tips + +The [SQL Tips](SQL_Tips/README.md) section is dedicated to providing practical advice and best practices for writing efficient and effective SQL queries. Topics covered include: +- Query optimization techniques +- Common pitfalls and how to avoid them +- Advanced SQL functions and their use cases + + +## SQL Oracle23ai + +Explore the latest advancements with Oracle's cutting-edge AI capabilities in the [SQL Oracle23ai](SQL_Oracle23ai/README.md) section. Here, you will find: +- New features and enhancements in Oracle23ai +- Tutorials on integrating AI with your SQL queries +- Case studies and real-world applications of Oracle23ai + + +## SQL Do It Yourself + +The [SQL Do It Yourself](SQL_Do_It_Yourself/README.md) section encourages hands-on learning by providing a variety of exercises and projects. This includes: +- Practice problems with solutions +- Step-by-step project guides +- Interactive SQL challenges to test your skills + # Useful Links @@ -46,7 +60,7 @@ To support your development journey, here are some valuable resources related to - [The Magic of SQL by Chris Saxon - YouTube Channel](https://www.youtube.com/c/TheMagicofSQL) - [SQL and Database explained! By Connor McDonald - YouTube Channel](https://www.youtube.com/@DatabaseDude) - +We hope you find these resources helpful in your SQL journey. If you have any questions or suggestions, please feel free to contribute or reach out to the team. Happy querying! # License @@ -54,4 +68,4 @@ Copyright (c) 2024 Oracle and/or its affiliates. Licensed under the Universal Permissive License (UPL), Version 1.0. -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/data-platform/data-development/sql/sql-do-ityourself/README.md b/data-platform/data-development/sql/sql-do-ityourself/README.md index d0ef23746..224ad4294 100644 --- a/data-platform/data-development/sql/sql-do-ityourself/README.md +++ b/data-platform/data-development/sql/sql-do-ityourself/README.md @@ -5,13 +5,12 @@ This section covers Public Oracle SQL Tools: Use of SQL in extensive way: from S Providing links to public useful documentation about SQL use and tips. SQL Tips on Fortnight basis and and articles/blogs written by Oracle SQL and Database Advocates and Product Managers and SQL examples are shown here. A SQL Clue Card is also provided. -Reviewed: 18.11.2024 +Reviewed: 20.11.2024 # Table of Contents 1. [Team Publications](#team-publications) 2. [Useful Links](#useful-links) -3. [Reusable Assets Overview](#reusable-assets-overview) # 1. Team Publications diff --git a/data-platform/data-development/sql/sql-oracle23ai/README.md b/data-platform/data-development/sql/sql-oracle23ai/README.md index 2434fbb68..9bbb5c0b0 100644 --- a/data-platform/data-development/sql/sql-oracle23ai/README.md +++ b/data-platform/data-development/sql/sql-oracle23ai/README.md @@ -1,68 +1,96 @@ +# SQL Oracle Database 23ai +This section covers the new features introduced in the Oracle Database 23ai release, focusing on enhancements and functionalities in SQL for application development. +This asset contains a brief set of Oracle Database 23ai new features examples which are then included in a specific asset file for consultation. +The list of New Features will be periodically updated and adjusted. -# Oracle 23ai +## Features -This section covers the new features introduced in the Oracle Database 23ai release, focusing on enhancements and functionalities in SQL for application development. +### Aggregation over INTERVAL Data Types +- **Description**: Pass INTERVAL data types to SUM and AVG aggregate functions. +- **Example**: ```sqlSELECT SUM(interval_column) FROM table_name;` -Reviewed: 18.11.2024 +### Client Describe Call Support for Tag Options +- **Description**: Store and retrieve metadata about database objects using annotations. +- **Example**: `ALTER TABLE table_name ANNOTATE 'key' = 'value';` -# When to use this asset? +### DEFAULT ON NULL for UPDATE Statements +- **Description**: Define columns as DEFAULT ON NULL for update operations. +- **Example**: `UPDATE table_name SET column_name = DEFAULT ON NULL WHERE condition;` -To look for examples and deep dive into Oracle 23ai New Features in Database specific areas for Developers. +### Data Quality Operators +- **Description**: Introduces PHONIC_ENCODE and FUZZY_MATCH operators for string matching. +- **Example**: `SELECT PHONIC_ENCODE('word'), FUZZY_MATCH('string1', 'string2') FROM dual;` -# How to use this asset? +### Data Use Case Domains +- **Description**: Define and apply constraints for common values like credit card numbers. +- **Example**: `CREATE DOMAIN email_domain AS VARCHAR2(255) CHECK (REGEXP_LIKE(value, '^[\w._%+-]+@[\w.-]+\.[a-zA-Z]{2,}$'));` -This asset contains a brief set of Oracle Database 23ai new features examples which are then included in a specific asset file for consultation. -The list of New Features will be periodically updated and adjusted. +### Direct Joins for UPDATE and DELETE Statements +- **Description**: Join target tables in UPDATE and DELETE statements using the FROM clause. +- **Example**: `UPDATE table1 SET column1 = table2.column2 FROM table2 WHERE table1.id = table2.id;` + +### GROUP BY Column Alias or Position +- **Description**: Use column aliases or SELECT item positions in GROUP BY clauses. +- **Example**: `SELECT column1 AS col1, SUM(column2) FROM table_name GROUP BY col1;` + +### IF [NOT] EXISTS Syntax Support +- **Description**: Support for IF EXISTS and IF NOT EXISTS syntax in DDL operations. +- **Example**: `CREATE TABLE IF NOT EXISTS table_name (column1 datatype);` + +### New Database Role for Application Developers +- **Description**: Introduces DB_DEVELOPER_ROLE with necessary privileges for developers. +- **Example**: `GRANT DB_DEVELOPER_ROLE TO user_name;` + +### Oracle SQL Access to Kafka +- **Description**: Efficient access to data streams from Apache Kafka and OCI Streaming Service. +- **Example**: `SELECT * FROM kafka_table WHERE topic = 'topic_name';` + +### SELECT Without FROM Clause +- **Description**: Run SELECT expression-only queries without a FROM clause. +- **Example**: `SELECT 1+1;` + +### SQL BOOLEAN Data Type +- **Description**: Supports ISO SQL standard-compliant BOOLEAN data type. +- **Example**: `CREATE TABLE table_name (column1 BOOLEAN);` - +### SQL UPDATE RETURN Clause Enhancements +- **Description**: Enhanced RETURNING INTO clause for reporting old and new values. +- **Example**: `UPDATE table_name SET column1 = 'new_value' RETURNING column1 INTO :old_value;` + +### Schema Annotations +- **Description**: Store and retrieve metadata about database objects using name-value pairs. +- **Example**: `ANNOTATE SCHEMA 'key' = 'value';` + +### Table Value Constructor +- **Description**: Supports VALUES clause for SELECT, INSERT, and MERGE statements. +- **Example**: `INSERT INTO table_name VALUES (1, 'value');` + +### Ubiquitous Search With DBMS_SEARCH Packages +- **Description**: Index multiple schema objects for full-text search using DBMS_SEARCH. +- **Example**: `EXEC DBMS_SEARCH.CREATE_INDEX('index_name', 'table_name');` + +Reviewed: 20.11.2024 + +# Table of Contents + +1. [Team Publications](#team-publications) +2. [Useful Links](#useful-links) + +# Team Publications +- [23ai New Features Series - Part 1](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart1-developer-activity-7196221427056889856-3HF2?utm_source=share&utm_medium=member_desktop) +- [23ai New Features Series - Part 2](https://www.linkedin.com/posts/cristina-varas-menadas-591825119_oracle-oracle23ai-sql-activity-7198945654272864256-wduQ/?utm_source=share&utm_medium=member_ios) +- [23ai New Features Series - Part 3](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart3-developer-activity-7201958687005982721-Oo3B?utm_source=share&utm_medium=member_desktop) +- [23ai New Features Series - Part 4](https://www.linkedin.com/posts/sonnemeyer_23aispecialists-23ainewsabrpart4-teammates-activity-7204946957843267584-x664?utm_source=share&utm_medium=member_desktop) +- [23ai New Features Series - Part 5](https://www.linkedin.com/posts/ppaolucci_cristinavarasmenadas-sonjameyer-activity-7207374755878887425-a--C?utm_source=share&utm_medium=member_desktop) + +# Useful Links +- [Oracle Database 23ai New Features](https://apex.oracle.com/database-features/) + +# Release Information +- **Version**: 23ai +- **Applicable Offerings**: All Oracle Database offerings + +For more detailed information, please visit the [Oracle Database Features](https://apex.oracle.com/database-features/) page. # License @@ -70,6 +98,4 @@ Copyright (c) 2024 Oracle and/or its affiliates. Licensed under the Universal Permissive License (UPL), Version 1.0. -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. - - +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/data-platform/data-development/sql/sql-tools/README.md b/data-platform/data-development/sql/sql-tools/README.md index 022f24aad..bcd714c36 100644 --- a/data-platform/data-development/sql/sql-tools/README.md +++ b/data-platform/data-development/sql/sql-tools/README.md @@ -1,25 +1,16 @@ -# Oracle SQL Tools +# Oracle SQL Tools This section covers Oracle SQL Tools: from SQL Developer Desktop to SQL Developer Web on GUI ADB Database Actions to SQLcl. Providing links to public useful documentation and articles/blogs written by Oracle SQL and Database Advocates and Product Managers. +We use this asset when we want to explore the various SQL IDE development tools like SQL Developer, sqlcl and VS Code with SQL Developer extentions. - - -# When to use this asset? -When we want to explore the various SQL IDE development tools like SQL Developer, sqlcl and VS Code with SQL Developer extentions. - - - +Reviewed: 20.11.2024 # Table of Contents 1. [Team Publications](#team-publications) 2. [Useful Links](#useful-links) - -Reviewed: 24.10.2024 - - # Team Publications - Application Continuity - [Does your Database really know your application - Part 1](https://medium.com/@paulbrad71/application-continuity-on-adb-does-your-database-know-your-application-376e44d8a30) - Published on [Medium](http://www.medium.com) on 13th March 2023; @@ -44,7 +35,7 @@ Reviewed: 24.10.2024 - [SQL Developer Extension for VS Code v24.2 is now available](https://www.thatjeffsmith.com/archive/2024/07/sql-developer-extension-for-vs-code-24-2-is-now-available/) -# Dev Gyms +## Dev Gyms - [The amazing Dev Gym](https://devgym.oracle.com/pls/apex/f?p=10001:20011::::20011::) - Join the Oracle Product Management Dev Gym (FREE!) and build your Oracle technology muscles by taking workouts, quizzes, and even entire classes on SQL, PL/SQL, database design, logic and more. - [Dev Gym Workout Class - Getting Started on Oracle Cloud](https://devgym.oracle.com/pls/apex/f?p=10001:29:4444375111375:::29:P29_CLASS_ID:20342&cs=1qfK8bInptu93se5QM5iMzsT-iih_gFsY78j308d3ZUzrCzX2c0-kgiimo59pm0c_NblFShALC1oOB8MosiaSYQ) diff --git a/data-platform/exadata-cloud-at-customer/exacc-infra/single-node-vm-cluster/README.md b/data-platform/exadata-cloud-at-customer/exacc-infra/single-node-vm-cluster/README.md index 507a9b7e6..48826a8bf 100644 --- a/data-platform/exadata-cloud-at-customer/exacc-infra/single-node-vm-cluster/README.md +++ b/data-platform/exadata-cloud-at-customer/exacc-infra/single-node-vm-cluster/README.md @@ -14,6 +14,8 @@ Run each single instance database as a singleton workload or PDB in a shared RAC Run each single instance database as a singleton workload or PDB in a dedicated CDB Run each single instance database as a two–node cluster, but shut down unnecessary instances +Reviewed 19/11/2024 + # Useful Links * [About Single-Node VM Cluster](https://docs.oracle.com/en-us/iaas/exadata/doc/ecc-manage-vm-clusters.html#GUID-F528AA9C-2130-4E15-B8DE-DF65FD580789) @@ -29,6 +31,4 @@ Copyright (c) 2024 Oracle and/or its affiliates. Licensed under the Universal Permissive License (UPL), Version 1.0. -See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. - -Reviewed 30/10/2024 +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. \ No newline at end of file diff --git a/data-platform/exadata-cloud-at-customer/exacc-value/exacc-mos-note-list/README.md b/data-platform/exadata-cloud-at-customer/exacc-value/exacc-mos-note-list/README.md new file mode 100644 index 000000000..b1b546066 --- /dev/null +++ b/data-platform/exadata-cloud-at-customer/exacc-value/exacc-mos-note-list/README.md @@ -0,0 +1,56 @@ +# Exadata Cloud@Customer MOS Note List + +# +Collecting all ExaDB-C@C relevant My Oracle Support (MOS) notes + +To provide an easily accessible repository pointing to MOS notes and keeping this repository up-to-date. + +After initial collection review and update as necessary or at least once per month. + +Anyone working with ExaDB-C@C service. + +The ExaDB-C@C Specialist Team is updating the list manually and refreshing it as necessary. + +- [2333222.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2333222.1)- Exadata Cloud Service Software Versions: Grid, OS and DB versions + +- [2768569.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2768569.1) - Exadata Database Service API/CLI Alignment Matrix]: Feature and functionality comparison of the different ExaDB-C@C management tools + +- [2997504.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2997504.1) - Patching Guidelines For Lower Database Versions (Upgrade Support Required) + +- [742060.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=742060.1) - DB version release roadmap (not ExaDB-C@C specific, but applicable) + +- [2663308.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2663308.1) - Using External Storage with Exadata: Storage connection to the DB servers + +- [2809393.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2809393.1) - Exadata Cloud Compute Node Backup and Restore Operations: DB servers' local disk backup and restore practices + +- [2760233.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2760233.1) - Move Data Guard Redo Traffic to the Backup Network in ExaCC Gen 2: How to move DG traffic off the client network + +- [2495335.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2495335.1) - Updating the Cloud Tooling for Exadata Cloud Environment (DBaaS Tool a.k.a Cloud Tooling): How to check the version of DBaaS and update if necessary + +- [2761360.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2761360.1) - Oracle ACFS Snapshot Use Cases on Exadata: How to use ACFS on ExaDB-C@C + +- [2831147.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2831147.1) - Sparse Disk Group resize/shrinking issues OS_MB is not matching with Total_MB post resize: Known issue around SPARSE DG resizing + +- [2774779.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2774779.1) - (ODyS) Oracle Dynamic Scaling Suite Main Index Page: Dynamic Scaling Index note + +- [2719916.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2719916.1) - (ODyS) Oracle Dynamic Scaling engine - Scale-up and Scale-down automation utility for OCI DB System (ExaDB-D/ExaDB-C@C): Dynamic Scaling operations + +- [2770544.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2770544.1) - (ODyS) Oracle Dynamic Scaling - Remote Plug-in: Dynamic Scaling Remote Plug-in guide + +- [2762561.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2762561.1) - (ODyS) Oracle Dynamic Scaling - Charts maker for Dynamic Scaling engine utility: Dynamic Scaling chart creation from log files + +- [2774467.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2774467.1) - (ODyS) Oracle Dynamic Scaling on Docker: Setting up Dynamic Scaling on Docker + +- [2828699.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2828699.1) - (ODyS) Oracle Dynamic Scaling on Podman: Setting up Dynamic Scaling on Podman + +- [2940064.1](https://support.oracle.com/epmos/faces/DocumentDisplay?id=2940064.1) - (ODyS) Oracle Dynamic Scaling on Kubernetes: Setting up Dynamic Scaling on Kubernetes + +Reviewed 22/11/2024 + +# License + +Copyright (c) 2024 Oracle and/or its affiliates. + +Licensed under the Universal Permissive License (UPL), Version 1.0. + +See [LICENSE](https://github.com/oracle-devrel/technology-engineering/blob/main/LICENSE) for more details. diff --git a/data-platform/modernise/README.md b/data-platform/modernise/README.md index 83b338c30..0eb4f70a2 100644 --- a/data-platform/modernise/README.md +++ b/data-platform/modernise/README.md @@ -42,6 +42,8 @@ Reviewed: 15.11.2024 - Cloud Coaching Webinar explaining different multicloud network architectures for migrating databases to OCI using Zero Downtime Migration - [Migrate On-Premises Database to OCI using Oracle Database Cloud Backup Module and RMAN ](https://amalrajputhenchira.wordpress.com/2024/08/13/migrate-on-premises-oracle-database-to-oci-using-oracle-database-cloud-backup-module-and-rman/) - Detailed, step-by-step instructions for performing Physical Offline Migration of On-Premises database to Co-Managed Database in OCI using Oracle Database Cloud Backup Module and RMAN , available on Amalraj Puthenchira's public platform +- [How to Modernise OS and Shape of your Oracle Base Database Standalone Systems on Intel ? ](https://amalrajputhenchira.wordpress.com/2024/12/03/how-to-modernise-os-and-shape-of-your-oracle-base-database-standalone-systems-on-intel/) + - Detailed, step-by-step instructions for Modernising OS and Shape of Oracle Base Database Standalone Systems on Intel , available on Amalraj Puthenchira's public platform # Useful Links diff --git a/data-platform/modernise/data-pump/README.md b/data-platform/modernise/data-pump/README.md index a6e04d358..b43c7cb85 100644 --- a/data-platform/modernise/data-pump/README.md +++ b/data-platform/modernise/data-pump/README.md @@ -43,6 +43,8 @@ Reviewed: 31.10.2024 - The Data Pump Log Analyzer is a powerful Python script designed to parse and analyze Oracle Data Pump log files. The tool provides valuable insights into Data Pump operations key metrics and performance data - [Data Pump Log Analyzer: Comprehensive Guide](https://macsdata.com/oracle/data-pump-log-analyzer-guide) - The ultimate resource for mastering every option in detail and to unlock the full potential of the Python script, available on Marcus Doeringer's public platform +- [How to Perform DataPump Import to Oracle 19c Database using Export dumps on OCI Object Storage ?](https://amalrajputhenchira.wordpress.com/2025/01/17/how-to-perform-datapump-import-to-oracle-19c-database-using-export-dumps-on-oci-object-storage/) + - Detailed, step-by-step instructions for Performing DataPump Import to Oracle Database 19c with Export dumps on OCI Object Storage , available on Amalraj Puthenchira's public platform # Useful Links - [Loading Data from Google Cloud Storage to Oracle Database](https://database-heartbeat.com/2024/10/01/google-storage-to-oracle-database/) diff --git a/data-platform/modernise/database-upgrade/README.md b/data-platform/modernise/database-upgrade/README.md index def2bb558..92a8346dd 100644 --- a/data-platform/modernise/database-upgrade/README.md +++ b/data-platform/modernise/database-upgrade/README.md @@ -23,7 +23,10 @@ Reviewed: 22.10.2024 - Step-by-step guide by Mihai Costeanu demonstrating how to upgrade your 19c Autonomous Database to 23ai using OCI console ADB cloning function - [Fast Track your Oracle Database Upgrade from 19c to 23ai in Exadata Database Service on Dedicated Infrastructure using AutoUpgrade](https://amalrajputhenchira.wordpress.com/2024/11/08/fast-track-your-oracle-database-upgrade-from-19c-to-23ai-in-exadata-database-service-on-dedicated-infrastructure-using-autoupgrade-2/) - Detailed, step-by-step instructions for performing Database Upgrade from 19c to 23ai in Exadata Database Service on Dedicated Infrastructure using AutoUpgrade , available on Amalraj Puthenchira's public platform - +- [Upgrade your Oracle Base Database in OCI from 19c to 23ai - Part 1 - GI Upgrade](https://amalrajputhenchira.wordpress.com/2025/01/09/upgrade-your-oracle-base-database-in-oci-to-23ai-part-1-gi-upgrade/) + - Detailed, step-by-step instructions for Upgrading Grid Infrastructure of Oracle Base Database in OCI from 19c to 23ai , available on Amalraj Puthenchira's public platform +- [Upgrade your Oracle Base Database in OCI from 19c to 23ai - Part 2 - Database Upgrade](https://amalrajputhenchira.wordpress.com/2025/01/15/upgrade-your-oracle-base-database-in-oci-from-19c-to-23ai-part-2-database-upgrade/) + - Detailed, step-by-step instructions for Upgrading Database of Oracle Base Database in OCI from 19c to 23ai , available on Amalraj Puthenchira's public platform # Useful Links - [Database Server Upgrade/Downgrade Compatibility Matrix on MOS (Doc ID 551141.1)](https://support.oracle.com/epmos/faces/DocumentDisplay?id=551141.1) - [LiveLabs: Hitchhiker's Guide for Upgrading to Oracle Database 23ai](https://apexapps.oracle.com/pls/apex/f?p=133:180:1060521706151::::wid:3943) diff --git a/data-platform/modernise/goldengate/README.md b/data-platform/modernise/goldengate/README.md index bbd67ea45..0c06602d7 100644 --- a/data-platform/modernise/goldengate/README.md +++ b/data-platform/modernise/goldengate/README.md @@ -15,7 +15,7 @@ Reviewed: 21.08.2024 - [How to connect OCI Goldengate using a Private IP & OCI Bastion during Database Migration](https://medium.com/@snoozrocks/how-to-connect-oci-goldengate-using-a-private-ip-oci-bastion-during-database-migration-b04012ed8e30) - This blog post provides a high-level overview of the steps to connect OCI Goldengate Console using OCI Bastion Service using private subnet - +- [Step by Step Guide for Readiness Precheck For Database Migrations using Oracle GoldenGate Microservices](https://medium.com/@snoozrocks/step-by-step-guide-for-readiness-precheck-for-database-migrations-using-oracle-goldengate-90b93876fae2) # Useful Links - [Oracle Goldengate 21c](https://docs.oracle.com/en/middleware/goldengate/core/21.3/index.html) diff --git a/data-platform/modernise/oci-database-migration/README.md b/data-platform/modernise/oci-database-migration/README.md index 5ac3c2970..04b1d3726 100644 --- a/data-platform/modernise/oci-database-migration/README.md +++ b/data-platform/modernise/oci-database-migration/README.md @@ -2,7 +2,7 @@ Oracle Cloud Infrastructure Database Migration (DMS) is a fully managed cloud service for migrating Oracle and MySQL databases to Oracle Cloud Infrastructure (OCI). The Oracle database migration is powered by the Zero Downtime Migration engine and provides a complete self-service experience for logical offline or online migration using Oracle GoldenGate for minimal downtime. As a source, DMS supports on-premises Oracle databases running on Linux, AIX, or Solaris as well as Amazon RDS and co-managed databases in OCI that can be migrated to Autonomous Database Serverless/Dedicated, OCI co-managed databases and Oracle Database@Azure. For MySQL migrations DMS supports MySQL Database Server installations as well as MySQL services from OCI, Amazon, Azure and Google Cloud that can be migrated to OCI MySQL Heatwave. -Reviewed: 21.10.2024 +Reviewed: 15.01.2025 # Table of Contents @@ -19,6 +19,8 @@ Reviewed: 21.10.2024 - YouTube video explaining how OCI Database Migration is integrated with OCI GoldenGate for replication during online migrations. More information about this feature of OCI Database Migration is also available [here](https://blogs.oracle.com/dataintegration/post/oci-database-migration-announces-oci-goldengate-integration). - [A Practical Guide to Oracle Cloud Migration Using OCI DMS](https://blogs.oracle.com/dataintegration/post/migrate-data-from-onprem-oracle-11g-database-to-oci-with-minimal-downtime-using-oci-database-migration-dms) - This blog helps you in planning, designing, and implementing the process of migrating your On-premises Oracle Database 11g to OCI Base Database Service (Oracle 19c) using OCI Database Migration Service (DMS). +- [Migrate an 11g Database on OCVS into Autonomous Database](https://techrochet.com/migrate-an-11g-database-on-ocvs-into-autonomous-database) + - A step-by-step guide on how to migrate an 11g (11.2.0.4) database on Oracle VMware Service (OCVS) into an Autonomous Database using Database Migration Service (DMS) # Useful Links diff --git a/data-platform/modernise/zero-downtime-migration/README.md b/data-platform/modernise/zero-downtime-migration/README.md index 03feaeffb..de319fcf4 100644 --- a/data-platform/modernise/zero-downtime-migration/README.md +++ b/data-platform/modernise/zero-downtime-migration/README.md @@ -17,6 +17,8 @@ Reviewed: 08.11.2024 - Red Hot Webinar delivered by Amalraj Puthenchira and Carlos Alvarez capturing an in depth description of Zero Downtime Migration - features, functionality, use-cases and technical demonstration - [Migrate your Oracle Database to OCI using ZDM (Zero-Downtime Migration) - Cloud Coaching Webinar](https://www.youtube.com/watch?v=SXb7KVZjpV8) - Cloud Coaching Webinar, including a technical demonstration, delivered by Amalraj Puthenchira and Bilegt Bat Ochir around the use of ZDM for efficient migration into OCI. +- [Fast-track your Journey to AI with Oracle and Azure (Migration to Oracle ADB23ai@Azure using Zero-Downtime Migration Logical Online) - Developer Coaching Webinar](https://www.youtube.com/watch?v=SanGj96PoxI) + - Developer Coaching Webinar, including a technical demonstration, delivered by Mihai Costeanu and Emiel Ramakers around the use of ZDM for efficient migration to Oracle ADB23ai@Azure . ## Technical Guides diff --git a/data-platform/open-source-data-platforms/oci-postgresql/README.md b/data-platform/open-source-data-platforms/oci-postgresql/README.md index 015d5fac7..566ecb220 100644 --- a/data-platform/open-source-data-platforms/oci-postgresql/README.md +++ b/data-platform/open-source-data-platforms/oci-postgresql/README.md @@ -18,6 +18,8 @@ OCI Database with PostgreSQL is designed for high availability by offering durab - [OCI PostgreSQL to OCI PostgreSQL cross-region replication with OCI GoldenGate — Part 1](https://medium.com/@devpiotrekk/oci-postgresql-to-oci-postgresql-cross-region-replication-with-oci-goldengate-introduction-e0492fc37b92) - [OCI PostgreSQL to OCI PostgreSQL cross-region replication with OCI GoldenGate — Part 2](https://medium.com/@devpiotrekk/oci-postgresql-to-oci-postgresql-cross-region-replication-with-oci-goldengate-oci-postgresql-d4fcffc47498) - [OCI PostgreSQL to OCI PostgreSQL cross-region replication with OCI GoldenGate — Part 3](https://medium.com/@devpiotrekk/oci-postgresql-to-oci-postgresql-cross-region-replication-with-oci-goldengate-oci-goldengate-4ccd5dea4d6c) +- [OCI PostgreSQL replication with pglogical](https://medium.com/@devpiotrekk/replicating-oci-database-with-postgresql-using-pglogical-118182ff08f9) +- [OCI PostgreSQL vector search with pgvector - Part 1](https://medium.com/@devpiotrekk/vector-search-with-pgvector-and-oci-database-with-postgresql-part-1-0915e5296148) # Useful Links diff --git a/manageability-and-operations/observability-and-manageability/logging/README.md b/manageability-and-operations/observability-and-manageability/logging/README.md index bf103444f..5cecc752e 100644 --- a/manageability-and-operations/observability-and-manageability/logging/README.md +++ b/manageability-and-operations/observability-and-manageability/logging/README.md @@ -2,7 +2,7 @@ The Oracle Cloud Infrastructure Logging service is a highly scalable and fully managed single pane of glass for all the logs in your tenancy. Logging provides access to logs from Oracle Cloud Infrastructure resources. These logs include critical diagnostic information that describes how resources are performing and being accessed. -Reviewed: 23.09.2024 +Reviewed: 10.01.2025 # Table of Contents @@ -20,6 +20,7 @@ Reviewed: 23.09.2024 - [Stream OCI Logs to Splunk](https://learnoci.cloud/stream-oci-logs-to-splunk-v9-1-c71c93e470fe?sk=8a7c3f6201bfcd847a83d36247eddfa7) - [Enable OCI VCN Flow logs in an easy way](https://karthicin.medium.com/enable-oci-vcn-flow-logs-in-easy-way-c986c6cda6c0) - [OCI Cross-tenancy log management](https://learnoci.cloud/oci-cross-tenancy-log-management-8165c6048827) +- [How to export OCI logs to file](https://learnoci.cloud/how-to-export-oci-logs-to-file-e0461f369f61) # Useful Links - [Logging](https://docs.oracle.com/en-us/iaas/Content/Logging/home.htm) diff --git a/manageability-and-operations/observability-and-manageability/oci-monitoring/README.md b/manageability-and-operations/observability-and-manageability/oci-monitoring/README.md index 90bedb96d..828868c68 100644 --- a/manageability-and-operations/observability-and-manageability/oci-monitoring/README.md +++ b/manageability-and-operations/observability-and-manageability/oci-monitoring/README.md @@ -2,7 +2,7 @@ Use Monitoring to query metrics and manage alarms. Metrics and alarms help monitor the health, capacity, and performance of your cloud resources. -Reviewed: 02.10.2024 +Reviewed: 25.11.2024 # Table of Contents @@ -17,6 +17,7 @@ Reviewed: 02.10.2024 - [Customised Alarm Notification in OCI](https://karthicin.medium.com/customised-alarm-notification-in-oci-e5b367ca20bc) - [How to monitor the resource usage on your OCI Instances using Cloud Guard Instance Security Queries](https://learnoci.cloud/how-to-monitor-the-resource-usage-on-your-oci-instances-using-cloud-guard-instance-security-queries-342836ca2811) - [How to install Arkime(Moloch) using embedded Open Search](https://learnoci.cloud/how-to-install-arkime-moloch-using-embedded-open-search-19a7a58f8eff) +- [OCI resource scheduler](https://learnoci.cloud/oci-resource-scheduler-997f83e2b063) # Useful Links diff --git a/manageability-and-operations/operations-advisory/README.md b/manageability-and-operations/operations-advisory/README.md index 86c0e549f..4fccd118b 100644 --- a/manageability-and-operations/operations-advisory/README.md +++ b/manageability-and-operations/operations-advisory/README.md @@ -4,7 +4,7 @@ The Operations Advisory provides critical guidance to Customers on how to define - [Multi-cloud operations - 3rd party tools that support OCI](https://learnoci.cloud/multi-cloud-operations-3rd-party-tools-that-support-oci-091207d923ed) -Reviewed: 04.11.2024 +Reviewed: 08.01.2025 # Table of Contents @@ -23,6 +23,15 @@ Reviewed: 04.11.2024 - [Create a DataGuard Association on Oracle Cloud Exa servers using Ansible Collections](https://www.linkedin.com/pulse/create-dataguard-association-oracle-cloud-exa-servers-eugene-simos-hflkf) - [Why you should start thinking about your cloud operations right at the start of your cloud journey and what to consider](https://medium.com/@manu72tomcat/why-you-should-start-thinking-about-your-cloud-operations-right-at-the-start-of-your-cloud-journey-6000b36d9786) - [Script to stop/start OCI DB Node](https://karthicin.medium.com/script-to-stop-start-oci-db-node-41ccad09fd35) +- [Using Digger a CI/CD tool for terraform/Opentofu](https://karthicin.medium.com/using-digger-a-ci-cd-tool-for-terraform-opentofu-712acd09d73e) +- [Let's try k8sgp](https://karthicin.medium.com/lets-try-k8sgpt-50884e5e4ba6) +- [Let's try kyverno-json for Terraform](https://karthicin.medium.com/lets-try-kyverno-json-for-terraform-5b468ffc7a22) +- [Why Day-2 Operations Matter: Introduction - Episode 1](https://www.linkedin.com/posts/manuelafioramonti_why-day-2-operations-matter-and-how-to-tackle-activity-7264926567192002560-W84W?utm_source=share&utm_medium=member_desktop) +- [Why Day-2 Operations Matter: A dive into SecOps - Episode 2](https://www.linkedin.com/posts/manuelafioramonti_cloud-operations-episode-2-a-shallow-activity-7266788536681463808-7q7O?utm_source=share&utm_medium=member_desktop) +- [Why Day-2 Operations Matter: OS management and its challenges - Episode 3](https://www.linkedin.com/posts/manuelafioramonti_why-day-2-operations-matter-episode-3-activity-7269724104906199040-5xVi?utm_source=share&utm_medium=member_desktop) +- [Why Day-2 Operations Matter: Engineered Systems - Episode 4](https://www.linkedin.com/pulse/why-day-2-operations-matter-engineered-systems-4-manuela-fioramonti-65ccf/?trackingId=MC8FZNrum0OgCFOpJqORnw%3D%3D) +- [Why Day-2 Operations Matter: Autonomous Database - Episode 5](https://www.linkedin.com/pulse/why-day-2-operations-matter-autonomous-database-magic-fioramonti-lhg5f/?trackingId=s3bkPoanDc0iiUglgz9Knw%3D%3D) +- [Why Day-2 Operations Matter: Episode 6- Applications](https://www.linkedin.com/pulse/why-day-2-operations-matter-episode-6-applications-fioramonti-cwqof/?trackingId=0VGMfW8sTlfLtxWi5yIbOA%3D%3D) diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/README.md b/security/security-design/shared-assets/oci-security-health-check-standard/README.md index 7a1a755c3..de7703cee 100644 --- a/security/security-design/shared-assets/oci-security-health-check-standard/README.md +++ b/security/security-design/shared-assets/oci-security-health-check-standard/README.md @@ -2,7 +2,7 @@ Owner: Olaf Heimburger -Version: 241011 +Version: 241206 Reviewed: 01.02.2024 @@ -19,6 +19,15 @@ The *OCI Security Health Check - Standard Edition* checks an OCI tenancy for [CI This asset covers the OCI platform as specified in the *CIS Oracle Cloud Infrastructure Foundations Benchmark*, only. Any workload provisioned in Databases, Compute VMs (running any Operating System), the Container Engine for Kubernetes, or in the VMware Solution is *out of scope* of the *OCI Security Health Check*. +**This is not an official Oracle application and it is not supported by Oracle Support.** + +## Before you begin + +The main goals of this script are: + +- Make the run as easy and smooth as possible. +- Do not affect your desktop whenever possible. + ## Complete Runtime Example See the *OCI Security Health Check - Standard Edition* in action and watch the [OCI Health Checks - Self Service video](https://www.youtube.com/watch?v=EzjKLxfxaAM). @@ -29,22 +38,22 @@ See the *OCI Security Health Check - Standard Edition* in action and watch the [ Before running the *OCI Security Health Check - Standard Edition* you should download and verify it. - - Download the latest distribution [oci-security-health-check-standard-241011.zip](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip). + - Download the latest distribution [oci-security-health-check-standard-241206.zip](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip). - Download the respective checksum file: - - [oci-security-health-check-standard-241011.sha512](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512). - - [oci-security-health-check-standard-241011.sha512256](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512256). + - [oci-security-health-check-standard-241206.sha512](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512). + - [oci-security-health-check-standard-241206.sha512256](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512256). - Verify the integrity of the distribution. Both files must be in the same directory (for example, in your downloads directory). On MacOS: ``` cd - shasum -a 512256 -c oci-security-health-check-standard-241011.sha512256 + shasum -a 512256 -c oci-security-health-check-standard-241206.sha512256 ``` On Linux (including Cloud Shell): ``` cd - sha512sum -c oci-security-health-check-standard-241011.sha512 + sha512sum -c oci-security-health-check-standard-241206.sha512 ``` **Reject the downloaded file if the check fails!** @@ -57,10 +66,10 @@ In OCI Cloud Shell you can do a short cut without downloading the files mentione 2. Open Cloud Shell 3. Run these commands in your Cloud Shell: ``` - wget -q https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip - wget -q https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512 - sha512sum -c oci-security-health-check-standard-241011.sha512 - unzip -q oci-security-health-check-standard-241011.zip + wget -q https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip + wget -q https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512 + sha512sum -c oci-security-health-check-standard-241206.sha512 + unzip -q oci-security-health-check-standard-241206.zip ``` ## Prepare the OCI Tenancy @@ -76,7 +85,8 @@ quickest way. If you decide to use this option, please continue reading in ### Recurring usage -For recurring usage, setting up a group for auditing is recommended. For setting this up follow the steps documented next. +For recurring usage, setting up a group for auditing is recommended. For setting this up follow the steps documented in the next section. +This applies for scenarios using the OCI Cloud Shell with public Internet access. For additional usage scenarios see the detailed instructions [README](files/oci-security-health-check-standard/README.md). ### Setting up an *Auditor* group and policy @@ -88,20 +98,22 @@ To create a group for auditing do the following steps: - Create a policy `pcy-auditing` with these statements (if your tenancy does not have Domains, replace `'Default'/'grp-auditors'` with `grp-auditors`): ``` allow group 'Default'/'grp-auditors' to inspect all-resources in tenancy - allow group 'Default'/'grp-auditors' to read instances in tenancy - allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read buckets in tenancy - allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy - allow group 'Default'/'grp-auditors' to read public-ips in tenancy + allow group 'Default'/'grp-auditors' to read dns in tenancy + allow group 'Default'/'grp-auditors' to read domains in tenancy allow group 'Default'/'grp-auditors' to read file-family in tenancy allow group 'Default'/'grp-auditors' to read instance-configurations in tenancy + allow group 'Default'/'grp-auditors' to read instances in tenancy + allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy allow group 'Default'/'grp-auditors' to read network-security-groups in tenancy + allow group 'Default'/'grp-auditors' to read public-ips in tenancy allow group 'Default'/'grp-auditors' to read resource-availability in tenancy - allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read users in tenancy allow group 'Default'/'grp-auditors' to read vss-family in tenancy - allow group 'Default'/'grp-auditors' to read dns in tenancy allow group 'Default'/'grp-auditors' to use cloud-shell in tenancy + allow group 'Default'/'grp-auditors' to use cloud-shell-public-network in tenancy ``` - Assign a user to the `grp-auditors` group. - Log out of the OCI Console. @@ -117,7 +129,7 @@ After a completed run you will find a directory with a name starting with your t To start with reviewing the results, open the file named `tenancy_name_YYYYMMDDHHmmss_standard_cis_html_summary_report.html`. It may look like this example: -![Flyer](./files/resources/Example_Output.png) +![Example](./files/resources/Example_Output.png) # Known Issues diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.md b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.md index 76e73fb61..e963d5c36 100644 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.md +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.md @@ -2,7 +2,7 @@ Owner: Olaf Heimburger -Version: 241011 (cis_report.py version 2.8.4+) +Version: 241206 (cis_report.py version 2.8.6) ## When to use this asset? @@ -12,8 +12,14 @@ The *OCI Security Health Check - Standard Edition* checks an OCI tenancy for CIS This asset covers the OCI platform as specified in the *CIS Oracle Cloud Infrastructure Foundations Benchmark*, only. Any workload provisioned in Databases, Compute VMs (running any Operating System), the Container Engine for Kubernetes, or in the VMware Solution is *out of scope* of the *OCI Security Health Check*. -This is not an official Oracle application and it is not supported -by Oracle Support. +**This is not an official Oracle application and it is not supported by Oracle Support.** + +## Before you begin + +The main goals of this script are: + +- Make the run as easy and smooth as possible. +- Do not affect your desktop whenever possible. ## Usage @@ -21,22 +27,22 @@ by Oracle Support. Before running the *OCI Security Health Check - Standard Edition* you should download and verify it. - - Download the latest distribution [oci-security-health-check-standard-241011.zip](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip). + - Download the latest distribution [oci-security-health-check-standard-241206.zip](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip). - Download the respective checksum file: - - [oci-security-health-check-standard-241011.sha512](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512). - - [oci-security-health-check-standard-241011.sha512256](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512256). + - [oci-security-health-check-standard-241206.sha512](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512). + - [oci-security-health-check-standard-241206.sha512256](https://github.com/oracle-devrel/technology-engineering/raw/main/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512256). - Verify the integrity of the distribution. Both files must be in the same directory (for example, in your downloads directory). On MacOS: ``` cd - shasum -a 512256 -c oci-security-health-check-standard-241011.sha512256 + shasum -a 512256 -c oci-security-health-check-standard-241206.sha512256 ``` On Linux (including Cloud Shell): ``` cd - sha512sum -c oci-security-health-check-standard-241011.sha512 + sha512sum -c oci-security-health-check-standard-241206.sha512 ``` **Reject the downloaded file when the check fails!** @@ -67,45 +73,109 @@ To create a group for auditing do the following steps: - For tenancies **without** Identity Domains use ``` allow group grp-auditors to inspect all-resources in tenancy - allow group grp-auditors to read instances in tenancy - allow group grp-auditors to read load-balancers in tenancy + allow group grp-auditors to read audit-events in tenancy allow group grp-auditors to read buckets in tenancy - allow group grp-auditors to read nat-gateways in tenancy - allow group grp-auditors to read public-ips in tenancy + allow group grp-auditors to read dns in tenancy + allow group grp-auditors to read domains in tenancy allow group grp-auditors to read file-family in tenancy allow group grp-auditors to read instance-configurations in tenancy + allow group grp-auditors to read instances in tenancy + allow group grp-auditors to read load-balancers in tenancy + allow group grp-auditors to read nat-gateways in tenancy allow group grp-auditors to read network-security-groups in tenancy + allow group grp-auditors to read public-ips in tenancy allow group grp-auditors to read resource-availability in tenancy - allow group grp-auditors to read audit-events in tenancy allow group grp-auditors to read users in tenancy allow group grp-auditors to read vss-family in tenancy - allow group grp-auditors to read dns in tenancy allow group grp-auditors to use cloud-shell in tenancy + allow group grp-auditors to use cloud-shell-public-network in tenancy ``` - For tenancies **with** Identity Domains use ``` allow group 'Default'/'grp-auditors' to inspect all-resources in tenancy - allow group 'Default'/'grp-auditors' to read instances in tenancy - allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read buckets in tenancy - allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy - allow group 'Default'/'grp-auditors' to read public-ips in tenancy + allow group 'Default'/'grp-auditors' to read dns in tenancy + allow group 'Default'/'grp-auditors' to read domains in tenancy allow group 'Default'/'grp-auditors' to read file-family in tenancy allow group 'Default'/'grp-auditors' to read instance-configurations in tenancy + allow group 'Default'/'grp-auditors' to read instances in tenancy + allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy allow group 'Default'/'grp-auditors' to read network-security-groups in tenancy + allow group 'Default'/'grp-auditors' to read public-ips in tenancy allow group 'Default'/'grp-auditors' to read resource-availability in tenancy - allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read users in tenancy allow group 'Default'/'grp-auditors' to read vss-family in tenancy - allow group 'Default'/'grp-auditors' to read dns in tenancy allow group 'Default'/'grp-auditors' to use cloud-shell in tenancy + allow group 'Default'/'grp-auditors' to use cloud-shell-public-network in tenancy ``` - Assign a user to the `grp-auditors` group - Log out of the OCI Console ### Run the OCI Security Health Check in OCI Cloud Shell -The recommended way is to run the *OCI Security Health Check - Standard* in the OCI Cloud Shell. It does not require any additional configuration on a local desktop machine. +The recommended way is to run the *OCI Security Health Check - Standard* in the [OCI Cloud Shell](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/cloudshellintro.htm). It does not require any additional configuration on a local desktop machine. + +#### Required IAM Policy statements + +The following policy statement is part of the recommended policy statements for the `grp-auditors` group: +``` +allow group 'Default'/'grp-auditors' to use cloud-shell in tenancy +``` + +#### Networking Options for OCI Cloud Shell + +OCI Cloud Shell sessions do not allow for any incoming connections, and there is no public IP address available. + +So far, the *OCI Security Health Check - Standard Edition* in OCI Cloud Shell has been tested with Public Network Access only. + +For details on OCI Cloud Shell Networking refer to [OCI Cloud Shell Networking](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/cloudshellintro_topic-Cloud_Shell_Networking.htm#cloudshellintro_topic-Cloud_Shell_Networking) documentation. + + + #### Upload the release file @@ -117,10 +187,10 @@ The recommended way is to run the *OCI Security Health Check - Standard* in the - Upload the distribution file. - Extract it ``` - unzip -q oci-security-health-check-standard-241011.zip + unzip -q oci-security-health-check-standard-241206.zip ``` -### Run the script +#### Run the script - Change directory into `oci-security-health-check-standard`: ``` $ cd oci-security-health-check-standard @@ -142,6 +212,7 @@ The recommended way is to run the *OCI Security Health Check - Standard* in the ``` ./standard.sh -h ``` + ### Using an OCI Compute VM (Oracle Linux) - Create a Dynamic Group @@ -190,11 +261,11 @@ The recommended way is to run the *OCI Security Health Check - Standard* in the Follow the instructions to select /usr/bin/python3.9 - Log out - - From your desktop, upload the `oci-security-health-check-standard-241011.zip` file to the Compute VM using any SFTP client. + - From your desktop, upload the `oci-security-health-check-standard-241206.zip` file to the Compute VM using any SFTP client. - Log into the Compute VM - Extract the distribution ``` - unzip -q oci-security-health-check-standard-241011.zip + unzip -q oci-security-health-check-standard-241206.zip ``` - Change directory into `oci-security-health-check-standard`: ``` diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.txt b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.txt index 256dd75fa..528308620 100644 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.txt +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/README.txt @@ -2,7 +2,7 @@ OCI Security Health Check - Standard Edition ============================================ Owner: Olaf Heimburger -Version: 241011 (cis_report.py version 2.8.4) +Version: 241206 (cis_report.py version 2.8.6) When to use this asset? @@ -42,36 +42,40 @@ Usage - Create a policy pcy-auditing with these statements: - For tenancies without Identity Domains use allow group grp-auditors to inspect all-resources in tenancy - allow group grp-auditors to read instances in tenancy - allow group grp-auditors to read load-balancers in tenancy + allow group grp-auditors to read audit-events in tenancy allow group grp-auditors to read buckets in tenancy - allow group grp-auditors to read nat-gateways in tenancy - allow group grp-auditors to read public-ips in tenancy + allow group grp-auditors to read dns in tenancy + allow group grp-auditors to read domains in tenancy allow group grp-auditors to read file-family in tenancy allow group grp-auditors to read instance-configurations in tenancy + allow group grp-auditors to read instances in tenancy + allow group grp-auditors to read load-balancers in tenancy + allow group grp-auditors to read nat-gateways in tenancy allow group grp-auditors to read network-security-groups in tenancy + allow group grp-auditors to read public-ips in tenancy allow group grp-auditors to read resource-availability in tenancy - allow group grp-auditors to read audit-events in tenancy allow group grp-auditors to read users in tenancy allow group grp-auditors to read vss-family in tenancy - allow group grp-auditors to read dns in tenancy allow group grp-auditors to use cloud-shell in tenancy + allow group grp-auditors to use cloud-shell-public-network in tenancy - For tenancies *with* Identity Domains use allow group 'Default'/'grp-auditors' to inspect all-resources in tenancy - allow group 'Default'/'grp-auditors' to read instances in tenancy - allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read buckets in tenancy - allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy - allow group 'Default'/'grp-auditors' to read public-ips in tenancy + allow group 'Default'/'grp-auditors' to read dns in tenancy + allow group 'Default'/'grp-auditors' to read domains in tenancy allow group 'Default'/'grp-auditors' to read file-family in tenancy allow group 'Default'/'grp-auditors' to read instance-configurations in tenancy + allow group 'Default'/'grp-auditors' to read instances in tenancy + allow group 'Default'/'grp-auditors' to read load-balancers in tenancy + allow group 'Default'/'grp-auditors' to read nat-gateways in tenancy allow group 'Default'/'grp-auditors' to read network-security-groups in tenancy + allow group 'Default'/'grp-auditors' to read public-ips in tenancy allow group 'Default'/'grp-auditors' to read resource-availability in tenancy - allow group 'Default'/'grp-auditors' to read audit-events in tenancy allow group 'Default'/'grp-auditors' to read users in tenancy allow group 'Default'/'grp-auditors' to read vss-family in tenancy - allow group 'Default'/'grp-auditors' to read dns in tenancy allow group 'Default'/'grp-auditors' to use cloud-shell in tenancy + allow group 'Default'/'grp-auditors' to use cloud-shell-public-network in tenancy - Assign a user to the grp-auditors group - Log out of OCI Console @@ -81,7 +85,7 @@ Usage - From the menu select the Cloud Shell item. - When running it the first time: - Upload the provided ZIP file. - - Extract it with unzip -q oci-security-health-check-standard-241011.zip + - Extract it with unzip -q oci-security-health-check-standard-241206.zip - Change directory into oci-security-health-check-standard $ cd oci-security-health-check-standard $ screen @@ -138,11 +142,11 @@ Usage - Log out - From your desktop, upload the - "oci-security-health-check-standard-241011.zip" file to the Compute VM + "oci-security-health-check-standard-241206.zip" file to the Compute VM using any SFTP client. - Log into the Compute VM - Extract the distribution - unzip -q oci-security-health-check-standard-241011.zip + unzip -q oci-security-health-check-standard-241206.zip - Change directory into "oci-security-health-check-standard": cd oci-security-health-check-standard diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/scripts/cis_reports/cis_reports.py b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/scripts/cis_reports/cis_reports.py index c9d0822ad..5a49f7398 100644 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/scripts/cis_reports/cis_reports.py +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/scripts/cis_reports/cis_reports.py @@ -42,9 +42,9 @@ except Exception: OUTPUT_DIAGRAMS = False -RELEASE_VERSION = "2.8.4" -PYTHON_SDK_VERSION = "2.129.4" -UPDATED_DATE = "July 26, 2024" +RELEASE_VERSION = "2.8.6" +PYTHON_SDK_VERSION = "2.139.0" +UPDATED_DATE = "November 20, 2024" ########################################################################## @@ -1730,7 +1730,8 @@ def __identity_read_tenancy_policies(self): policies_data = oci.pagination.list_call_get_all_results( self.__regions[self.__home_region]['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Policy resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Policy resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data for policy in policies_data: @@ -1826,7 +1827,9 @@ def __os_read_buckets(self): buckets_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Bucket resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Bucket resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id + ).data # Getting Bucket Info for bucket in buckets_data: @@ -1890,7 +1893,9 @@ def __block_volume_read_block_volumes(self): volumes_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Volume resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Volume resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id + ).data # Getting Block Volume inf @@ -1961,7 +1966,8 @@ def __boot_volume_read_boot_volumes(self): boot_volumes_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query BootVolume resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query BootVolume resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data for boot_volume in boot_volumes_data: @@ -2031,7 +2037,8 @@ def __fss_read_fsss(self): fss_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query FileSystem resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query FileSystem resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data for fss in fss_data: @@ -2093,7 +2100,8 @@ def __network_read_network_security_groups_rules(self): nsgs_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query NetworkSecurityGroup resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query NetworkSecurityGroup resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data # Looping through NSGs to to get @@ -2157,7 +2165,8 @@ def __network_read_network_security_lists(self): security_lists_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query SecurityList resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query SecurityList resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data # Looping through Security Lists to to get @@ -2224,7 +2233,8 @@ def __network_read_network_subnets(self): subnets_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Subnet resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Subnet resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data try: @@ -2303,7 +2313,8 @@ def __network_read_drg_attachments(self): drg_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query DrgAttachment resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query DrgAttachment resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2388,7 +2399,8 @@ def __network_read_drgs(self): drg_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Drg resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Drg resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2472,7 +2484,8 @@ def __network_read_fastonnects(self): fastconnects = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query VirtualCircuit resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query VirtualCircuit resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2586,7 +2599,8 @@ def __network_read_ip_sec_connections(self): ip_sec_connections_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query IPSecConnection resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query IPSecConnection resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data for ip_sec in ip_sec_connections_data: @@ -2728,7 +2742,8 @@ def __adb_read_adbs(self): adb_query_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query AutonomousDatabase resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query AutonomousDatabase resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2777,7 +2792,8 @@ def __oic_read_oics(self): oic_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query IntegrationInstance resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query IntegrationInstance resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2854,7 +2870,8 @@ def __oac_read_oacs(self): oac_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query AnalyticsInstance resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query AnalyticsInstance resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data compartments = set() @@ -2924,7 +2941,8 @@ def __events_read_event_rules(self): events_rules_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query EventRule resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query EventRule resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data for event_rule in events_rules_data: @@ -2959,7 +2977,8 @@ def __logging_read_log_groups_and_logs(self): log_groups = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query LogGroup resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query LogGroup resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data # Looping through log groups to get logs @@ -3063,7 +3082,8 @@ def __kms_read_keys(self): keys_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query Key resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query Key resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data vaults_set = set() @@ -3308,7 +3328,8 @@ def __ons_read_subscriptions(self): subs_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query OnsSubscription resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query OnsSubscription resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data debug("\t__ons_read_subscriptions: Recieved " + str(len(subs_data)) + " subscriptions in region " + str(region_key)) for sub in subs_data: @@ -3380,7 +3401,8 @@ def __sch_read_service_connectors(self): service_connectors_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query ServiceConnector resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'") + query="query ServiceConnector resources return allAdditionalFields where compartmentId != '" + self.__managed_paas_compartment_id + "'"), + tenant_id=self.__tenancy.id ).data # Getting Bucket Info @@ -3457,7 +3479,8 @@ def __search_resources_in_root_compartment(self): structured_search_query = oci.resource_search.models.StructuredSearchDetails(query=query_non_compliant) search_results = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, - search_details=structured_search_query + search_details=structured_search_query, + tenant_id=self.__tenancy.id ).data for item in search_results: @@ -3472,7 +3495,8 @@ def __search_resources_in_root_compartment(self): structured_search_all_query = oci.resource_search.models.StructuredSearchDetails(query=query_all_resources) structured_search_all_resources = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, - search_details=structured_search_all_query + search_details=structured_search_all_query, + tenant_id=self.__tenancy.id ).data for item in structured_search_all_resources: @@ -3506,7 +3530,8 @@ def __search_query_resource_type(self, resource_type, search_client): results = oci.pagination.list_call_get_all_results( search_client.search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query=query) + query=query), + tenant_id=self.__tenancy.id ).data return oci.util.to_dict(results) @@ -3598,7 +3623,8 @@ def __certificates_read_certificates(self): certificates_data = oci.pagination.list_call_get_all_results( region_values['search_client'].search_resources, search_details=oci.resource_search.models.StructuredSearchDetails( - query="query certificate resources return allAdditionalFields") + query="query certificate resources return allAdditionalFields"), + tenant_id=self.__tenancy.id ).data cert_compartments = {} debug("\t__certificates_read_certificates: Got Ceritificates from ") @@ -4608,13 +4634,16 @@ def __obp_analyze_tenancy_data(self): elif attachment['network_type'].upper() == 'VIRTUAL_CIRCUIT': # Checking for Provision and BGP enabled Virtual Circuits and that it is associated - for virtual_circuit in self.__network_fastconnects[attachment['drg_id']]: - if attachment['network_id'] == virtual_circuit['id']: - if virtual_circuit['lifecycle_state'].upper() == 'PROVISIONED' and virtual_circuit['bgp_session_state'].upper() == "UP": - # Good VC to increment number of VCs and append the provider name - fast_connect_providers.add(virtual_circuit['provider_name']) - number_of_valid_fast_connect_circuits += 1 - + try: + for virtual_circuit in self.__network_fastconnects[attachment['drg_id']]: + if attachment['network_id'] == virtual_circuit['id']: + if virtual_circuit['lifecycle_state'].upper() == 'PROVISIONED' and virtual_circuit['bgp_session_state'].upper() == "UP": + # Good VC to increment number of VCs and append the provider name + fast_connect_providers.add(virtual_circuit['provider_name']) + number_of_valid_fast_connect_circuits += 1 + except Exception: + debug("__obp_analyze_tenancy_data: Fast Connect Connections check: DRG ID not found " + str(drg_id)) + self.__errors.append({"id" : str(drg_id), "error" : str("__obp_analyze_tenancy_data: Fast Connect Connections check: DRG ID not found")}) try: record = { "drg_id": drg_id, diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/standard.sh b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/standard.sh index 44996f5e0..2a235bfd5 100755 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/standard.sh +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/oci-security-health-check-standard/standard.sh @@ -7,7 +7,7 @@ # # Author: Olaf Heimburger # -VERSION=241011 +VERSION=241206 OS_TYPE=$(uname) ASSESS_DIR=$(dirname $0) @@ -175,12 +175,13 @@ done if [ $IS_ADVANCED -ne 1 ]; then RUN_SHOWOCI=0 RUN_CIS=1 + CIS_DATA_OPT="" else if [ -z "$CIS_DATA_OPT" ]; then CIS_DATA_OPT="--obp --all-resources" fi if [ -z "$SHOWOCI_DATA_OPT" ]; then - SHOWOCI_DATA_OPT="-nsum -a -dsa" + SHOWOCI_DATA_OPT="-ns -a -dsa" fi fi @@ -214,12 +215,14 @@ if [ ! -d ${PYTHON_ENV} ]; then ${PYTHON_CMD} -m venv ${PYTHON_ENV} fi -PIP_OPTS="-q --user --no-warn-script-location" +PIP_OPTS="-q --no-warn-script-location" if [ -d ${PYTHON_ENV} ]; then source ${PYTHON_ENV}/bin/activate - if [ -z "${CLOUD_SHELL_TOOL_SET}" ]; then - ${PYTHON_CMD} -m pip install pip --upgrade ${PIP_OPTS} - fi + PYTHON_CMD=$(which python3) + # if [ -z "${CLOUD_SHELL_TOOL_SET}" ]; then + # ${PYTHON_CMD} -m pip install pip --upgrade ${PIP_OPTS} + # fi + ${PYTHON_CMD} -m pip install pip --upgrade ${PIP_OPTS} fi printf "INFO: Checking for required libraries...\n" @@ -268,37 +271,39 @@ else fi printf "INFO: %s\n" "${INFO_STR}" -CIS_OPTS="-t ${TENANCY} ${CIS_REGION_OPT} ${CIS_DATA_OPT} ${AUTH_OPT}" +CIS_OPTS="-t ${TENANCY} ${CIS_REGION_OPT} ${CIS_DATA_OPT} ${AUTH_OPT} --report-summary-json --report-prefix ${OUTPUT_DIR_NAME}" SHOWOCI_OPTS="-t ${TENANCY} ${SHOWOCI_REGION_OPT} ${AUTH_OPT} ${SHOWOCI_DATA_OPT}" trap "cleanup; echo The script has been canceled; exiting" 1 2 3 6 _W_=$(which script | wc -c) if [ $RUN_CIS -eq 1 ]; then out=$(echo -n ${OUTPUT_DIR} | sed -e 's;\./;;g') + CIS_OPTS="${CIS_OPTS} --report-directory ${out}" if [ ${_W_} -gt 0 ]; then if [ "${OS_TYPE}" == 'Darwin' ]; then - ${SCRIPT_CMD} -q ${out}/assess_cis_report.txt ${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS} --report-summary-json --report-directory ${out} --report-prefix ${OUTPUT_DIR_NAME} + ${SCRIPT_CMD} -q ${out}/assess_cis_report.txt ${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS} else - ${SCRIPT_CMD} -c "${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS} --report-directory ${out} --report-prefix ${OUTPUT_DIR_NAME}" ${out}/assess_cis_report.txt + ${SCRIPT_CMD} -c "${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS}" ${out}/assess_cis_report.txt fi else - ${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS} --report-directory ${out} + ${PYTHON_CMD} ${CIS_SCRIPT} ${CIS_OPTS} fi fi if [ $RUN_SHOWOCI -eq 1 ]; then if [ -z "${BUFFERED}" ]; then export PYTHONUNBUFFERED=TRUE fi + # SHOWOCI_CSV="-csv_nodate -csv ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}" + SHOWOCI_XLSX="-xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}" + SHOWOCI_JSON="-jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json" if [ ${_W_} -gt 0 ]; then if [ "${OS_TYPE}" == 'Darwin' ]; then - echo "${SCRIPT_CMD} -q ${OUTPUT_DIR}/assess_showoci.txt ${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} -jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json -xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}" - ${SCRIPT_CMD} -q ${OUTPUT_DIR}/assess_showoci.txt ${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} -jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json -xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME} + ${SCRIPT_CMD} -q ${OUTPUT_DIR}/assess_showoci.txt ${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} ${SHOWOCI_JSON} ${SHOWOCI_XLSX} ${SHOWOCI_CSV} else - echo "${SCRIPT_CMD} -c "${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} -jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json -xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}" ${OUTPUT_DIR}/assess_showoci.txt" - ${SCRIPT_CMD} -c "${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} -jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json -xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}" ${OUTPUT_DIR}/assess_showoci.txt + ${SCRIPT_CMD} -c "${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} ${SHOWOCI_JSON} ${SHOWOCI_XLSX} ${SHOWOCI_CSV}" ${OUTPUT_DIR}/assess_showoci.txt fi else - ${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} -jf ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME}.json -xlsx_nodate -xlsx ${OUTPUT_DIR}/showoci_${OUTPUT_DIR_NAME} + ${PYTHON_CMD} ${SHOWOCI_SCRIPT} ${SHOWOCI_OPTS} ${SHOWOCI_JSON} ${SHOWOCI_XLSX} ${SHOWOCI_CSV} fi fi DIR_PARENT_OUTPUT="$(dirname ${OUTPUT_DIR})" diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512 b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512 deleted file mode 100644 index 3bf3349cd..000000000 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512 +++ /dev/null @@ -1 +0,0 @@ -b73309d7fae146aa3464ade684f9728c1d7d258bf5d6325f81914417a74c9678111f7bdbdbfcd046d9a30826fcc66af12dd622dc68f8099a5c1ce1ed5fd76a1c oci-security-health-check-standard-241011.zip diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512256 b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512256 deleted file mode 100644 index c742319b3..000000000 --- a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.sha512256 +++ /dev/null @@ -1 +0,0 @@ -df870a5770f37e353d84bbb6e56185e092729636b5fe2525b6659b55300b2353 oci-security-health-check-standard-241011.zip diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip deleted file mode 100644 index df38da373..000000000 Binary files a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241011.zip and /dev/null differ diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512 b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512 new file mode 100644 index 000000000..2d0ca88d9 --- /dev/null +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512 @@ -0,0 +1 @@ +20869ea4d8dc39230092329356e408bccf1bef64c794e27758e5be6b78b91101c4d3779376076dd0c94528eaae3ecaa5ba0c9ed332a9404c1aa9b5f4d4bbfcc9 oci-security-health-check-standard-241206.zip diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512256 b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512256 new file mode 100644 index 000000000..efbd3830b --- /dev/null +++ b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.sha512256 @@ -0,0 +1 @@ +79f33b82702de4f482850e34a91a1c69c435b8e33e6e247387dabc567d295b5f oci-security-health-check-standard-241206.zip diff --git a/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip new file mode 100644 index 000000000..27f2860d5 Binary files /dev/null and b/security/security-design/shared-assets/oci-security-health-check-standard/files/resources/oci-security-health-check-standard-241206.zip differ