diff --git a/constants/docsSideNav.ts b/constants/docsSideNav.ts index 735d5fcea..dc15a106d 100644 --- a/constants/docsSideNav.ts +++ b/constants/docsSideNav.ts @@ -1138,13 +1138,8 @@ const docsSideNav = [ }, { type: 'doc', - route: '/docs/userguide/python-logs-auto-instrumentation', - label: 'Python Logs Auto-Instrumentation', - }, - { - type: 'doc', - route: '/docs/userguide/collecting_application_logs_otel_sdk_python', - label: 'Using OTel Python SDK', + route: '/docs/logs-management/send-logs/python-logs', + label: 'Python Logs', }, { type: 'doc', @@ -3166,4 +3161,4 @@ const docsSideNav = [ }, ] -export default docsSideNav \ No newline at end of file +export default docsSideNav diff --git a/data/docs/instrumentation/opentelemetry-python.mdx b/data/docs/instrumentation/opentelemetry-python.mdx index 5d437d9cd..6408fec69 100644 --- a/data/docs/instrumentation/opentelemetry-python.mdx +++ b/data/docs/instrumentation/opentelemetry-python.mdx @@ -5,30 +5,9 @@ title: Python OpenTelemetry Instrumentation description: Send events from your Python application to SigNoz hide_table_of_contents: true --- -{/* -import InstrumentationFAQ from '@/components/shared/instrumentation-faq.md' */} This document contains instructions on how to set up OpenTelemetry instrumentation in your Python applications and view your application traces in SigNoz. -{/* This document contains instructions on how to set up OpenTelemetry instrumentation in your Python applications. */} -{/* OpenTelemetry, also known as OTel for short, is an open source observability framework that can help you generate and collect telemetry data - traces, metrics, and logs from your Python application. */} -{/* -Once the telemetry data is collected, you can configure an exporter to send the data to SigNoz. - -There are three major steps to using OpenTelemetry: - -- Instrumenting your Python application with OpenTelemetry -- Configuring exporter to send data to SigNoz -- Validating that configuration to ensure that data is being sent as expected. */} - -{/*
- OpenTelemetry helps to generate and collect telemetry data from your application which is then sent to an observability backend like SigNoz -
OpenTelemetry helps generate and collect telemetry data from Python applications which can then be sent to SigNoz for storage, visualization, and analysis.
-
*/} - - -{/* Let’s understand how to download, install, and run OpenTelemetry in Python. */} - ## Requirements - Python 3.8 or newer @@ -790,50 +769,10 @@ Validate your traces in SigNoz: 2. In SigNoz, open the `Services` tab. Hit the `Refresh` button on the top right corner, and your application should appear in the list of `Applications`. 3. Go to the `Traces` tab, and apply relevant filters to see your application’s traces. -{/* You might see other dummy applications if you’re using SigNoz for the first time. You can remove it by following the docs [here](https://signoz.io/docs/operate/docker-standalone/#remove-the-sample-application). */} - -{/*
- Python Application in the list of services being monitored in SigNoz -
Python Application in the list of services being monitored in SigNoz
-
- */} - -{/* ## Database Instrumentation - -Make sure that the DB client library you are using has the corresponding instrumentation library, and the version of the DB client library is supported by OpenTelemetry. - -### MongoDB - -You can use `opentelemetry-distro` to initialize instrumentation for your MongoDB database calls. You need to ensure that the version of your DB client library is supported by OpenTelemetry. For MongoDB, the instrumentation library is `opentelemetry-instrumentation-pymongo`. - -You can check the supported versions [here](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation). - -### Redis - -You can use `opentelemetry-distro` to initialize instrumentation for your Redis database calls. You need to ensure that the version of your DB client library is supported by OpenTelemetry. For Redis, the instrumentation library is `opentelemetry-instrumentation-redis`. - -You can check the supported versions [here](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation). - -### MySQL - -You can use `opentelemetry-distro` to initialize instrumentation for your MySQL database calls. You need to ensure that the version of your DB client library is supported by OpenTelemetry. For MySQL, we have two instrumentation libraries: - -- opentelemetry-instrumentation-mysql -- opentelemetry-instrumentation-pymysql - -You can check the supported versions [here](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation). - -### Postgres - -You can use `opentelemetry-distro` to initialize instrumentation for your PostgreSQL database calls. You need to ensure that the version of your DB client library is supported by OpenTelemetry. For Postgres, the instrumentation library is `opentelemetry-instrumentation-psycopg2`. - -You can check the supported versions [here](https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation). - - -`psycopg2-binary` is not supported by opentelemetry auto instrumentation libraries as it is not recommended for production use. Please use `psycopg2` to see DB calls also in your trace data in SigNoz +## Send Python application logs and correlate with traces - */} +To send logs from your Python application and correlate them with traces, please follow the instructions mentioned in this [documentation](https://signoz.io/docs/logs-management/send-logs/python-logs). ## Running applications with Gunicorn, uWSGI diff --git a/data/docs/logs-management/send-logs/python-logs.mdx b/data/docs/logs-management/send-logs/python-logs.mdx new file mode 100644 index 000000000..6c9f899b1 --- /dev/null +++ b/data/docs/logs-management/send-logs/python-logs.mdx @@ -0,0 +1,63 @@ +--- +date: 2025-09-01 +title: Sending Python Logs +id: python-logs +hide_table_of_contents: true +--- + +You can send Python logs to SigNoz using Auto instrumentation which is the fast, no code change way to send logs to SigNoz. + +## Setup + +### Step 1: Instrument your application + +Follow the steps in our [Python instrumentation document](https://signoz.io/docs/instrumentation/opentelemetry-python/#send-traces-directly-to-signoz-cloud) to auto-instrument your application. + + +### Step 2: Run your application + +Now, run your application with the following environment variables set: + +```bash +OTEL_SERVICE_NAME= \ +OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest..signoz.cloud:443" \ +OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=" \ +OTEL_EXPORTER_OTLP_PROTOCOL=grpc \ +OTEL_TRACES_EXPORTER=otlp \ +OTEL_LOGS_EXPORTER=otlp \ +OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \ +opentelemetry-instrument +``` + +This attaches an OTLP log handler, injects trace context into each log record. This will allow you to correlate logs with traces. + +- `OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED` when set to `true` will [attach the OTLP log handler](https://opentelemetry.io/docs/zero-code/python/configuration/#logging) to the root logger. That is what sends the logs out. + + +
+Correlated Logs and Traces +
Correlated Logs and Traces
+
+ + +## Python application on Kubernetes + +If your application is running on Kubernetes, you can use the [k8s-infra-chart](https://signoz.io/docs/collection-agents/k8s/k8s-infra/install-k8s-infra/) to get your logs directly to SigNoz. + +The chart has [presets.logsCollection.enabled](https://signoz.io/docs/collection-agents/k8s/k8s-infra/configure-k8s-infra/#3-container-log-collection) property to **true** which adds a `filelogreceiver`. This receiver is configured to read the files where Kubernetes +container runtime writes all containers console output **(/var/log/pods/*/*/*.log)**. + +### Parse attributes + +Once you have the logs in SigNoz, you can parse different attributes from the log body using [Logs Pipelines](https://signoz.io/docs/logs-pipelines/introduction/). +Logs pipelines give you the ability to pre-process logs by providing you with a number of [log processors](https://signoz.io/docs/logs-pipelines/processors/). +Checkout [these guides](https://signoz.io/docs/category/guides/) to learn how you can use logs pipelines for different use-cases. + +## Fine Grained Control + +If you need more control over your logging, you can use the OpenTelemetry Python SDK to manually instrument your application. It provides a handler that can be used to transport logs to any OTLP-compatible backend. + +Here are some references to get you started with manual instrumentation: +- [Configure OTel logs SDK in Python](https://signoz.io/opentelemetry/logging-in-python/) part of implementing OTel in Python applications [series](https://signoz.io/opentelemetry/python-overview/) +- Official OpenTelemetry [SDK example for python](https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/logs) logs +- OpenTelemetry [docs for python instrumentation](https://opentelemetry.io/docs/languages/python/instrumentation/#logs) logs section diff --git a/data/docs/userguide/collecting_application_logs_otel_sdk_python.mdx b/data/docs/userguide/collecting_application_logs_otel_sdk_python.mdx deleted file mode 100644 index 96dbe1046..000000000 --- a/data/docs/userguide/collecting_application_logs_otel_sdk_python.mdx +++ /dev/null @@ -1,67 +0,0 @@ ---- -date: 2024-12-18 -title: Collecting Application Logs Using OTEL Python SDK -id: collecting_application_logs_otel_sdk_python -hide_table_of_contents: true ---- - -You can directly send logs of your application to SigNoz using the Python SDKs provided by OpenTelemetry. [This is an example](https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/logs) of how to -set it up effectively. - - -The default logging level in Python is WARNING. To send all logs to SigNoz, change the default log level to DEBUG. - -```python -import logging -logging.basicConfig(level=logging.DEBUG) -``` - - - - - - -To send logs to SigNoz Cloud, set the following environment variables before running your application: - -1. **OTEL_EXPORTER_OTLP_ENDPOINT**: - Set this environment variable to `https://ingest..signoz.cloud:443` - - - Set the `` to match your SigNoz Cloud [region](https://signoz.io/docs/ingestion/signoz-cloud/overview/#endpoint) - - -2. **OTEL_EXPORTER_OTLP_HEADERS**: - Set this to `signoz-access-token=`, where `` is your [ingestion key](https://signoz.io/docs/ingestion/signoz-cloud/keys/). - -3. **Run Command**: - Use the following command to run your script: - - ```bash - OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest..signoz.cloud:443" OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=" python3 example.py - ``` - - - - - -To send logs to a Self-Hosted SigNoz, set the following environment variables before running your application: - -1. **OTEL_EXPORTER_OTLP_ENDPOINT**: - - Set this environment variable to `http://:4317` - - `IP of SigNoz backend` is the IP of the machine where you installed SigNoz. If you have installed SigNoz on `localhost`, the endpoint will be `http://localhost:4317` for gRPC exporter and `http://localhost:4318` for HTTP exporter. - - - The port numbers are 4317 and 4318 for the gRPC and HTTP exporters respectively. Remember to allow incoming requests to port **4317**/**4318** of machine where SigNoz backend is hosted. - -2. **Run Command**: - Use the following command to run your script: - - ```bash - OTEL_EXPORTER_OTLP_ENDPOINT="http://:4317" python3 example.py - ``` - - - - - diff --git a/data/docs/userguide/python-logs-auto-instrumentation.mdx b/data/docs/userguide/python-logs-auto-instrumentation.mdx deleted file mode 100644 index 65582945d..000000000 --- a/data/docs/userguide/python-logs-auto-instrumentation.mdx +++ /dev/null @@ -1,100 +0,0 @@ ---- -date: 2024-06-06 -title: Python Logs Auto-Instrumentation -id: python-logs-auto-instrumentation ---- - - -## Collecting Python Application Logs Using Auto-Instrumentation - -If you are using python auto-instrumentation for instrumenting your python application you can send logs to SigNoz easily with auto-instrumentation. - -To enable logs auto-instrumentation just add this environment variable - -```bash -OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true -``` - -## Example application -Here is a sample python application. - -1. Create a file named `main.py` and paste the following code: - ```python - from flask import Flask - import logging - logging.basicConfig(level=logging.INFO) - logger = logging.getLogger(__name__) - - app = Flask(__name__) - - @app.route('/') - def hello_world(): - logger.warning("hello world log message") - return 'Hello World' - - if __name__ == '__main__': - app.run() - ``` - -2. Create a virual environment - - ```bash - python -m venv venv - source ./venv/bin/activate - ``` - -3. Install dependencies - - ```bash - pip install opentelemetry-distro - pip install flask requests - pip install opentelemetry-exporter-otlp - ``` - -4. Run the opentelemetry-bootstrap command: - ```bash - opentelemetry-bootstrap -a install - ``` - -5. Run the application - - ```bash - OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true opentelemetry-instrument --traces_exporter none --metrics_exporter none --logs_exporter console python main.py - ``` - -You will be able to see the otel logs on the console once you visit `http://localhost:5000` - -Run the below command to start sending your traces to SigNoz. - - - - - - -```bash -OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \ -OTEL_EXPORTER_OTLP_ENDPOINT=ingest..signoz.cloud:443 \ -OTEL_EXPORTER_OTLP_HEADERS=signoz-ingestion-key= \ -opentelemetry-instrument --traces_exporter otlp --metrics_exporter otlp --logs_exporter otlp python main.py -``` - -- Replace `` with your SigNoz Cloud [ingestion key](https://signoz.io/docs/ingestion/signoz-cloud/keys/) -- Set the `` to match your SigNoz Cloud [region](https://signoz.io/docs/ingestion/signoz-cloud/overview/#endpoint) - - - - - - - -```bash -OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \ -OTEL_EXPORTER_OTLP_ENDPOINT= \ -opentelemetry-instrument --traces_exporter otlp --metrics_exporter otlp --logs_exporter otlp python main.py -``` - -- The value of `OTLP_ENDPOINT` will be you otlp receiver endpoint -- You might need to add `OTEL_EXPORTER_OTLP_INSECURE=true` if your endpoint is not secured. - - - \ No newline at end of file diff --git a/next.config.js b/next.config.js index 9e1eba554..c984e8dc0 100644 --- a/next.config.js +++ b/next.config.js @@ -659,6 +659,16 @@ module.exports = () => { source: "/docs/llm/llamaindex-monitoring/", destination: "/docs/llamaindex-monitoring/", permanent: true + }, + { + source: "/docs/userguide/python-logs-auto-instrumentation/", + destination: "/docs/send-logs/python-logs/", + permanent: true + }, + { + source: "/docs/userguide/collecting_application_logs_otel_sdk_python/", + destination: "/docs/send-logs/python-logs/", + permanent: true } ] },