Skip to content

OBSDOCS-1801: Loki - Adjust Queries for both OTEL and ViaQ#94828

Merged
gabriel-rh merged 1 commit intoopenshift:standalone-logging-docs-mainfrom
theashiot:OBSDOCS-1801
Jul 16, 2025
Merged

OBSDOCS-1801: Loki - Adjust Queries for both OTEL and ViaQ#94828
gabriel-rh merged 1 commit intoopenshift:standalone-logging-docs-mainfrom
theashiot:OBSDOCS-1801

Conversation

@theashiot
Copy link
Copy Markdown
Contributor

@theashiot theashiot commented Jun 16, 2025

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 16, 2025
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jun 16, 2025

@theashiot: This pull request references OBSDOCS-1801 which is a valid jira issue.

Details

In response to this:

Version(s): 6.0+

Issue: https://issues.redhat.com/browse/OBSDOCS-1801

Link to docs preview:

QE review:

  • QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jun 16, 2025
@theashiot theashiot force-pushed the OBSDOCS-1801 branch 4 times, most recently from 62aad81 to d86fbca Compare June 23, 2025 11:43
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 23, 2025
@openshift-ci openshift-ci Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 23, 2025
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 23, 2025
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jun 23, 2025

@theashiot: This pull request references OBSDOCS-1801 which is a valid jira issue.

Details

In response to this:

Version(s): 6.0+

Issue: https://issues.redhat.com/browse/OBSDOCS-1801

Link to docs preview:
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/configuring/configuring-the-log-store.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/custom-logging-alerts.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/default-logging-alerts.html

QE review:

  • QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Copy Markdown
Contributor

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a first look at it. I'll give it a more thorough read tomorrow.

| Tenant type | Valid namespaces for `AlertingRule` CRs
| application |
| audit | `openshift-logging`
| infrastructure | `openshift-/\*`, `kube-/\*`, `default`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to parse well. The \ and / are not meant to be in the output. To me it looks as if we can just use openshift-* and kube-*.

@theashiot
Copy link
Copy Markdown
Contributor Author

Thanks, @xperimental for the review! I've fixed the issues with rendering.

best,
ashwin

Copy link
Copy Markdown
Contributor

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking a while for the second review...

Comment thread modules/configuring-logging-loki-ruler.adoc Outdated
| application |
| audit | `openshift-logging`
| infrastructure | `openshift-/\*`, `kube-/\*`, `default`
| infrastructure | `openshift-\*`, `kube-*`, `default`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| infrastructure | `openshift-\*`, `kube-*`, `default`
| infrastructure | `openshift-*`, `kube-*`, `default`

Copy link
Copy Markdown
Contributor Author

@theashiot theashiot Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "\" is an escape sequence character. This renders as:

openshift-*, kube-*, default

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'm not sure what it is escaping then, because if it's the * we should need one for the second occurrence as well. I'm also not sure if the current preview is up-to-date with the state of the PR.

<1> The namespace where this `AlertingRule` CR is created must have a label matching the LokiStack `spec.rules.namespaceSelector` definition.
<2> The `labels` block must match the LokiStack `spec.rules.selector` definition.
<3> `AlertingRule` CRs for `infrastructure` tenants are only supported in the `openshift-\*`, `kube-\*`, or `default` namespaces.
<3> `AlertingRule` CRs for `infrastructure` tenants are only supported in the `openshift-\*`, `kube-*`, or `default` namespaces.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<3> `AlertingRule` CRs for `infrastructure` tenants are only supported in the `openshift-\*`, `kube-*`, or `default` namespaces.
<3> `AlertingRule` CRs for `infrastructure` tenants are only supported in the `openshift-*`, `kube-*`, or `default` namespaces.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

Comment thread modules/logging-enabling-loki-alerts.adoc Outdated
Comment thread logging_alerts/custom-logging-alerts.adoc
Copy link
Copy Markdown
Contributor

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine for me.

@theashiot
Copy link
Copy Markdown
Contributor Author

Thanks, @xperimental for the review!

@theashiot
Copy link
Copy Markdown
Contributor Author

Hi @anpingli, @kabirbhartiRH, @QiaolingTang, can you please provide a QE review for the changes?

thanks,
ashwin

@theashiot
Copy link
Copy Markdown
Contributor Author

Hi @anpingli, @kabirbhartiRH, @QiaolingTang, can you please provide a QE review for the changes?

thanks,
ashwin

----
<1> Enable Loki alerting and recording rules in your cluster.
<2> Add a custom label that can be added to namespaces where you want to enable the use of logging alerts and metrics.
<2> Specify the selector for the alerting and recording resources if you have deployed many `LokiStack` CRs in a cluster.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove 'if you have deployed many LokiStack CRs in a cluster'. this label is required too when only one lokiStack is deployed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

namespaceSelector: #<4>
matchLabels:
openshift.io/<label_name>: "true" <3>
openshift.io/<label_name>: "true" #<5>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use same example: <label_name>: "true".

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

= Logging collector alerts

In logging 5.8 and later versions, the following alerts are generated by the {clo}. You can view these alerts in the {ocp-product-title} web console.
The following alerts are generated by the {clo}. You can view these alerts in the {ocp-product-title} web console.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CollectorHighErrorRate and CollectorVeryHighErrorRate is for 5.x only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is not included so the content won't appear in the docs.

|Collectors are consuming too much node disk on the host.
|Warning

|`CollectorHigh403ForbiddenResponseRate`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This alert is for 6.3 only now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i revised the the target for this PR to 6.3. I'll create a separate PR for 6.2 and older versions.

@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jul 11, 2025

@theashiot: This pull request references OBSDOCS-1801 which is a valid jira issue.

Details

In response to this:

Version(s): 6.3

Issue: https://issues.redhat.com/browse/OBSDOCS-1801

Link to docs preview:
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/configuring/configuring-the-log-store.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/custom-logging-alerts.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/default-logging-alerts.html

QE review:

  • QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@mburke5678 mburke5678 added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-in-progress Signifies that the peer review team is reviewing this PR peer-review-needed Signifies that the peer review team needs to review this PR labels Jul 11, 2025
@theashiot theashiot force-pushed the OBSDOCS-1801 branch 3 times, most recently from d4df9d9 to e8dd65b Compare July 14, 2025 20:59
Copy link
Copy Markdown
Contributor Author

@theashiot theashiot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @mburke5678 for the wonderful review! i've made the suggested changes.
best,
ashwin

|Collectors are consuming too much node disk on the host.
|Warning

|`CollectorHigh403ForbiddenResponseRate`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i revised the the target for this PR to 6.3. I'll create a separate PR for 6.2 and older versions.

= Logging collector alerts

In logging 5.8 and later versions, the following alerts are generated by the {clo}. You can view these alerts in the {ocp-product-title} web console.
The following alerts are generated by the {clo}. You can view these alerts in the {ocp-product-title} web console.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is not included so the content won't appear in the docs.

Comment thread logging_alerts/custom-logging-alerts.adoc Outdated
Comment thread logging_alerts/custom-logging-alerts.adoc Outdated
= Configuring the ruler

When the LokiStack ruler component is enabled, users can define a group of link:https://grafana.com/docs/loki/latest/query/[LogQL] expressions that trigger logging alerts or recorded metrics.
When the `LokiStack` ruler component is enabled, users can define a group of link:https://grafana.com/docs/loki/latest/query/[LogQL] expressions that trigger logging alerts or recorded metrics.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread modules/loki-alerts.adoc Outdated
Comment thread modules/loki-alerts.adoc Outdated
Comment thread modules/loki-alerts.adoc Outdated
Comment thread modules/loki-alerts.adoc Outdated

|`LokiReadPathHighLoad`
|`The read path is experiencing high load.`
|The read path has high volume of queries causing longer response times.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, reads much better

Comment thread modules/loki-alerts.adoc

|`LokistackSchemaUpgradesRequired`
|`The LokiStack "{{ $labels.stack_name }}" in namespace "<namespace>" is using a storage schema
configuration that does not contain the latest schema version. It is recommended to update the schema configuration to update the schema version to the latest`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, @mburke5678! This is the output text. I believe this should first be fixed upstream first, cc @xperimental

@anpingli
Copy link
Copy Markdown

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2025
@theashiot
Copy link
Copy Markdown
Contributor Author

Thanks, @anpingli!

@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jul 15, 2025

@theashiot: This pull request references OBSDOCS-1801 which is a valid jira issue.

Details

In response to this:

Version(s): 6.3

Issue: https://issues.redhat.com/browse/OBSDOCS-1801

Link to docs preview:
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/configuring/configuring-the-log-store.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/custom-logging-alerts.html
https://94828--ocpdocs-pr.netlify.app/openshift-logging/latest/logging_alerts/default-logging-alerts.html

QE review:

  • QE has approved this change.

Additional information:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 15, 2025
@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Jul 15, 2025

New changes are detected. LGTM label has been removed.

@theashiot
Copy link
Copy Markdown
Contributor Author

@theashiot A few comments. Otherwise LGTM.

I know it is not part of this PR, but in reviewing the preview, I noticed this:
image

Thanks for catching this! I added the required procedure module labels and intro.

@theashiot
Copy link
Copy Markdown
Contributor Author

/merge-review-needed

@theashiot
Copy link
Copy Markdown
Contributor Author

/label merge-review-needed

@openshift-ci openshift-ci Bot added the merge-review-needed Signifies that the merge review team needs to review this PR label Jul 15, 2025
Copy link
Copy Markdown
Contributor

@gabriel-rh gabriel-rh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM - need to fix 2 minor issues

<2> Add a custom label that can be added to namespaces where you want to enable the use of logging alerts and metrics.
<2> Specify the selector for the alerting and recording resources.
<3> Add a custom label that can be added to namespaces where you want to enable the use of logging alerts and metrics.
<4> Specify the namespaces in which the alerting and recording rules are defined for the {loki-op}. If undefined, only the rules defined in the same namespace as the `LokiStack` CR is are used.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR is are used

I would drop the "is" - but only a suggestion for later

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

The Alerting UI is accessible through the *{perspective}* perspective of the {ocp-product-title} web console.

// tag::ADM[]
You can access the *Alerting* user interface (UI) is through the *{perspective}* perspective of the {ocp-product-title} web console.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"is through" - I think the "is" here needs to be removed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

* From the *Developer* perspective, go to *Observe* and go to the *Alerts* tab.
* Select the project that you want to manage alerts for from the *Project:* list.

You can access the *Alerting* user interface (UI) is through the *{perspective}* perspective of the {ocp-product-title} web console.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and again - remove "is"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@gabriel-rh gabriel-rh added merge-review-in-progress Signifies that the merge review team is reviewing this PR and removed merge-review-needed Signifies that the merge review team needs to review this PR labels Jul 16, 2025
@theashiot
Copy link
Copy Markdown
Contributor Author

Thanks, @gabriel-rh for the review! I've made the required changes.

best,

ashwin

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Jul 16, 2025

@theashiot: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@gabriel-rh gabriel-rh merged commit c6a53ad into openshift:standalone-logging-docs-main Jul 16, 2025
2 checks passed
@gabriel-rh
Copy link
Copy Markdown
Contributor

/cherrypick standalone-logging-docs-6.3

@openshift-cherrypick-robot
Copy link
Copy Markdown

@gabriel-rh: new pull request created: #96166

Details

In response to this:

/cherrypick standalone-logging-docs-6.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. merge-review-in-progress Signifies that the merge review team is reviewing this PR peer-review-done Signifies that the peer review team has reviewed this PR size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants