Common llm response processing

[BillDowney]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.14.2

Cloud or Self Hosted

Self Hosted (Source)

Steps to reproduce

This change causes OpenAI-API-compatible interfaces to return all errors in the [data:] block without Spaces

✔️ Expected Behavior

Return the json data behind the correct data

❌ Actual Behavior

data json formatting error

[dosubot]

The issue arises from the change in how the prefix "data:" is removed from the string. The original code chunk.strip().lstrip("data:").lstrip() removes all leading characters that are in the set of "d", "a", "t", "a", and ":", which can inadvertently strip additional characters if they appear at the start of the string. The modified code chunk.strip().removeprefix("data:") only removes the exact prefix "data:". If the chunk does not start exactly with "data:", the prefix will not be removed correctly, potentially leaving the JSON data malformed and causing a json.JSONDecodeError when attempting to parse the JSON ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}