feat: implement v1/chat/completions endpoint as stream consumption

[LeadcodeDev]

Hello (again) 👋

For the same project that I have to make my first contribution to this project, I allow myself to submit a second to support the streaming on the endpoint v1/chat/completions allowing at the same time to respond to the request expressed here #167.

Looking forward to exchanging if necessary.

Have a nice day !

[dongri]

@LeadcodeDev Thank you for adding the stream and for the pull request!

OPENAI_API_KEY=sk-xxxx cargo run --package openai-api-rs --example chat_completion_stream
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.78s
     Running `target/debug/examples/chat_completion_stream`
Content: ""
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)
Failed to parse response: Error("trailing characters", line: 3, column: 1)

This kind of error was returned. Could you look into it?

[LeadcodeDev]

Hello 👋

This mistake is not actually one.
When we receive chunks via a stream, we receive only one piece of our final response by iteration, so the whole of your payload JSON is not completed, we end up with a JSON "truncate", that's why here.

I will withdraw the postponement of this error because we do not wish to treat it but rather wait for the next iteration to aggregate the rest of our response.

[dongri]

OPENAI_API_KEY=sk-xxx cargo run --package openai-api-rs --example chat_completion_stream
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.34s
     Running `target/debug/examples/chat_completion_stream`
Content: ""
Done

Was it expected that the "Content" section was empty and nothing was displayed?

[LeadcodeDev]

In the Content you shoud have streamed response "chunk per chunk".

I have try with Gemini and custom URL.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = env::var("OPENAI_API_KEY").unwrap().to_string();
    let mut client = OpenAIClient::builder()
        .with_api_key(api_key)
        .with_endpoint("https://generativelanguage.googleapis.com/v1beta/openai/")
        .build()?;

    let req = ChatCompletionStreamRequest::new(
        "gemini-2.5-flash".to_string(),
        vec![chat_completion::ChatCompletionMessage {
            role: chat_completion::MessageRole::user,
            content: chat_completion::Content::Text(String::from("What is bitcoin?")),
            name: None,
            tool_calls: None,
            tool_call_id: None,
        }],
    );

    let mut result = client.chat_completion_stream(req).await?;
    while let Some(response) = result.next().await {
        match response.clone() {
            ChatCompletionStreamResponse::ToolCall(toolcalls) => {
                println!("Tool Call: {:?}", toolcalls);
            }
            ChatCompletionStreamResponse::Content(content) => {
                println!("Content: {:?}", content);
            }
            ChatCompletionStreamResponse::Done => {
                println!("Done");
            }
        }
    }

    Ok(())
}

warning: `openai-api-rs` (example "chat_completion_stream") generated 1 warning (run `cargo fix --example "chat_completion_stream"` to apply 1 suggestion)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.06s
     Running `target/debug/examples/chat_completion_stream`
Content: "Bitcoin is a **decentralized digital currency**, often referred to as a **cryptocurrency**. It was created in 2009 by an anonymous entity known as Satoshi Nakamoto.\n\nHere's a breakdown of what that means and"
Content: " how it works:\n\n1.  **Digital Currency:**\n    *   Unlike traditional physical money (like dollar bills or coins), Bitcoin exists purely as digital data. You can't hold a Bitcoin in your hand.\n    *   It"
Content: "'s stored and exchanged electronically over the internet.\n\n2.  **Decentralized:**\n    *   This is one of Bitcoin's most defining characteristics. It means there is no central authority, like a bank, government, or single company"
Content: ", that controls it.\n    *   Instead, it's run by a vast, distributed network of computers (nodes) around the world. This makes it resistant to censorship, single points of failure, and government control.\n    *   Transactions"
Content: " are made directly between users (peer-to-peer) without an intermediary.\n\n3.  **Cryptocurrency:**\n    *   Bitcoin uses **cryptography** (advanced encryption techniques) to secure transactions, verify the transfer of assets"
Content: ", and control the creation of new units.\n    *   This cryptography ensures the integrity of the network and the security of users' funds.\n\n4.  **Blockchain Technology:**\n    *   Bitcoin is built on a technology called the **"
Content: "blockchain**.\n    *   **What it is:** The blockchain is a public, distributed ledger that records all Bitcoin transactions. Imagine it as a continuously growing list of records, called \"blocks,\" which are linked together using cryptography.\n    *   "
Content: "**How it works:** When a transaction occurs, it's grouped with other transactions into a \"block.\" This block is then verified by network participants (miners) and added to the end of the existing chain of blocks.\n    *"
Content: "   **Key properties:**\n        *   **Immutability:** Once a block is added, it's extremely difficult to change or remove, making the transaction history secure and tamper-proof.\n        *   **Transparency:** Every"
Content: " transaction ever made on the Bitcoin network is publicly visible on the blockchain (though the identities of the participants are pseudonymous, using wallet addresses).\n\n5.  **Mining:**\n    *   New Bitcoins are introduced into the system through a process called"
Content: " **mining**.\n    *   **How it works:** Miners use powerful computers to solve complex computational puzzles. The first miner to solve the puzzle gets to add the next block of verified transactions to the blockchain and is rewarded with newly created Bitcoins ("
Content: "the \"block reward\") and transaction fees.\n    *   **Purpose:** Mining not only creates new Bitcoin but also secures the network by verifying and adding transactions to the blockchain.\n\n6.  **Limited Supply:**\n    *   Unlike"
Content: " traditional fiat currencies which can be printed indefinitely, Bitcoin has a finite supply. Only **21 million Bitcoins** will ever be created.\n    *   This scarcity is often compared to precious metals like gold and is a key factor in its value"
Content: " proposition.\n\n**In summary:**\n\nBitcoin is a revolutionary form of digital money that operates independently of central banks and governments. It uses advanced cryptography and a decentralized network (blockchain) to enable secure, transparent, and peer-to-peer transactions."
Content: " Its limited supply and decentralized nature are often cited as its most significant advantages, leading to its growing adoption as both a payment method and a store of value."
Done

I haven't chatgpt key to try like you 😢

EDIT : Could you try with another models to see if the problem exists ?

[dongri]

I found the cause of the error with the OpenAI API.

Received chunk: data: {"id":"chatcmpl-CQlCJiHP52aWhiirTMeEk8SkZfTn6","object":"chat.completion.chunk","created":1760493991,"model":"gpt-4o-2024-08-06","service_tier":"default","system_fingerprint":"fp_cbf1785567","choices":[{"index":0,"delta":{"content":" popular"},"logprobs":null,"finish_reason":null}],"obfuscation":"YhS7A"}

data: {"id":"chatcmpl-CQlCJiHP52aWhiirTMeEk8SkZfTn6","object":"chat.completion.chunk","created":1760493991,"model":"gpt-4o-2024-08-06","service_tier":"default","system_fingerprint":"fp_cbf1785567","choices":[{"index":0,"delta":{"content":" but"},"logprobs":null,"finish_reason":null}],"obfuscation":"eLeR83E5F"}

There was multiple lines of data like this.

Since I've identified the cause, I'll go ahead and merge this PR for now, and then make the fix on my end.

[dongri]

#187

[LeadcodeDev]

I thank you very much for the acceptance of my PR (although not 100% functional you have to believe...) as well as for the patch made afterwards.

Looking forward to contributing