Merge pull request #234 from alexrudall/faraday

Add streaming with Faraday

Author: alexrudall

.rubocop.yml

@@ -12,6 +12,9 @@ Layout/LineLength:
   Exclude:
     - "**/*.gemspec"
 
+Metrics/AbcSize:
+  Max: 20
+
 Metrics/BlockLength:
   Exclude:
     - "spec/**/*"

CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [4.0.0] - 2023-04-25
+
+### Added
+
+- Add the ability to stream Chat responses from the API! Thanks to everyone who requested this and made suggestions.
+- Added instructions for streaming to the README.
+
+### Changed
+
+- Switch HTTP library from HTTParty to Faraday to allow streaming and future feature and performance improvements.
+- [BREAKING] Endpoints now return JSON rather than HTTParty objects. You will need to update your code to handle this change, changing `JSON.parse(response.body)["key"]` and `response.parsed_response["key"]` to just `response["key"]`.
+
 ## [3.7.0] - 2023-03-25
 
 ### Added

Gemfile.lock

@@ -1,8 +1,9 @@
 PATH
   remote: .
   specs:
-    ruby-openai (3.7.0)
-      httparty (>= 0.18.1)
+    ruby-openai (4.0.0)
+      faraday (>= 1)
+      faraday-multipart (>= 1)
 
 GEM
   remote: https://rubygems.org/
@@ -15,13 +16,15 @@ GEM
       rexml
     diff-lcs (1.5.0)
     dotenv (2.8.1)
+    faraday (2.7.4)
+      faraday-net_http (>= 2.0, < 3.1)
+      ruby2_keywords (>= 0.0.4)
+    faraday-multipart (1.0.4)
+      multipart-post (~> 2)
+    faraday-net_http (3.0.2)
     hashdiff (1.0.1)
-    httparty (0.21.0)
-      mini_mime (>= 1.0.0)
-      multi_xml (>= 0.5.2)
     json (2.6.3)
-    mini_mime (1.1.2)
-    multi_xml (0.6.0)
+    multipart-post (2.3.0)
     parallel (1.22.1)
     parser (3.2.2.0)
       ast (~> 2.4.1)
@@ -56,6 +59,7 @@ GEM
     rubocop-ast (1.28.0)
       parser (>= 3.2.1.0)
     ruby-progressbar (1.13.0)
+    ruby2_keywords (0.0.5)
     unicode-display_width (2.4.2)
     vcr (6.1.0)
     webmock (3.18.1)

README.md

@@ -6,7 +6,7 @@
 
 Use the [OpenAI API](https://openai.com/blog/openai-api/) with Ruby! 🤖❤️
 
-Generate text with ChatGPT, transcribe and translate audio with Whisper, or create images with DALL·E...
+Stream text with GPT-4, transcribe and translate audio with Whisper, or create images with DALL·E...
 
 [Ruby AI Builders Discord](https://discord.gg/k4Uc224xVD)
 
@@ -34,10 +34,6 @@ and require with:
 require "openai"
 ```
 
-## Upgrading
-
-The `::Ruby::OpenAI` module has been removed and all classes have been moved under the top level `::OpenAI` module. To upgrade, change `require 'ruby/openai'` to `require 'openai'` and change all references to `Ruby::OpenAI` to `OpenAI`.
-
 ## Usage
 
 - Get your API key from [https://beta.openai.com/account/api-keys](https://beta.openai.com/account/api-keys)
@@ -57,8 +53,8 @@ For a more robust setup, you can configure the gem with your API keys, for examp
 
 ```ruby
 OpenAI.configure do |config|
-    config.access_token = ENV.fetch('OPENAI_ACCESS_TOKEN')
-    config.organization_id = ENV.fetch('OPENAI_ORGANIZATION_ID') # Optional.
+    config.access_token = ENV.fetch("OPENAI_ACCESS_TOKEN")
+    config.organization_id = ENV.fetch("OPENAI_ORGANIZATION_ID") # Optional.
 end
 ```
 
@@ -70,7 +66,7 @@ client = OpenAI::Client.new
 
 #### Custom timeout or base URI
 
-The default timeout for any OpenAI request is 120 seconds. You can change that passing the `request_timeout` when initializing the client. You can also change the base URI used for all requests, eg. to use observability tools like [Helicone](https://docs.helicone.ai/quickstart/integrate-in-one-line-of-code):
+The default timeout for any request using this library is 120 seconds. You can change that by passing a number of seconds to the `request_timeout` when initializing the client. You can also change the base URI used for all requests, eg. to use observability tools like [Helicone](https://docs.helicone.ai/quickstart/integrate-in-one-line-of-code):
 
 ```ruby
 client = OpenAI::Client.new(
@@ -130,6 +126,23 @@ puts response.dig("choices", 0, "message", "content")
 # => "Hello! How may I assist you today?"
 ```
 
+### Streaming ChatGPT
+
+You can stream from the API in realtime, which can be much faster and used to create a more engaging user experience. Pass a [Proc](https://ruby-doc.org/core-2.6/Proc.html) to the `stream` parameter to receive the stream of text chunks as they are generated. Each time one or more chunks is received, the Proc will be called once with each chunk, parsed as a Hash. If OpenAI returns an error, `ruby-openai` will pass that to your proc as a Hash.
+
+```ruby
+client.chat(
+    parameters: {
+        model: "gpt-3.5-turbo", # Required.
+        messages: [{ role: "user", content: "Describe a character called Anna!"}], # Required.
+        temperature: 0.7,
+        stream: proc do |chunk, _bytesize|
+            print chunk.dig("choices", 0, "delta", "content")
+        end
+    })
+# => "Anna is a young woman in her mid-twenties, with wavy chestnut hair that falls to her shoulders..."
+```
+
 ### Completions
 
 Hit the OpenAI API for a completion using other GPT-3 models:
@@ -188,9 +201,9 @@ and pass the path to `client.files.upload` to upload it to OpenAI, and then inte
 ```ruby
 client.files.upload(parameters: { file: "path/to/sentiment.jsonl", purpose: "fine-tune" })
 client.files.list
-client.files.retrieve(id: 123)
-client.files.content(id: 123)
-client.files.delete(id: 123)
+client.files.retrieve(id: "file-123")
+client.files.content(id: "file-123")
+client.files.delete(id: "file-123")
 ```
 
 ### Fine-tunes
@@ -208,9 +221,9 @@ You can then use this file ID to create a fine-tune model:
 response = client.finetunes.create(
     parameters: {
     training_file: file_id,
-    model: "text-ada-001"
+    model: "ada"
 })
-fine_tune_id = JSON.parse(response.body)["id"]
+fine_tune_id = response["id"]
 ```
 
 That will give you the fine-tune ID. If you made a mistake you can cancel the fine-tune model before it is processed:
@@ -224,7 +237,7 @@ You may need to wait a short time for processing to complete. Once processed, yo
 ```ruby
 client.finetunes.list
 response = client.finetunes.retrieve(id: fine_tune_id)
-fine_tuned_model = JSON.parse(response.body)["fine_tuned_model"]
+fine_tuned_model = response["fine_tuned_model"]
 ```
 
 This fine-tuned model name can then be used in completions:
@@ -236,7 +249,7 @@ response = client.completions(
         prompt: "I love Mondays!"
     }
 )
-JSON.parse(response.body)["choices"].map { |c| c["text"] }
+response.dig("choices", 0, "text")
 ```
 
 You can delete the fine-tuned model when you are done with it:
@@ -305,9 +318,9 @@ The translations API takes as input the audio file in any of the supported langu
 response = client.translate(
     parameters: {
         model: "whisper-1",
-        file: File.open('path_to_file', 'rb'),
+        file: File.open("path_to_file", "rb"),
     })
-puts response.parsed_response['text']
+puts response["text"]
 # => "Translation of the text"
 ```
 
@@ -319,9 +332,9 @@ The transcriptions API takes as input the audio file you want to transcribe and
 response = client.transcribe(
     parameters: {
         model: "whisper-1",
-        file: File.open('path_to_file', 'rb'),
+        file: File.open("path_to_file", "rb"),
     })
-puts response.parsed_response['text']
+puts response["text"]
 # => "Transcription of the text"
 ```
 

lib/openai.rb

@@ -1,5 +1,7 @@
-require "httparty"
+require "faraday"
+require "faraday/multipart"
 
+require_relative "openai/http"
 require_relative "openai/client"
 require_relative "openai/files"
 require_relative "openai/finetunes"

lib/openai/client.rb

@@ -1,5 +1,7 @@
 module OpenAI
   class Client
+    extend OpenAI::HTTP
+
     def initialize(access_token: nil, organization_id: nil, uri_base: nil, request_timeout: nil)
       OpenAI.configuration.access_token = access_token if access_token
       OpenAI.configuration.organization_id = organization_id if organization_id
@@ -50,55 +52,5 @@ def transcribe(parameters: {})
     def translate(parameters: {})
       OpenAI::Client.multipart_post(path: "/audio/translations", parameters: parameters)
     end
-
-    def self.get(path:)
-      HTTParty.get(
-        uri(path: path),
-        headers: headers,
-        timeout: request_timeout
-      )
-    end
-
-    def self.json_post(path:, parameters:)
-      HTTParty.post(
-        uri(path: path),
-        headers: headers,
-        body: parameters&.to_json,
-        timeout: request_timeout
-      )
-    end
-
-    def self.multipart_post(path:, parameters: nil)
-      HTTParty.post(
-        uri(path: path),
-        headers: headers.merge({ "Content-Type" => "multipart/form-data" }),
-        body: parameters,
-        timeout: request_timeout
-      )
-    end
-
-    def self.delete(path:)
-      HTTParty.delete(
-        uri(path: path),
-        headers: headers,
-        timeout: request_timeout
-      )
-    end
-
-    private_class_method def self.uri(path:)
-      OpenAI.configuration.uri_base + OpenAI.configuration.api_version + path
-    end
-
-    private_class_method def self.headers
-      {
-        "Content-Type" => "application/json",
-        "Authorization" => "Bearer #{OpenAI.configuration.access_token}",
-        "OpenAI-Organization" => OpenAI.configuration.organization_id
-      }
-    end
-
-    private_class_method def self.request_timeout
-      OpenAI.configuration.request_timeout
-    end
   end
 end

lib/openai/http.rb

@@ -0,0 +1,93 @@
+module OpenAI
+  module HTTP
+    def get(path:)
+      to_json(conn.get(uri(path: path)) do |req|
+        req.headers = headers
+      end&.body)
+    end
+
+    def json_post(path:, parameters:)
+      to_json(conn.post(uri(path: path)) do |req|
+        if parameters[:stream].is_a?(Proc)
+          req.options.on_data = to_json_stream(user_proc: parameters[:stream])
+          parameters[:stream] = true # Necessary to tell OpenAI to stream.
+        end
+
+        req.headers = headers
+        req.body = parameters.to_json
+      end&.body)
+    end
+
+    def multipart_post(path:, parameters: nil)
+      to_json(conn(multipart: true).post(uri(path: path)) do |req|
+        req.headers = headers.merge({ "Content-Type" => "multipart/form-data" })
+        req.body = multipart_parameters(parameters)
+      end&.body)
+    end
+
+    def delete(path:)
+      to_json(conn.delete(uri(path: path)) do |req|
+        req.headers = headers
+      end&.body)
+    end
+
+    private
+
+    def to_json(string)
+      return unless string
+
+      JSON.parse(string)
+    rescue JSON::ParserError
+      # Convert a multiline string of JSON objects to a JSON array.
+      JSON.parse(string.gsub("}\n{", "},{").prepend("[").concat("]"))
+    end
+
+    # Given a proc, returns an outer proc that can be used to iterate over a JSON stream of chunks.
+    # For each chunk, the inner user_proc is called giving it the JSON object. The JSON object could
+    # be a data object or an error object as described in the OpenAI API documentation.
+    #
+    # If the JSON object for a given data or error message is invalid, it is ignored.
+    #
+    # @param user_proc [Proc] The inner proc to call for each JSON object in the chunk.
+    # @return [Proc] An outer proc that iterates over a raw stream, converting it to JSON.
+    def to_json_stream(user_proc:)
+      proc do |chunk, _|
+        chunk.scan(/(?:data|error): (\{.*\})/i).flatten.each do |data|
+          user_proc.call(JSON.parse(data))
+        rescue JSON::ParserError
+          # Ignore invalid JSON.
+        end
+      end
+    end
+
+    def conn(multipart: false)
+      Faraday.new do |f|
+        f.options[:timeout] = OpenAI.configuration.request_timeout
+        f.request(:multipart) if multipart
+      end
+    end
+
+    def uri(path:)
+      OpenAI.configuration.uri_base + OpenAI.configuration.api_version + path
+    end
+
+    def headers
+      {
+        "Content-Type" => "application/json",
+        "Authorization" => "Bearer #{OpenAI.configuration.access_token}",
+        "OpenAI-Organization" => OpenAI.configuration.organization_id
+      }
+    end
+
+    def multipart_parameters(parameters)
+      parameters&.transform_values do |value|
+        next value unless value.is_a?(File)
+
+        # Doesn't seem like OpenAI need mime_type yet, so not worth
+        # the library to figure this out. Hence the empty string
+        # as the second argument.
+        Faraday::UploadIO.new(value, "", value.path)
+      end
+    end
+  end
+end

lib/openai/version.rb

@@ -1,3 +1,3 @@
 module OpenAI
-  VERSION = "3.7.0".freeze
+  VERSION = "4.0.0".freeze
 end

ruby-openai.gemspec

@@ -25,7 +25,6 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
-  spec.add_dependency "httparty", ">= 0.18.1"
-
-  spec.post_install_message = "Note if upgrading: The `::Ruby::OpenAI` module has been removed and all classes have been moved under the top level `::OpenAI` module. To upgrade, change `require 'ruby/openai'` to `require 'openai'` and change all references to `Ruby::OpenAI` to `OpenAI`."
+  spec.add_dependency "faraday", ">= 1"
+  spec.add_dependency "faraday-multipart", ">= 1"
 end