Do nonblocking socket read before select for packet receive#104
Open
Do nonblocking socket read before select for packet receive#104
Conversation
59edb0f to
d23a669
Compare
Author
|
The original commit that I had on the PR, b6cda35, failed the Travis run for 1.8.7. I hadn't remembered that the non-blocking APIs that the commit was using did not exist until 1.9. In a new commit, d23a669, I added some logic which allows for the non-blocking APIs to only be used if available. The tests appear to be passing across Ruby versions and continue to no longer trigger the indefinite hang on |
jbarlow-mcafee
added a commit
to opendxl-community/opendxl-client-ruby
that referenced
this pull request
Oct 18, 2018
This commit changes the ruby-mqtt dependency to be pinned to the 0.5.0 version instead of a branched version. An exact pin was used to guard against potential breaking changes in a newer version of the upstream project (specifically around client connection handling). The previous branched version included a change which avoids frequent hangs when running under JRuby -- see njh/ruby-mqtt#104. By reverting to the upstream release, the JRuby hangs would be reintroduced for now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've periodically run into some cases where the
receive_packetmethod hangs indefinitely on anIO.selecteven though the underlying network socket has actually received some bytes from the network for incoming MQTT packets. I've only been able to consistently reproduce this when running with JRuby 9k for some reason but I believe the underlying condition affects MRI Ruby as well.Specifically, I believe the condition that I'm running into is the same as the buffering issue documented for
OpenSSL::SSL::SSLSocketandIO.selectcalls at http://ruby-doc.org/core-2.2.0/IO.html#method-c-select.I experimented with changing the logic in
receive_packetto use a combination of aread_nonblockcall (reading only a single byte at most) followed by anIO.selectcall in the event that no data is available to be read and anIO.WaitReadable-derived exception is thrown. This PR shows the approach that I used. It does make the logic a bit more messy in that for cases that the first byte in the packet is read beforeMQTT:Packet.read()is called, it passes along the byte so that thePacket.read()method can use it.With the approach on this PR, I'm no longer able to reproduce the indefinite
IO.select()hangs, on MRI Ruby or JRuby. The read throughput also seems like it might be a bit better with a heavy amount of publish packets coming into the socket since available data seems to be received much more quickly.Thanks for your consideration for this PR and thanks much for all of the great work that you've put into this library!