Added how-to find mode value in SArray #39
Conversation
|
@guy4261 is it worth pointing out that this could be done faster (albeit sacrificing some accuracy) using sa.sketch_summary().frequent_items()? There are some caveats there, though: won't work for very low-frequency mode values, and won't necessarily give the correct result if there are multiple likely candidates. |
| return sf2[sf2["count"] == max_count]["value"] | ||
|
|
||
| # Create an SArray with two modes (most-common elements: 2 and 3) | ||
| # sa = gl.SArray([1, 2, 2, 3, 3]) |
There was a problem hiding this comment.
Why is this commented-out? Should we uncomment it to make it clear that it's code you can run?
|
@znation I followed your two comments, made it runnable (although not printing, like some of the examples) and added the sketch usage example. Also added a warning about handling empty SArrays. Please review, thanks! |
| raise ValueError("Can't find mode(s) in empty SArray") | ||
|
|
||
| frequent_items_sketch = sa.sketch_summary().frequent_items() | ||
| modes_sketch = [k for (k, v) in frequent_items_sketch.iteritems() |
There was a problem hiding this comment.
This may be slow if the list of frequent items is long (and there is not really an upper bound to how long it can be). Here it calls itervalues for each item of the dict and max on that. For perf this should be refactored to use a single call to max(itervalues) and a comparison to that stored value.
|
@guy4261 Thanks! One minor comment for a perf improvement, otherwise looks good to me. |
Finally added how to find a mode value to the how-to repository
http://forum.dato.com/discussion/1785/finding-the-mode-of-sarray?#