Move stats pulling over to CHS

[vreelb]

CHS now keeps track of +/-, so stats no longer need to be pulled from CHN. This will simplify things in the long run, as rosters are pulled from CHS already anyhow.

Unfortunately the parser needs to be rewritten for the new CHS stats pages in order to take advantage of it, but that shouldn't be too difficult.

[vreelb]

This also means that women's teams will finally have +/- as well!

[hamilr2]

👍

[hamilr2]

Unless you've already started on this, I'd like to discuss our scraping techniques. While I think jQuery is an easy and logical way to parse HTML, I'm not sure it entirely makes sense to scrape "client side".

Regardless of what we end up doing, I'd also like to head in a direction where our scraping functions are decoupled from functions that digest/ingest the result. Ideally, we'd have a bunch of different scraping functions that take a couple params and return JSON or Array Objects.

Worth discussing, though. Wednesday at 9ET/6PT?

[csm10495]

Someone at RCOS talked about this thing called kimono. It turns a webpage into a rest api. In playing with it for like 3 minutes, here is an example of a JSON for RPI's stats. (it could be made better if we put more effort)

Example: http://pastebin.com/NiniURms

[hamilr2]

Okay, that's a really cool tool. The pricing/request restrictions are RIDICULOUSLY open for free accounts.

I was having trouble getting it to do logical things with some irregular tabular pages like the CHS box scores and live box scores... Part of the problem is they have a lot of ill-defined header rows, and rows with varying types of data. Will probably need to do those via some other method, or perhaps have another layer between scraping and digestion to help normalize the returned data out of Kimono.

I can definitely see it being useful for standardized tabular data and other non-hockey tasks.

It also would satisfy the separation of scraper and digestion/ingestion quite nicely!

[vreelb]

Semi-closed in de8a6b9.

Importing now fully uses CHS, updating should be moved over to use CHS as well.