Use the new ocaml-planet package!

[amirmc]

@kayceesrk has been working on an ocaml-planet package, and v0.1.1 is about to go into the OPAM repo. The pkg will be in use on the OCaml Labs webpages (source).

I think we should adjust the code here so that we depend on that pkg instead. KC, would you have the time to make the changes? @Chris00 did the work to get Syndic working on OCaml.org for the community feed so it would be good to get his opinion on this.

[amirmc]

(edited prev comment to point at ocamllabs/ocl-reporter#48 as it's actually awaiting merge)

[kayceesrk]

would you have the time to make the changes

Sure. ocaml-planet reuses most of rss2html from OCaml.org. The changes should be straight-forward.

[Chris00]

Sorry for the delay in replying, I am quite busy ATM.

I think it is good that the planet code is split out of ocaml.org as it can be useful to other people. However, making it a library is more demanding than copying some code from ocaml.org. A few shortcomings I see are:

• A library should not print on stderr to signal errors. This is really bad design.
• A library should be more carefully designed than a single purpose program. Here, the way to cache HTTP requests (please, not in ocamlorg-* files) should be improved. Indeed, this is supposed to be able to be used in contexts where multiple instances of the program using the library would be run concurrently and there is not protection for reading/writing the cache files. For ocaml.org, some rules in the Makefile ensure that all URLs are fetched before compiling the rest of the site. I think a separate cleverer cache library should be designed.
• I do not understand why the generation of OPML files is not part of this library.
• It would be nice for internal interfaces to be documented.
• The files generated by oasis should not be under Git, especially when they are unmodified.

[kayceesrk]

Thanks very much for the comments! I'll work on improving the interface. I realize that the library isn't clean; wanted to quickly put something together for ocaml.io.

[Chris00]

I'd also like to add a bit of context.

It is good that river is based on the cohttp branch of ocaml.orgbecause that it what was intended to become the new feed processor — downloading feeds in parallel is way faster.

The datastructures used in rss2html predates the existence of Syndic and should be updated according to its capabilities — it hasn't been done for lack of time. In particular, the post data type was used to provide a common denominator to the various feed formats. Nowadays, th function Rss2.to_atom exists — and is already used by aggregate — so the representation of posts should probably be Atom.entry (one has to double check that the conversion performed in Rss2.to_atom are compatible with the way the data is extracted from the Atom entries so the current behavior is preserved). One may want to update the Atom.feed authors with the name provided in the planet file so that the aggregation put that name in the Atom.entry source field (this is for a feed used by several authors).

The generation of OPML from a list of Atom feeds is probably better done in Syndic — I will try to write it soon. (To get the names used in the planet file, which may differ from the list of authors returned by the feed, the update of Atom.feed authors is important.)

Finally, river should possibly offer a "standard" HTML output of OPML and feed entries as I feel the basic rendering structure should not be different from one site to the next (it already offers prefix_of_html which is good — except that I do not feel that the name convey what the function does; maybe truncate is better). The HTML would be quite generic and customizable using CSS (and possibly some optional arguments to the rendering functions. Whether the JS toggling should be part of this, I don't know (but I thnk some CSS toggling is possible, one may want to provide that).

[Chris00]

So I put my work where my mouth is and added a few functions to Syndic. Of particular interest, see Rss2.to_atom which can now set the source link of the feed, Omlp1.of_atom generates an OPML list from a list of Atom feeds and Atom.set_main_author make convenient to set the author from (RSS2) feeds that do not declare one. I adapted ocaml.org accordingly, using Atom.entry as the representation of a post. Don't worry, there is still some meat for river tough!

[kayceesrk]

@Chris00 Thanks for the updates! The new updates definitely can make river lighter. I am yet to work on the comments that you've made earlier, as I am busy with other stuff. But I plan to look at it next week. Once I have my edits, I hope you'll be able to review it again.

[Chris00]

On Thu, 2 Apr 2015 08:58:02 -0700, KC Sivaramakrishnan wrote:

@Chris00 Thanks for the updates! The new updates definitely can make river
lighter. I am yet to work on the comments that you've made earlier, as I am
busy with other stuff. But I plan to look at it next week. Once I have my
edits, I hope you'll be able to review it again.

I am busy too, so do not worry you have time! ;-)

[Chris00]

@kayceesrk Where are we on this?