[New Content]: Ranking of planners via a percentage score

[nergmada]

New Content Proposal:
Score planners from 0 - 100% to provide people with an informative way to judge holistic planner quality.

• 15% for instructions that provide informative guidance on how to compile planner for Ubuntu or Fedora linux
  • 5% for additional instructions for fedora, mac and Windows systems
• 15% fully fleshed documentation page, detailing planners core contribution, and full feature support table
  • 5% if planner support page is in-line with Planning.wiki standards
• 5% for single executable binary (including shell script for platforms that allows user to go from PDDL to plan with a single command (assuming PDDL correctly formatted, plan is solveable, PDDL is valid for planner etc.)
• 55% IPC planner score

Section:
Reference
Planners

[haz]

Controversial...

Why not have those as defaults, and the actual amounts adjustable by those viewing the page? Maybe I just care about 100% IPC score, or 100% instructions for those that work on Mac, or ...

[nergmada]

A scored system like this is designed to be controversial and force planning developers to consider the more practical side of the software they develop. Particularly with respect. To deployment

At the moment the score idea is still in its early phase but is designed to punish and devalue planners which are not intended for long term support

This lines up with the long term aim of this wiki which is to increase accessibility. A score heavily weighted towards planners that are easier to compile and deploy makes sense

[haz]

Drawn as an ultimatum, it may just have the unintended consequence of pushing people away from the wiki. If you have the leverage, then a hammer sometimes is viable (e.g., forcing planners submitted to the IPC to be open source), but I'm not sure the site is there yet.

If it's at least customizable, then it takes some of the edge off, and perhaps wouldn't put planner devs on the defensive as much. Worst case scenario is folks stop publishing their code at all, given the lack of time to make it "high score ready". Evidence has shown that it's always more useful in the long run to release things, regardless of their state. But still many have reservations at doing so, and I fear "punishing" those that have decided to may not tilt things the right way.

[nergmada]

This feature is being deployed with pull request #60