Things to do better next time

[asmeurer]

In case anyone wants to do this again and submit a paper to PeerJ (@certik, you said you might want to do this for SymEngine), here are some suggestions:

• Make the metadata in the authors.json match the metadata in PeerJ (department, institution, etc.), so that you can assemble the affiliation exactly the same as in PeerJ (they require them to match exactly).
• Make all authors sign up for PeerJ from the start. Author signoff should include having accepted the PeerJ invitation, since PeerJ will not publish the paper if they haven't done this.
• Make sure all authors know that if they change their metadata in PeerJ they need to change it in the paper too.
• Generate the Makefile automatically somehow (with proper dependencies).
• Along with that, automatically generate a zip of tex files to be uploaded.

I'll update this with more things if I think of them. In all the process went pretty well.

I think the process of pushing to a shared branch works pretty well for multiple collaborators. It's about as good as you can get while keeping git tracking (the only thing better is Google Docs, but then you lose attribution and commit messages). But you have to make sure everyone knows to constantly pull and push their changes.

PeerJ is pretty nice. There are some bugs, but the staff has been helpful, and the online instructions are extremely helpful.

[moorepants]

Here are some other things that I think should be done better next time (not related to PeerJ):

• I think our authorship criteria was generally a good idea but was used a bit rigidly in some cases, e.g. we asked some authors to make a commit to SymPy to frivolously meet this. I don't think that the only only way to measure "Substantial contributions to the conception or design of the work; or the
  acquisition, analysis, or interpretation of data for the work;" is a commit to the core SymPy code repository.
• I think we should consider having the author of all SymPy papers to simply be the "SymPy Development Team", especially a paper like this, and then we list all people who have commits to all of our repositories and list separately those people who actually wrote the paper. This allows anyone to write stuff about SymPy under our banner but would require the attribution of everyone that makes this software and community happen. (update 2016-12-14, interesting post by Matt Turk that is relevant: https://medium.com/@matthewturk/citations-credit-and-the-yt-method-paper-1034e4080eb#.ouz3z41rj)
• I would like to see an upfront statement about what roles co-authors are expected to play. In this case, Aaron and Anthony had a vision and asked for co-authorship from others, but I'm not sure everyone asked to be a co-author had the exact same vision. For example: Do co-authors have to write anything in the paper? Do co-authors have say in what goes in the paper? Do they have votes on decisions or is it tacit consensus like how SymPy is developed? Do they have to contribute to writing any code for SymPy? In the other papers I've been co-author on it has always been a smaller group and each author played a substantial role in writing, reviewing, and agreeing on the content and message of the paper. This is the first paper that I didn't really feel that way and have some some misgivings about being an author. I'm not quite sure how to manage this with such a large group but maybe there are some ideas we can build on.
  
  I would not have misgivings here if I was asked something like this originally: "We'd like you to be a co-author to our paper. Can you write a ~1/4 of a page about X, which we will include in our paper? Also, can you review the whole paper once drafted and give us feedback that we will use at our discretion?" That makes it much clearer what the co-author is agreeing to do and what the bounds of their contribution are. It's different than the push/pull required to shape something as organic as the SymPy codebase, which I'm not sure works as well for a paper with a specific theme and deadline.
• If we don't make the author "SymPy Development Team", then the author order should be agreed upon by all authors. It seems like we ordered authors in terms of # of commits to SymPy core but made some exceptions. Author order means different things in different fields and in some cases plays a significant role in whether people keep their jobs and get their raises. We can't satisfy everyone, but the author order should be explicitly agreed upon.