Today we’re starting something new at JOSS: we’re collaborating with American Astronomical Society (AAS) Journals to offer software reviews for some of the papers submitted to their journals. As part of this process, AAS Publishing will make a small contribution to our parent organization NumFOCUS to support the running costs of JOSS. We’re excited about raising the standard of research software in astronomy and astrophysics, and want to use this blog post as an opportunity to give a little background on the collaboration and how we plan to operate.

Ever since I announced JOSS back in May of 2016, I’ve always been clear that the primary purpose of a JOSS paper is to enable citation credit to be given to authors of research software. Raising the bar on the expected quality of research software has always been a strong motivation for the journal too.

After 30 months, with over 500 JOSS submissions (448 published at the time of writing) by more than 400 amazing volunteers, I think it’s safe to say it’s working¹. One of my favorite things about working on JOSS is watching authors, reviewers, and editors all working together to improve a piece of software. Some of my favorite comments from the past couple of years:

Reviewing for @JOSS_TheOJ and #JOSE_theOJ (of the Open Journals: https://github.com/openjournals ) is an exercise in restoration of faith in the “scientific process”. Both times it has felt like I’m doing something worthwhile through a collaborative conversation with the author · @drvinceknight

@bovee Done! Thanks for all your feedback and for helping with this submission. I never thought publishing something could be so enjoyable · JOSS Review

Awesome, thanks everyone, the review process really made this package better! · JOSS Review

Some of you might know that my background is in the astronomical sciences, and over the past couple of years I’ve been delighted to see that the AAS has been working to open their journals to ‘software paper’ submissions https://journals.aas.org/policy-statement-on-software/. Take a moment to read this policy – it’s excellent. In many ways, AAS journals are among the most progressive of society-run journals out there, and have demonstrated a commitment to authors of research software with this policy. There’s a problem though: even though software papers are possible in many AAS publications, the AAS Journals don’t instruct their referees to review the software itself; there’s nothing like the formal JOSS review, and there’s no requirement for an open source license.

In discussions with the AAS team over the past year, we realized there was an opportunity to work together to improve the quality of the software associated with their submissions by giving authors of AAS papers a chance to publish with JOSS too.

How this works

Under this new collaboration, authors will submit their paper to one of the AAS journals as usual (https://journals.aas.org), if their submission includes a substantial software component to the paper, they can choose to submit a JOSS paper to accompany their AAS submission.

If they decide to follow this route, the authors will prepare and submit a JOSS paper as usual at http://joss.theoj.org and the software will go through the standard JOSS editorial and review process, in the open on GitHub https://github.com/openjournals/joss-reviews. As part of this review, it will be made transparent to all parties involved that the JOSS paper is associated with a submission to a AAS publication.

We’ll ask the reviewers to acknowledge that this is happening and if they’re not comfortable with this arrangement, then we will find an alternative reviewer.

Upon successful peer review of the AAS journal article and the JOSS submission, each paper will cite and link to the other.

The income

JOSS is a volunteer, community-run journal and we try to keep our running costs as low as possible. They’re not zero, however, and we’re always making small infrastructure changes to our toolchain that require real money to support development costs. We recognize, though, that adding money into any volunteer project like JOSS is not without risk and so we’re taking a few proactive steps to make the process as transparent as possible:

• We’re keeping an open record showing the papers we’ve jointly reviewed with AAS.
• Any submission to JOSS that has come via AAS will be clearly marked as such in the review process. We’ll create some additional documentation on https://joss.readthedocs.io explaining what’s going on.
• Should any of the reviewers not be comfortable with a contribution being made to JOSS/NumFOCUS based on their review, we will find an alternative reviewer.

What’s next?

Honestly, I’m not exactly sure. At this point, I consider this collaboration with AAS an experiment in sustainability and an opportunity to extend the reach of JOSS to a new audience. The JOSS review is designed to improve the quality of the software submitted, and increasingly I believe JOSS represents a badge of quality that has value in the wider community. Collaborating with AAS on this project is our first attempt at exploring this.

References

Smith A. Announcing The Journal of Open Source Software - Arfon Smith. Published online May 5, 2016. Accessed December 19, 2018. https://www.arfon.org/announcing-the-journal-of-open-source-software

Smith AM, Niemeyer KE, Katz DS, et al. Journal of Open Source Software (JOSS): design and first-year review. PeerJ Computer Science. 2018;4:e147. doi:10.7717/peerj-cs.147