Call for editors

JOSS is continuing to grow, and we are looking to add more editors again. We’re especially interested in recruiting editors with expertise in bioinformatics, neuroinformatics/neuroimaging, material science, ecology, machine learning & data science, and the social sciences.

Since our launch in May 2016, our existing editorial team has handled over 1800 submissions (1200 published at the time of writing, 170 under review) and the demand from the community continues to grow. The last three months have been our busiest yet, with JOSS publishing more than one paper per day, and we see no sign of this demand dropping.

Editors at JOSS make a minimum 1-year commitment, with additional years possible by mutual consent. With some of our existing editorial team reaching the end of their terms with JOSS and this increase in submissions, the time is right to bring on another cohort of editors.

Editing for a journal during a pandemic

After a pause on submissions in early 2020, JOSS has been open for submissions during most of the pandemic. We recognize that making time for volunteer commitments such as JOSS is especially challenging at this time and are taking steps to reduce the load on authors, editors, and reviewers and continually striving to find the right balance between accommodating the very real challenges many of us now face in our daily lives, and providing a service to the research software community.

Editing for JOSS is a regular task, but not one that takes a huge amount of time. JOSS editors are most effective if they are able to check in on their submissions a couple of times per week. Our goal is that a JOSS editor handles about three submissions at any one time making for about 25 submissions per year.

Background on JOSS

If you think you might be interested, take a look at our editorial guide, which describes the editorial workflow at JOSS, and also some of the reviews for recently accepted papers. Between these two, you should be able to get a good overview of what editing for JOSS looks like.

Further background about JOSS can be found in our PeerJ CS paper, which summarizes our first year, and our Editor-in-Chief’s original blog post, which announced the journal and describes some of our core motivations for starting the journal.

More recently we’ve also written in detail about our commitment to the Principles of Open Scholarly Infrastructure the costs related with running JOSS, scaling our editorial processes, and talked about the collaborative peer review that JOSS promotes.

Who can apply

We welcome applications from potential editors with significant experience in one or more of the following areas: open source software, open science, software engineering, peer-review, noting again that editors with expertise in bioinformatics, neuroinformatics/neuroimaging, material science, ecology, machine learning & data science, and the social sciences are most needed.

Members of the JOSS editorial team have diverse backgrounds and we welcome JOSS editors from academia, government, and industry. We especially welcome applications from prospective editors who will contribute to the diversity (ethnic, gender, disciplinary, and geographical) of our board. We also value having a range of junior and senior editors.

How to apply

✨✨✨ To apply please fill in this short form by 23 April 2021. ✨✨✨

Selection process

The JOSS editorial team will review your applications and make recommendations. Highly-ranked candidates will then have a short (~30 minute) phone call/video conference interview with the editor(s)-in-chief. Successful candidates will then join the JOSS editorial team for a probationary period of 3 months before becoming a full member of the editorial team. You will get an onboarding “buddy” from the experienced editors to help you out during that time.

Thanks to our editors who are stepping down

A few of our editors are completing terms and stepping down from editorial duties at JOSS. Lorena A Barba (@labarba), Kathryn Huff (@katyhuff), Karthik Ram (@karthik), and Bruce E. Wilson (@usethedata) have been amazing editors to have on the team and we will miss them very much!

Arfon Smith, Editor-in-Chief, Journal of Open Source Software.

JOSS's Commitment to the Principles of Open Scholarly Infrastructure

The Journal of Open Source Software (JOSS) is committed to the Principles of Open Scholarly Infrastructure, and here we summarize our status in doing so, followed by a more detailed discussion of how we do so, as well as explaining some when we do not, and some work in progress.

This document was assembled by Daniel S. Katz, Arfon Smith, Kyle E. Niemeyer, Kathryn D. Huff, and Lorena A. Barba, and reviewed and approved by the active JOSS editorial board and topic editors, and the Open Journals Steering Council.

Summary

Governance
💛 Coverage across the research enterprise
💛 Stakeholder Governed
💚 Non-discriminatory membership
💚 Transparent operations
💚 Cannot lobby
💛 Living will
💚 Formal incentives to fulfil mission & wind-down

Sustainability
💚 Time-limited funds are used only for time-limited activities
💛 Goal to generate surplus
💛 Goal to create contingency fund to support operations for 12 months
💚 Mission-consistent revenue generation
💚 Revenue based on services, not data

Insurance
💚 Open source
💚 Open data (within constraints of privacy laws)
💚 Available data (within constraints of privacy laws)
💚 Patent non-assertion

(💚 = good, 💛 = less good)

Discussion

Governance

💛 Coverage across the research enterprise

  • It is increasingly clear that research transcends disciplines, geography, institutions and stakeholders. The infrastructure that supports it needs to do the same.

Research software is essential to all types of research, and in response, JOSS’s coverage includes research software in any discipline, from any place, and from any institution.

The scope of what we publish is only limited by a few guidelines. JOSS publications must be research software of sufficient scholarly effort, which means that some software essential to research is excluded because it is either not research software, (e.g., a C compiler) or too small (e.g., a few hundred lines of Python that implement an existing tool or that provides a wrapper to access a fine-grained data source.)

JOSS strives for broad coverage of research disciplines in its editorial board, to better serve a broad community of authors.

💛 Stakeholder Governed

  • A board-governed organisation drawn from the stakeholder community builds more confidence that the organisation will take decisions driven by community consensus and consideration of different interests.

Open Journals is fiscally sponsored by NumFOCUS, and has a documented governance structure. The steering council, being a small group, is limited in its representation, in terms of geographic, ethnic, gender, and organizational diversity. The editorial board members mostly represent North America and Europe, are mostly white, are mostly male, and are mostly hands-on researchers, primarily from universities and national laboratories.

💚 Non-discriminatory membership

  • We see the best option as an “opt-in” approach with a principle of non-discrimination where any stakeholder group may express an interest and should be welcome. The process of representation in day to day governance must also be inclusive with governance that reflects the demographics of the membership.

Additions to the editorial board, which is the first layer of governance, are made via selections from responses to open calls and are non-discriminatory.

💚 Transparent operations

  • Achieving trust in the selection of representatives to governance groups will be best achieved through transparent processes and operations in general (within the constraints of privacy laws).

JOSS is publicly transparent, much more so than most other journals. Few issues are not publicly open, but these are generally open to all editors, including initial discussions about potential changes to the journal and discussions about the scope of submissions that may or may not be accepted for review. Even though some of these discussions may not occur in the open, we do publish the decisions themselves, along with an explanation.

💚 Cannot lobby

  • The community, not infrastructure organisations, should collectively drive regulatory change. An infrastructure organisation’s role is to provide a base for others to work on and should depend on its community to support the creation of a legislative environment that affects it.

The vast majority of time and effort used at JOSS is operational, with improvements of JOSS next, and finally some publicity about JOSS, which could be considered lobbying. However, this lobbying is mostly done by the JOSS editors because they think it is important to them, not to JOSS. The fact that they are also JOSS editors is a consequence of their feelings about the importance of recognizing contributions to research software, which also leads them to talk about this with others.

💛 Living will

  • A powerful way to create trust is to publicly describe a plan addressing the condition under which an organisation would be wound down, how this would happen, and how any ongoing assets could be archived and preserved when passed to a successor organisation. Any such organisation would need to honour this same set of principles.

As discussed below, there are circumstances in which we would consider the mission of JOSS fulfilled and the journal no-longer necessary. While we have not documented a plan for winding down JOSS, we believe that the core assets associated with the journal (software, article metadata, papers) are appropriately preserved as part of our ongoing operations. The articles published in JOSS are persistently archived such that an end of the journal will not affect the scholarly record.

💚 Formal incentives to fulfil mission & wind-down

  • Infrastructures exist for a specific purpose and that purpose can be radically simplified or even rendered unnecessary by technological or social change. If it is possible the organisation (and staff) should have direct incentives to deliver on the mission and wind down.

JOSS views itself as a temporary solution to provide a means for software developers and maintainers to receive credit for their work, and to have this work (research software) improved by the process of open peer review. We look forward to a time when software papers are not needed, when software is directly recognized and cited, and when software peer review (potentially using a future version of our criteria and processes) is more widespread. JOSS is volunteer-run as a service to the community, and most of the volunteers will be happy when a solution like JOSS is no longer needed, because software has found a more direct avenue to be valued and counted in the scholarly record.

Sustainability

💚 Time-limited funds are used only for time-limited activities

  • Day to day operations should be supported by day to day sustainable revenue sources. Grant dependency for funding operations makes them fragile and more easily distracted from building core infrastructure.

JOSS does not depend on grants for regular operations, but has attracted grant funding for specific activities to improve the tooling that facilitates running JOSS. We make effective use of time-limited funds such as grants to support enhancements to our services.

💛 Goal to generate surplus

  • Organisations which define sustainability based merely on recovering costs are brittle and stagnant. It is not enough to merely survive, it has to be able to adapt and change. To weather economic, social and technological volatility, they need financial resources beyond immediate operating costs.

As described in our blog post on the topic, our operational costs are deliberately very low. We currently do not generate a surplus and have no plans to. We also do not employ any staff and so “economic, social and technological volatility” would be expected to have limited impact on JOSS.

💛 Goal to create contingency fund to support operations for 12 months

  • A high priority should be generating a contingency fund that can support a complete, orderly wind down (12 months in most cases). This fund should be separate from those allocated to covering operating risk and investment in development.

We have sufficient funds available today to support our operations for substantially longer than 12 months. Allocating some of these to a formal contingency fund is something we are considering but have not yet done. As a fiscally sponsored project of NumFOCUS, JOSS can receive donations from individuals, and we can kick off a fund-raising campaign at short notice. JOSS can also apply for NumFOCUS Small Development Grants, which are awarded several times per year.

💚 Mission-consistent revenue generation

  • Potential revenue sources should be considered for consistency with the organisational mission and not run counter to the aims of the organisation. For instance…

JOSS revenue comes from three sources: a small amount from donations, a small amount from the American Astronomical Society (AAS) as fees for reviews of the software linked to AAS publications, and a larger amount from grants to the journal related to demonstrating its effectiveness, promoting the importance of research software, and recognizing research software’s contributors. These sources of revenue are fully consistent with the mission of JOSS.

💚 Revenue based on services, not data

  • Data related to the running of the research enterprise should be a community property. Appropriate revenue sources might include value-added services, consulting, API Service Level Agreements or membership fees.

JOSS receives no revenue for its data, which is completely open, but rather receives revenue for its services and for its community impact.

Insurance

💚 Open source

  • All software required to run the infrastructure should be available under an open source license. This does not include other software that may be involved with running the organisation.

All of JOSS’ tools are open source and available on GitHub under the Open Journals organization. This includes the JOSS website, our editorial bot Whedon, and the document production toolchain. Some of the collaboration tools we use as an editorial team are not open (e.g., GitHub, Slack, Google Docs), but these are not critical to the functioning of the journal and could be replaced by open alternatives.

💚 Open data (within constraints of privacy laws)

  • For an infrastructure to be forked it will be necessary to replicate all relevant data. The CC0 waiver is best practice in making data legally available. Privacy and data protection laws will limit the extent to which this is possible

Our papers and the (Crossref DOI) metadata associated with them are available on GitHub, with an open license. We deposit open citations with Crossref, and archive papers and our reviews with Portico.

💚 Available data (within constraints of privacy laws)

  • It is not enough that the data be made “open” if there is not a practical way to actually obtain it. Underlying data should be made easily available via periodic data dumps.

Our papers and the (Crossref DOI) metadata associated with them are available on GitHub, with an open license. These data are easily accessible to all motivated to make use of them.

We could potentially create data exports of the JOSS web application database; however this would just be an alternative representation of the data already available.

💚 Patent non-assertion

  • The organisation should commit to a patent non-assertion covenant. The organisation may obtain patents to protect its own operations, but not use them to prevent the community from replicating the infrastructure.

JOSS has no interest in patents, other than resisting the creation of patents that might prevent us from operating freely.

1000 papers published in JOSS

Today we reached a huge milestone at JOSS – we published our 1000th paper! JOSS is a developer friendly, free-to-publish, open-access journal for research software packages. Publishing 1000 papers (and reviewing the corresponding 1000 software packages) over the past ~4 years has been no small feat. This achievement has been possible thanks to the efforts of our journal team and community of reviewers who have all given their time to make JOSS a success. We take this opportunity to review some of what we’ve learnt over the past four years and outline some plans for the future.

A brief recap

Much has been written1 on the topic of research software, and the challenges that individuals face receiving credit for their work. Software is critical for modern research, yet people who invest time in writing high-quality tools often aren’t well rewarded for it. The scholarly metrics of the “impact” of a researcher’s work do a poor job of supporting software (and more).

JOSS was created as a workaround to some of the challenges for supporting software development in academia. Launched in May 2016, JOSS provides a simple, reliable process for receiving academic career credit (through citation of software papers) for writing open source research software. Authors write and submit a short article (usually under 1000 words) about their software, and JOSS reviews the paper and the software2, reviewing it for a variety of qualities including function, (re)usability, documentation3.

In establishing JOSS, we wanted the editorial experience to be very different from a traditional journal, and developer friendly – short papers authored in Markdown, review process on GitHub, open process and documentation – while at the same time following best practices in publishing depositing first-class metadata and open citations with Crossref, archiving papers and reviews with Portico, leaving copyright for the JOSS papers with authors and more.

We describe the journal this way: JOSS is an open access (Diamond OA) journal for reviewing open source research software. With a heavy focus on automation, our open and collaborative peer review process is designed to improve the quality of the software submitted and happens in the open on GitHub.

Some lessons learned publishing our first 1000 papers

JOSS is meeting a need of the research community

Something unclear when starting JOSS was whether demand from the research community would be sufficient. A few years in, we can safely conclude that JOSS is meeting a real need of the academic community.

It has taken us a little over four years to publish 1000 papers and before pausing submissions for two months starting in early March 2020 (to give relief to our volunteers during the pandemic), we were projecting to reach this milestone sometime in June this year. It took us a little under a year to publish our 100th paper, and an additional 8 months to reach our 200th. In that time we’ve grown our editorial board from an initial group of 10 to more than 50 today.

Screen Shot 2020-08-31 at 19 17 49

People are reading and citing JOSS papers

Well over half of all JOSS papers have been cited, and many have been cited hundreds of times.

While not designed for it, JOSS is proving a useful resource for people interested in new research software: JOSS currently receives ~10,000 monthly visitors to the journal website and provides language (e.g., https://joss.theoj.org/papers/in/Python) and topic-based (e.g., https://joss.theoj.org/papers/tagged/Exoplanets) search filters and feeds.

People are key

Every journal relies upon the expertise, knowledge, and time of their reviewers and JOSS is no different. 935 individuals have contributed reviews for our first 1000 papers, many having reviewed multiple times.

💖💖💖 THANK YOU 💖💖💖

Like many journals, as the number of submissions grows, JOSS has had to scale its human processes. Over the past few years we’ve added many more editors and associate editors-in-chief to enable papers to be handled efficiently by the editorial team. Simultaneously, we’ve developed our editorial robot Whedon from being an occasional assistant during the review process to being the backbone of the whole JOSS editorial process.

Automation is important

A big part of keeping our costs low is automating common editorial tasks where at all possible. The primary interface for editors managing JOSS submissions is a GitHub issue with the assistance of our Whedon bot who supports a broad collection of common editorial tasks. Other than having reviewers, editors and authors read papers, Pandoc-generated proofs are the final version, and no additional copy editing is done before a paper is published. PDF proofs and Crossref metadata are generated automatically by Whedon, and when the time comes, deposited with Crossref and published automatically too.

Screen Shot 2020-08-24 at 08 50 57

When starting JOSS, we thought that automation could be a big part of how things would work if the journal became successful. 1000 papers in, we believe it has been an absolutely critical part of our operations. We call this chatops-driven publishing.

Financial support

JOSS is committed to providing a high-quality service to the community at no cost for authors or readers (Diamond/Platinum Open Access). We’re transparent about our operating costs and have written about cost models for operating an online open journal.

While JOSS’ operating costs are modest, we’ve benefited from the support of a number of organizations including NumFOCUS (the ‘Open Journals’ organization is a sponsored project of NumFOCUS), the Gordon and Betty Moore Foundation, and the Alfred P. Sloan Foundation. It’s also possible to donate to JOSS if you would like to support us financially.

To the future!

With 1000 papers published and a further ~170 papers under review JOSS is busier than ever and there’s little sign of demand from the community slowing.

Over the next year or so, we’re going to be investing resources in a number of key areas to enable JOSS to scale further, improve the experience for all parties, and help others reuse the infrastructure we’ve developed for JOSS. We’ve captured much of what we want to achieve in this public roadmap. All of this will be possible thanks to a new grant from the Alfred P. Sloan Foundation, some highlights include:

Smarter reviewer assignment and management: Finding reviewers for a JOSS submission is still one of the most time-intensive aspects of the JOSS editorial process (though a large fraction of those we ask to review tend to accept, as they are excited by our mission). We think there’s lots of opportunity for substantially improving the success rate of finding potential reviewers through automation. Making sure we’re not overloading our best reviewers will also be an important aspect of this work.

A major refactor of our editorial bot Whedon: Whedon is a critical part of our infrastructure but has become hard to maintain, and almost impossible for other projects to reuse. We’re planning to rework Whedon into a general framework with a set of reusable modules for common editorial tasks.

Investments in open source: JOSS relies upon a small number of open source projects such as Pandoc and pandoc-citeproc to produce scholarly manuscripts (PDFs) and metadata outputs (e.g., Crossref and JATS). We’re going to work with the Pandoc core team to generalize some of the work we’ve done for JOSS into Pandoc core.

For many of us on the editorial team JOSS is a labor of love, and it has been quite a ride growing JOSS from an experimental new journal to a venue that is publishing more close to 500 papers per year. For those of you who have helped us on this journey by submitting a paper to JOSS or volunteering to review, thank you ⚡🚀💥.

The JOSS editorial team.

  1. For example: https://doi.org/10.12688/f1000research.11407.1 · https://doi.org/10.1109/MS.2020.2973362 · http://doi.org/10.5334/jors.242 

  2. Something hardly any other journals do. 

  3. The JOSS review structure was originally derived from the process developed by the rOpenSci community for their software review. 

Minimum publishable unit

tl;dr – JOSS is introducing new submission criteria whereby submissions under 1000 lines of code will automatically be flagged as potentially being out of scope, and those under 300 lines desk-rejected. This blog post describes some of the motivations behind this decision.

Sometime in 2020, JOSS will publish its 1000th paper – an incredible achievement by a volunteer team of editors and reviewers.

Since its inception a little over four years ago, the primary goal of JOSS has always been to provide credit for authors of research software. Quoting from the original blog post announcing JOSS:

The primary purpose of a JOSS paper is to enable citation credit to be given to authors of research software.

One challenge we’ve always struggled with as an editorial team is defining clear guidelines for submissions allowed (and not allowed) in JOSS. Our current submission criteria are available online and include language about software having to be a significant contribution, feature complete, and having an obvious research application. In these criteria we also explicitly exclude a category of software we generally call “minor utilities”.

The challenge of defining a unit of publication credit

We think of JOSS essentially granting “1 publication credit” for each software package that is reviewed and published in the journal. In empirical research, a publication is often the result of years of work. In engineering research, rarely does a paper represent less than one year of work. Other fields may vary, but let’s say that a scientific paper resulting from work measured in just months is rare or exceptional.

Since the earliest days of the journal, there has been a range of views within the editorial team on what level of effort we should require from authors for a submission to be allowed in JOSS – some JOSS editors feeling that every useful piece of software should be considered, others believing that the “bar” for publishing in JOSS should be higher than it currently is.

Building trust in JOSS

As an editorial team we want JOSS papers to count the same as any other publication in the CV of researchers who write software. With career credit the stated primary reason for starting JOSS, if this isn’t true then the mission of the journal is at risk.

In reality this means that our editorial policy requires us to balance two competing needs:

  1. Providing an excellent service to authors by offering peer-review of their software and the opportunity to receive career credit for their work.
  2. Building the trust of an existing academic culture to accept a JOSS paper as equal to any peer-reviewed journal paper.

These two aspects are in tension with each other because while we would dearly love to publish any and all research software in JOSS regardless of size/scale and level of effort to implement, part of building and maintaining that trust with the community relies on us ensuring that our published authors can continue to expect JOSS papers to “count” in their future merit reviews and promotions.

Updates to our submission requirements and scope-checking procedures

Over the past couple of years, as the number of submissions to JOSS has grown, we’ve found that our existing submission criteria and protocol for rejecting papers as out of scope have been taking a significant fraction of our editorial team time. With a volunteer team of editors, it’s essential that we use their time carefully and an update to our procedures for handling scope assessments is long overdue.

Going forward we’re going to adopt the following new process:

Automatically flagging small submissions

As part of the pre-review process, incoming submissions that are under 1000 lines of code1 (LOC) will be automatically flagged as potentially out of scope by the EiC on rotation.

Submissions under 300 lines2 of code will be desk rejected with no further review.

Mandatory “Statement of need” section in JOSS papers

While this has always been preferred, a clear Statement of need section in the JOSS paper is usually extremely valuable in helping the editorial team understand the rationale for the development of the software.

Gauging the scholarly content of the software as part of the review

Reviewers will be asked if the software under review is a “substantial scholarly effort” and guidelines will be provided on how they can make that assessment.

A streamlined editorial review process

Rather than each paper that is potentially out-of-scope being discussed in a separate thread on our editorial mailing list, JOSS is going to move to a weekly review of such papers by our editorial team. Topic editors will be asked to review papers flagged as potentially out of scope in their area and help the EiC team make a decision.

Arfon M. Smith on behalf of the JOSS editorial team

  1. In a high-level language such as Python or R. More verbose languages such as Java, C++, Fortran etc. will require more LOC. 

  2. We realize that introducing numerical thresholds may encourage some authors to unnecessarily “pad” their submissions with additional lines of code to meet our thresholds. As reviewers are already asked to judge the standard of the implementation as part of their review we expect that situations like these will usually be flagged during the review. 

Reopening JOSS

On March 12th of this year, we suspended new submissions to JOSS in order to reduce the load on our volunteer team of editors and reviewers. We revisited this suspension in early April and decided to continue the pause on new submissions for at least another month. Over the past two weeks, we’ve been reviewing our current status and discussing what it might look like to reopen JOSS.

After collecting inputs from the whole JOSS editorial team about their respective availabilities and discussing what to do next we’ve decided to reopen JOSS in a “reduced service mode” from May 20th, 2020.

Reduced service mode

So what does a “reduced service mode” mean exactly? Essentially this is about adjusting our editorial processes to give authors, editors, and reviewers more time to carry out their activities during the JOSS review process.

We expect that JOSS will be operating in this mode for the foreseeable future with the following new editorial processes:

Revised (extended) review time: Before the suspension, we encouraged reviewers to complete their reviews in two weeks. In this reduced service mode, we’re asking that reviewers try to complete their reviews in six weeks. Reviewers are of course welcome to complete their reviews faster than this and given the iterative nature of JOSS reviews we still encourage everyone to start their review as soon as possible.

Reduced paper throughput: As we reopen JOSS, we plan to limit the number of incoming submissions that will be assigned to a handling editor (and therefore go out for review). In these first few weeks we’re planning on a limit of five papers per week, with papers after this held in a first come, first served waitlist pre-review issue on GitHub.

Re-assign the submissions with unavailable editors: A small number of our editors are currently completely unable to edit due to other commitments. As we reopen JOSS, those papers will be reassigned to a new handling editor.

Growing our editorial team: Back on February, we issued a call for new editors which received an excellent response. Over the coming weeks we will begin on-boarding new editors to help grow the capacity of JOSS.

What this means for JOSS

During the pause on submissions, we’ve managed to publish 37 papers from the existing backlog which is an amazing feat in itself 💖. Since instigating the pause however, activity across all of JOSS had been markedly reduced.

JOSS relies upon editors and reviewers volunteering their time to help authors improve their research software in an iterative, open review process. For many of us, JOSS is a labor of love, and an opportunity to give back to the communities we work in.

While we can never know exactly how the COVID-19 pandemic is affecting every JOSS author, editor, and reviewer, we hope that with these changes to our editorial processes we have found the right balance between accommodating the very real challenges many of us now face in our daily lives, and providing a service to the research software community.

Arfon M. Smith on behalf of the JOSS editorial team

Call for editors

Once again we’re looking to grow our editorial team at JOSS. We’re especially interested in recruiting editors with expertise in bioinformatics, material science, physics, R/statistics, and the social sciences.

Since our launch in May 2016, our existing editorial team has handled over 800 submissions (830 published at the time of writing, 119 under review) and the demand from the community continues to grow. The last three months have been our busiest yet, with JOSS publishing a little over one paper per day, and we see no sign of this demand dropping.

New editors at JOSS are asked to make a minimum 1-year commitment, with additional years possible by mutual consent. As some of our existing editorial team are reaching the end of their term with JOSS, the time is right to bring on another cohort of editors.

Background on JOSS

If you think you might be interested, take a look at our editorial guide, which describes the editorial workflow at JOSS, and also some of the reviews for recently accepted papers. Between these two, you should be able to get a good overview of what editing for JOSS looks like.

Further background about JOSS can be found in our PeerJ CS paper, which summarizes our first year, and our Editor-in-Chief’s original blog post, which announced the journal and describes some of our core motivations for starting the journal.

More recently we’ve also written in detail about the costs related with running JOSS, scaling our editorial processes, and talked about the collaborative peer review that JOSS promotes.

How to apply

Firstly, we especially welcome applications from prospective editors who will contribute to the diversity (ethnic, gender, disciplinary, and geographical) of our board.

✨✨✨ If you’re interested in applying please fill in this short form by 4th March 2020. ✨✨✨

Who can apply

We welcome applications from potential editors with significant experience in one or more of the following areas: open source software, open science, software engineering, peer-review, noting again that editors with expertise in bioinformatics, material science, physics, R/statistics, and the social science are most neded.

The JOSS editorial team has a diverse background and there is no requirement for JOSS editors to be working in academia. Unfortunately individuals enrolled in a PhD program are not eligible to serve on the JOSS editorial team.

Selection process

The JOSS editorial team will review your applications and make their recommendations. Highly-ranked candidates will then have a short (~30 minute) phone call/video conference interview with the editor(s)-in-chief. Successful candidates will then join the JOSS editorial team for a probational period of 3 months before becoming a full member of the editorial team. You will get an onboarding “buddy” from the experienced editors to help you out during that time.

Lorena A. Barba, Daniel S. Katz, Kevin M. Moerman, Kyle E. Niemeyer, Kristen Thyng, Arfon M. Smith

New editors in 2019

2019 has been a big year for JOSS: This year, we’ve already published 300 papers and are on target to reach ~360 papers by the end of the calendar year. We’ve substantially improved our website and made a number of important changes to our editorial team to help us scale further.

In the last 12 months we’ve issued two calls for new editors (one in December 2018 and one in August 2019), and we’ve not done a very good job of announcing when new editors join the team so let’s fix that…

Two new associate editors in chief

In the last few months, we’ve added two associate-editors-in-chief with Kevin M. Moerman and Kristen Thyng stepping into these new roles ⚡.

New editors joining the JOSS team

Twenty one(!) new editors have joined the JOSS editorial team thus far in 2019. Ordered by the date they joined they are:

Viviane Pons: Mathematics, Computer Science
Associate professor at Université Paris-Sud. Mathematician, computer scientist and strong defender of open-source and open science in general. Contributor and user of the SageMath software. Member of the OpenDreamKit European project for open-source development in Mathematics.

Jack Poulson: Numerical optimization, numerical linear algebra, PDEs, high-performance computing, lattice reduction
Independent computational scientist running Tech Inquiry and Hodge Star Scientific Computing. Previously, research scientist at Google and assistant professor of mathematics at Stanford. His research interests: software engineering of high-performance mathematical libraries (e.g., conic optimization, lattice reduction, determinantal point processes, numerical PDEs), their connections to pure mathematics (e.g., differential geometry, conic analysis, representation theory).

George K. Thiruvathukal: HPC, software engineering, programming languages, systems, computational science, digital humanities
Professor of computer science at Loyola University, Chicago, and visiting faculty at the Argonne National Laboratory Leadership Computing Facility. Research interests: high-performance & distributed computing, cyber-physical systems, software engineering, programming languages and systems, history of computing, computational and data science, computing education, and ethical/legal/social issues in computing. Past editor-in-chief of IEEE Computing in Science and Engineering.

Lorena Pantano: Small RNAseq, RNAseq, miRNA, isomiRs, visualization, genomics, transcriptomic, non-codingRNA, data integration
Research scientist at Harvard T.H. Chan School of Public Health. Focused on genomic regulation and data integration, with more than a decade of experience in biological data analysis and contributing to novel algorithms to improve the quantification and visualization of genomic data.

Juanjo Bazán: Astrophysics, Mathematics
Astrophysics researcher at CIEMAT, mathematician and software engineer currently developing chemical evolution models for galaxies. Juanjo has worked as advisor on open source policies and contributed code to many popular libraries like Rails and Astropy. He is member of the founder team of Consul, the most widely used open sourced citizen participation software.

Bruce E. Wilson: Ecology, remote sensing, information sciences, material sciences
Manager for the Oak Ridge National Laboratory Distributed Active Archive Center for Biogeochemical Dynamics (ORNL DAAC) and Adjunct Professor of Information Sciences at the University of Tennessee, Knoxville. Originally trained as a chemist and statistician. Spent a few years as an Enterprise Architect. Research interests in citations, linked data, reproducible science, identity, cybersecurity, data reuse, and long-term data preservation.

Leonardo Uieda: Geoscience, Geophysics, Data Visualization
Geophysicist researching methods for determining the inner structure of the Earth from geophysical observations, mainly disturbances in the Earth’s gravity and magnetic fields. Developer of open-source software for processing, modeling, and visualizing geophysical data. Currently Visiting Research Scholar at the University of Hawai’i at Mānoa working on Generic Mapping Tools.

Alex Hanna: Social sciences, politics
Alex Hanna is a computational social scientist working on machine learning curriculum at Google. She received her PhD in sociology from the University of Wisconsin-Madison. Her research has focused on how new and social media has changed social movement mobilization and political participation. More recently, she has been interested in issues of fairness, accountability, and transparency in sociotechnical systems.

Charlotte Soneson: Bioinformatics, data visualization, transcriptomics, reproducible research
Research Associate at the Friedrich Miescher Institute for Biomedical Research in Basel, Switzerland, with a research background mainly in development and evaluation of analysis methods for transcriptomics data. Developer and maintainer of several open-source R packages for analysis, quality assessment and interactive visualization of high-throughput biological data.

Monica Bobra: Heliophysics, Data Science
Research scientist at Stanford University in the W. W. Hansen Experimental Physics Laboratory who studies the Sun and space weather as a member of the NASA Solar Dynamics Observatory science team and contributes to Heliopython and SunPy.

Yuan Tang: Machine Learning, Distributed Systems, Cloud Computing
Senior software engineer at Ant Financial, building AI infrastructure and AutoML platform. He’s a committer of TensorFlow, XGBoost, and Apache MXNet, maintainer of several Kubeflow projects, and author of numerous open source software projects. He’s also the author of the best-selling book TensorFlow in Practice, which is the first book teaching TensorFlow in Chinese and has been translated to several other languages such as traditional Chinese and Korean.

Olivia Guest: Computational cognitive modeling, cognitive science, cognitive neuroscience
Olivia Guest is a computational modeler in cognitive science and neuroscience. She creates and evaluates computational accounts for categorization and conceptual representation in healthy adults, patient groups, infants, and animals. She is also interested in using computational modeling and data science broadly in theoretical as well as applied contexts.

Marie E. Rognes: Applied mathematics, PDEs, numerical methods, biomechanics
Marie E. Rognes is Chief Research Scientist at Simula Research Laboratory, Oslo, Norway. Her research focuses on numerical methods for partial differential equations, software for scientific computing, with applications in biomechanics and neuroscience. She is a core member of the FEniCS and Dolfin-adjoint Projects.

Vincent Knight: Mathematics, Applied mathematics, Game Theory, Stochastic processes, Pedagogy, Python
Vince is a mathematician at Cardiff University. He is a maintainer and contributor to a number of open source software packages and a contributor to the UK Python community. His research interests are in the field of game theory and stochastic processes and he also has a keen interest in pedagogy. Vince is a fellow of the Software Sustainability Institute and is interested in reproducibility and sustainability of scientific/mathematical research.

Matthew Sottile: Computer science, programming languages, applied mathematics
Research scientist working on program synthesis tools for high performance computing at Noddle.io, and adjunct faculty at the Washington State University Department of Mathematics and Statistics. His PhD work in computer engineering focused on measurement and analysis methods for performance analysis of high performance systems. He believes that open and reproducible research software artifacts are critical to the advancement of computational science.

Mark A. Jensen: Bioinformatics and Computational Biology, Cancer Genomics, Population and Evolutionary Biology, Databases and Data Architecture, Software Development Lifecycle Management
Director of Data Management and Interoperability at the Frederick National Laboratory for Cancer Research, he leads efforts to design, build, and maintain scientist-friendly research data systems that integrate clinical and multiomic data across thousands of cancer patient-donors. He is active in open source software development and a supporter of the FAIR (Findable, Accessible, Interoperable, Reusable) movement in scientific data management. A molecular evolutionary biologist by training, he served as an Associate Editor for the Journal of Molecular Evolution from 2008-2013.

Melissa Weber Mendonça: Applied Math, Numerical Optimization, Numerical Linear Algebra, Fortran, Python, MATLAB, Julia, Teaching
Applied mathematician, working with numerical optimization and numerical linear algebra at the Federal University of Santa Catarina, in Brazil. Interested in open science, open research, and teaching practices in mathematics and computer science.

Katy Barnhart: Geology, Geophysics, Earth Surface Dynamics, Terrain Analysis
Research scientist in Earth surface dynamics at the University of Colorado Department of Geological Sciences and Cooperative Institute for Research in Environmental Sciences. Her research focuses on integrating observations, model development, and model analysis to understand the evolution of the Earth’s surface. She is a core developer of the Landlab toolkit.

William Rowe: Bioinformatics, genomics
Bioinformatician working at the University of Birmingham, UK. Research areas include data sketching for genomics, variation graphs, and metagenome profiling. Currently working on long-read sequencing applications for real-time genomic epidemiology.

Dan Foreman-Mackey: Astrophysics, probabilistic programming, data science, python
Dan Foreman-Mackey is an Associate Research Scientist at the Flatiron Institute in the Center for Computational Astrophysics. His research program focuses on the development and application of probabilistic data analysis techniques to make novel discoveries and solve fundamental problems in astrophysics.

Marcos Vital: R, biostatistics, data visualization, open science, quantitative ecology, evolutionary biology, conservation biology, teaching
Biologist, quantitative ecologist and open science enthusiast at Universidade Federal de Alagoas, Brazil, where he coordinates the Quantitative Ecology Lab. He is interested in a wide range of topics related to evolutionary ecology and conservation biology, but also in other fields such as active learning methods and digital games in education.

Editors retiring from the team

In 2019, a few of our editors have stepped down from editorial duties at JOSS: Jason Clark, Pjotr Prins, and Yo Yehudi have already retired from the JOSS editorial team and Roman Valls Guimera, Melissa Gymrek, and Lindsey Heagy will be retiring from the team soon. JOSS only works because of the volunteer efforts from our editors and reviewers and we’re deeply indebted to them 💖.

We thanks Jason, Pj, Yo, Roman, Melissa, and Lindsey for all of their contributions to JOSS!

Arfon Smith, Editor-in-Chief, Journal of Open Source Software.

Call for editors

Following our call for editors late last year, once again we’re looking to grow our editorial team.

Over the past ~40 months, our existing editorial team has handled close to 800 submissions (665 accepted at the time of writing, 101 under review) and the demand from the community continues to grow. The last three months have been our busiest yet, with JOSS publishing a little over one paper every day, and we see no sign of this demand dropping.

In order for JOSS to be able to continues to thrive, we think the time is right to bring on another cohort of editors.

Background on JOSS

If you think you might be interested, take a look at our editorial guide which describes the editorial workflow at JOSS and also some of the reviews for recently accepted papers. Between these two, you should be able to get a good overview of what editing for JOSS looks like.

Further background about JOSS can be found in our PeerJ paper which summarizes our first year and our Editor-in-Chief’s original blog post announcing the journal describes some of the core motivations for starting the journal.

More recently we’ve also written in detail about the costs related with running JOSS and scaling our editorial processes.

How to apply

Firstly, we especially welcome applications from prospective editors who will contribute to the diversity (ethnic, gender, and geographical) of our board.

We’re especially interested in growing our editorial team in the following subject areas although all are welcome to apply:

  • Applied mathematics
  • Bioinformatics
  • Computational chemistry
  • Ecology
  • Geoscience/earth sciences
  • Material science
  • R/stats

✨✨✨ If you’re interested in applying please fill in this short form by 4th September 2019. ✨✨✨

Selection process

The JOSS editorial team will review your applications and make their recommendations. Candidates ranking highly will then have a short (~30 minute) phone call/video conference interview with the editor-in-chief(s). Successful candidates will then join the JOSS editorial team for a probational period of 3 months before becoming a full member of the editorial team. You will get an onboarding “buddy” from the experienced editors to help you out during that time.

Lorena A. Barba, Daniel S. Katz, Kyle E. Niemeyer, Arfon M. Smith

Scaling the Journal of Open Source Software (JOSS)

The Journal of Open Source Software (JOSS) started with one editor-in-chief (EiC) and 10 topic editors, and in our first year, we published about 100 papers. Three years later, the rate of publication has increased to 300 papers per year. How did we scale up to this point, and how can we continue scaling the journal?

Scaling to-date

Today, we have one editor-in-chief (EiC), three associate editors-in-chief (AEiCs), and 23 topic editors. We recognize past service of editors that step down, for any reason, as emeritus editors on our website (currently, six). The journal is currently receiving about one new submission per day, a 3.5x increase in 2 years.

What scales and what doesn’t – or – what we did to scale to where we are now:

Assigning papers to editors: when a paper is submitted, it enters the “incoming” queue, from which an AEiC moves it to a Pre-Review issue and assigns it to a handling editor. We scaled this task, initially handled by the EiC, to the EiC and 3 AEiCs who rotate on-duty responsibilities weekly. The overall per-person workload for this task is thus manageable.
Assigning papers to reviewers: after a Pre-Review issue starts, the assigned handling editor has the task of finding and assigning reviewers to the paper. This task can scale up by 1) recruiting additional editors so that each editor has a manageable workload, as long as 2) there are sufficient appropriate reviewers available and 3) the editor can find them. JOSS currently assigns a minimum of two reviewers/paper. As we receive more submissions each year, more reviewers are needed. Although we can ask authors of accepted papers to review new submissions, with many more active papers simultaneously, we will need better tracking of reviews contributed, to avoid overburdening some reviewers, and we may need a better way to find appropriate reviewers than the keyword searches of existing volunteer reviewers and the personal knowledge and search skills of the editors that we currently use.
Overseeing active reviews: editors and AEiCs maintain a close eye on active reviews to keep things moving along, moderate conversations, and clarify the appropriate process when needed. This currently scales well for handling editors, who have a dashboard showing the state of their assigned papers with key activity stats. The AEiC on rotation is aided by the “In Progress Papers” dashboard, with a manageable workload to keep reviews from becoming stuck; this scales acceptably at this point with AEiC weekly rotations.
Accepting and publishing papers: the one AEiC on duty each week completes the final tasks needed after reviewers and handling editor recommend acceptance. Our current rotation system avoids excessive long-term workload on any one person, but may not scale to a much larger number of papers accepted each week, as part of the AEiC’s job is final proofreading and detail checking, which can be time consuming.
Editorial robot: Much of the editorial process is assisted by our editorial bot ‘Whedon’ which carries out a number of automated processes on behalf of editors (see this post for more details). These automated steps include preparing the final version of a publication, checking references for missing or malformed DOIs, adding the software release version and archival DOI to the metadata, building an XML file for Crossref metadata, and executing the Crossref DOI registration. All of these steps are possible with simple commands issued to Whedon in the review thread on GitHub. For example, typing the command @whedon accept in a comment on the Review issue will automatically compile the paper PDF from the author’s source in markdown, check the references, and create the Crossref XML, providing links to these files for the AEiC’s final visual check.
Overall process overview: We built a JOSS editor dashboard to 1) help the AEiCs balance editor loads, review unassigned submissions, and understand the status of all in-progress reviews. The dashboard also 2) helps editors keep track of all current submissions (in addition to an automated weekly email reminding editors of current assignments).

Future scaling

If the number of submissions increased by another factor of 3 or even 10, we would need to make additional changes to handle the increase. Some of these could be:

Increase the number of AEiCs

If we do this, we might also need to change the schedule we now have. Our idea of rotating a single AEiC on duty at any one time likely wouldn’t still work for assignment of papers to editors; we might need multiple AEiCs at a time with that assignment of papers to AEiCs done in either a round-robin or random manner.

A decreased rotation time might work for accepting papers and publishing them, but the round-robin and random assignment would also work.

Alternatively, each paper that comes in might be assigned to an AEiC for all work on the paper, from assigning an editor through to acceptance and publication, though this would remove the workload pauses that AEiCs currently have during their off-duty periods.

Further increase number of editors

This should scale reasonably well.

Developing additional tooling for handling the editorial process

As both the number of AEiCs and editors increase, additional tooling likely will be needed, including:

  • Semi-automated tools for authors to suggest reviewers from a pool of volunteers and previous JOSS authors, and for editors to choose reviewers from these same sources as well as from the community outside JOSS
  • Semi-automated tools for assignment of papers to editors
  • Semi-automated checking of status for both editors and AEiCs
  • Reliable statistics on reviewer contributions to avoid reviewer fatigue

Better support

We could off-load some of the AEiC work (e.g., helping authors who have problems with formatting papers or other issues) either to a paid staff member, or more likely, we could set up support mechanisms for authors to be helped by other authors or editors, perhaps by Stack Overflow or something similar. Given that we are working in an open source environment, this would be a way of getting some members of the community to take on a little more, which might later lead to some becoming editors.

An alternative to scaling JOSS itself would be to scale up by starting more JOSS-like journals, even using the same infrastructure, perhaps by discipline, by programming language, etc. (Note to commercial publishers: We would still make these open access and free to publish; we don’t mean that you should start JOSS-like journals and charge APCs.)

Daniel S. Katz, Lorena A. Barba, Kyle E. Niemeyer, Arfon M. Smith

Cost models for running an online open journal

The Journal of Open Source Software (JOSS) is a free, open-access online journal, with no article processing charge (APC). We are committed to operating as a free service to our community, and we do so thanks to the volunteer labor of editors and reviewers, and by taking advantage of existing infrastructure. In this post, we examine the true costs of running a journal such as JOSS, and make the case that even when considering all services we don’t currently pay for, the true cost per paper would not exceed $100. Current APCs at many “gold” open-access journals exceed that by one or more orders of magnitude, (see, for example, PNAS, Nature, IEEE, etc.)

Real costs we *have* to pay

  • Annual Crossref membership: $275/year
  • JOSS paper DOIs: $1/accepted paper
  • JOSS website hosting: $19/month
  • JOSS domain name registration: $10/year

At 300 papers/year, this is $813, or $2.71/paper. This is what we actually pay today, covered by a grant from the Alfred P. Sloan Foundation.

JOSS is a fiscally sponsored project of NumFOCUS, through which we can receive donations. We remind authors about the opportunity to donate as part of the message announcing the acceptance of their papers. Over about 3 years of operation, we have received $285 in donations from a small number of individuals. (If you wish to donate now, use the NumFOCUS secure form!)

Services for which we don’t currently pay

The one-time cost for initial development of Whedon, the service (bot) that runs in an issue tracker, and the JOSS web application that runs http://joss.theoj.org would have cost an estimated $50,000 (0.25 FTE). This was covered by a combination of volunteer effort by the JOSS EiC (Arfon Smith), and a small grant from the Sloan Foundation. If amortized over first 1,000 papers, this would add $50/paper during that period, but that’s also the time when the journal needs to keep costs down to encourage submissions. Let’s instead imagine we have to redesign and reimplement our system from scratch every 10 years, and add $5,000 per year for this.

In addition, we require some ongoing development of the existing system every year (adding features, fixing bugs, etc.), at an estimated cost of $5,000/year. This is now mostly done by volunteers, but also partly supported by the Sloan Foundation grant.

We currently host JOSS papers and run reviews on GitHub, which we don’t pay for. If we had to pay for this, or we wanted to run our own system for some reason, we could use GitLab for issue tracking instead, at $50/month.

Another aspect is running the organization that publishes the journal. Currently, we depend on NumFOCUS as our fiscal sponsor to provide us with the ability to accept funds such as the Sloan grant and donations, to pay for needed items and services, and to provide some financial management/accounting services. This is paid for by the overhead on the Sloan Foundation grant, which is handled by NumFOCUS. If we were to pay for these services directly, they might cost $10,000/year.

Summarizing these additional potential costs:

  • Reimplementation of software infrastructure every 10 years: $5,000/year
  • Ongoing software infrastructure development and maintenance: $5,000/year
  • GitLab instance: $50/month
  • Financial services: $10,000/year

Additional volunteer contributions

We depend on a set of volunteers to run JOSS. These include

  • 1 Editor-in-chief working 4 hrs/week (0.1 FTE)
  • 4 Associate Editors-in-Chief (including the EiC), working a total of 10 hours/week, where the 4 rotate, so only one is on duty in any given week (0.25 FTE)
  • 27 Editors (including the AEiCs), working a few hours per week (~1.5 FTE)
  • 651 reviewers (as of 30 May 2019), working when assigned a paper

JOSS does not pay any of these people. Nor are these roles in other scholarly journals paid positions (with rare exceptions), whether open access or not, whether the publishers are for-profit or not. We are aware that some publishers/journals pay stipends to the editor-in-chief (ranging from a few thousand dollars to in some cases ten to twenty thousand), and a couple of publishers/journals have in-house salaried editors.

Editing/Design

We rely on authors to provide “camera-ready” articles, though we help them generate these articles via an automated system that accepts markdown text and author-supplied images. We do not work directly with the authors to improve their article design/layout. Editors and reviewers do suggest changes to wording in articles as part of the review process, but we do not provide or apply professional copy-editing, which a few journals do. Given that the average journal doesn’t provide more author services than we do, and many provide less (editors do not provide as much language/grammar help), we will not consider any costs associated with these services.

Marketing

We do not have any marketing costs, other than giving out some stickers from time to time, at minimal cost. While we recognize that some large publishers regularly attend conferences where they have booths and have other marketing costs, we will not consider any costs associated with marketing since we don’t really do any.

Summary

Here are all the costs associated with running JOSS, assuming 300 papers/year:

  • Costs we now pay: $813/year
  • Services for which we don’t currently pay: estimated at $20,600/year
  • Volunteer editor services: 1.85 FTE effort or $370,000 (@$200,000/FTE)
  • Reviewer time (not paid for in almost all journals, nor costed by JOSS)

This would lead to a valuation of the work required per paper of about $1,300 (excluding reviewer efforts), but given current practices regarding editor compensation, including just $10,000 as editor stipend (likely on the high side of today’s practices), we obtain a total annual operating cost of $31,413, requiring an article processing charge (APC) of about $100 per paper.

If we were a for-profit organization, we would also add a profit margin. While 10% is considered a good profit margin in many industries, 30-35% is more common in scholarly publishing. This would still only lead to an APC of about $140/paper.

Existing publishers and professional societies might consider adopting an online and open infrastructure like JOSS’s to reduce their costs. In any event, JOSS will continue to be openly accessible and free to publish.

Daniel S. Katz, Lorena A. Barba, Kyle E. Niemeyer, Arfon M. Smith