Quality Control in Scientific Publishing: what is it actually like to review papers?

It’s Friday evening and because I’m so rock and roll, or, actually, because I’m so very far behind with jobs, I’m supposed to be spending this evening reviewing papers for four different journals. Confession: I’m actually watching The Craft and writing this instead. I’m hoping it will inspire me to get on with the task at hand.

PhD Comics = So true it’s painful

What is paper reviewing?

Manuscripts (scientific papers/articles) go through a process called ‘peer review’ as part of the publication process. It’s a key part of ensuring the quality of published work, which is then going to reach a much wider and, sometimes, ‘non expert’ audience.

My job as a reviewer is to do a few key things (from my position as a life scientist):

  • Help the editor ensure that the paper is ethical. For instance, I didn’t use animal models when other methods would be more suitable, or that clinical data was misused without permission.
  • Ensure that the paper is reproducible. Is there enough information in the methods that I could take them and try to reproduce the experiment to ensure that it works and can apply to other samples/data?
  • Confirm whether the work is novel and that it adds to the body of scientific work out there. This also means we attempt to identify plagiarism, but I would never claim to be able to know all of the literature out there to ensure this.
  • Respond to whether the work is of a suitable standard for publication. This is very open, but mainly means: is the question they’ve asked of the data the right one? is the experimental design able to answer the research question posed? is the literature and justification presented for interpretation appropriate? Have they correctly reported on the flaws and biases that are inevitably present in any piece of work?

What is the peer review process?

Once you submit a paper (see my post on Writing and Publishing Scientific Papers), your submission will be allocated to an editor. That editor will then select reviewers from (usually) a combination of the reviewers you’ve suggested when you submitted and the list of reviewers they have on file. One of the reasons it’s so important to choose a journal that matches what you’re submitting is that you want to make sure the reviewers they have on file are going to have the knowledge and expertise in your topic area.

Once the editor has picked their reviewers they will then email out an invitation to review to those scientists. Usually, at this point, the invitee can see the abstract and make a decision whether they have the knowledge and skills to be useful in undertaking the review. I get some truly random invites that I turn down linked to topics I know nothing about and so it is really important that as individuals we’re aware of the scope of our practice and don’t overstretch. Once the reviewer has accepted they then get access to the whole paper and a deadline for submitting their response, usually ~ 2 weeks.

When you submit your review you have to justify your responses and give specific feedback for the authors to address. There are four main categories of response:

  • Accept without revision
  • Accept with minor revisions
  • Accept with major revision (maybe able to request that the paper is re-reviewed as part of this)
  • Reject

Now this is where I have to fess up! I am great at the accepting the invite part. I am not as good at getting the job done because of my other work load. I am probably one of the reasons that your paper reviews take AGES to come back. Sorry about that.

What is it like to review papers and how do I start?

To be honest, I usually have a feel for how it’s going to play out from the abstract. This is why a well-written abstract is so crucial: it gives a really good idea about how the author is going to be able to present their chain of thought and to be succinct in what is a relatively short format of ~4000 words for most papers.

If the paper has good concepts but it needs extra data or re-writing to get there, I will usually take a fair amount of time to give a lot of comments. I know this sounds perverse. However, the more comments I give you, if I give major corrections, the more worthy I think your paper is. It takes time to give feedback and I don’t put in the energy if it doesn’t have merit.

I’m not a rejecter. I don’t often completely reject papers unless they are clinically unsafe. This sometimes happens when non-clinical researchers make clinical suggestions in terms of antibiotic use when they are not qualified to do so.

Two final things.

One – If I spend a lot of time giving a heap of comments to try and make the submission better and it comes back to me without any attempt being made at most of them, I will a) remember as not that much time will have passed and b) not be very happy that my time spent trying to make the work better has been ignored.

Two – Lots of people believe we are paid or get some benefit from reviewing papers. We don’t. We don’t even get a discount for submitting to the journals we review for. The benefit you get is learning and reflecting about what makes a good article and therefore how to make your own work better.

Right! I’d better get on with reviewing those papers now………………….

Top Tips:

  • Think carefully about who you suggest as reviewers.
  • Some submission pages have options to list people who you don’t want to review as they are competitors within your field. This isn’t such a big deal in my world but, if you’re doing pure research, it’s worth considering.
  • Take the opportunity to start reviewing papers early in your career. It will help you think about your own writing and will improve your submissions.
  • Don’t take reviewers’ comments personally – use the gin and tonic method described under Don’t Get Disheartened in my previous post. They are there to help you, so take the constructive and let the rest wash over you.
  • If you are a reviewer remember our job is to be constructive, try not to be ‘Reviewer Three’.

All blog opinions are my own

Writing and Publishing Scientific Papers – Is it as hard as it seems?

Before I start. In the interest of full disclosure, I’m a published academic but I have fewer than 30 papers, not the 200+ some of my colleagues have. See my publications page if you’re interested. There are reasons for this. One, I’ve only had my PhD since 2015. But the main reason is that I’m not your traditional academic. I have a clinical post which is >50% of my time and so my work is about moving research from the research (academic) setting into clinical practice to improve patient care.

Why I Research

I still really clearly remember the stress that writing and submitting my first 1st author scientific paper caused me. It’s difficult to describe the transition from being a good student to being an academic. As students, we fear failure. I, like most academics, have never failed an exam and have what can only be called a visceral dread of what it would mean. You then move into a world where 80% of grants will be rejected and failure becomes part of everyday life. When I submitted my first paper, I hadn’t come to terms with that yet. I was still worried about what rejection would mean for me and how people see me. Now it’s just a fact of life.

Some misconceptions about publishing.

A lot of my friends mistakenly believe that scientists get paid for publishing their work. The opposite is true. If I want my work to reach the maximum number of people, I have to pay (usually several thousand pounds) for my article to be open access (i.e. free to access). Therefore, dissemination of your work can be really expensive and not necessarily reach the right people, as most clinicians and patients won’t be able to read articles requiring payment. It’s one of the reasons why science communication is key.

One of the other common beliefs is that the collection of data is most of the work in getting a scientific publication. This may be personal to me, but I have never found this to be the case. I always have way more data than I have time to publish; I currently have over 18 papers in draft, as I struggle to find solid blocks of writing time. This could be because I find the planning of experiments and data collection an adventure. The writing is stressful as I’m always trying to fit something that requires focus and blocks of time around a dozen other tasks. I often don’t have the mental space to enjoy it.

What about the publishing process?

There are some main stages to paper drafting and submission which are worth bearing in mind:

  • Journal and editor selection
  • Drafting
  • Co-author edits
  • Submission
  • Revision
  • Hopefully publication (if not, back to the beginning)

Many authors jump straight into drafting without really spending enough (or any) time on journal selection. Many PhD students don’t do this as they see their supervisors just jumping straight in. That’s normally because supervisors know a lot more about the publishing landscape and so already know the background.

Why is journal selection important?

Manuscript publishing is like any other form of publishing. You need to choose the right journal for your content. Every journal will have specific topic areas they are interested in. They will also have specific formats they will want you to follow in terms of length, numbers of figures and tables, as well as referencing style. If you start drafting without having an idea of where you are going to submit you will often not put the correct emphasis on your writing to get it into your journal of choice. You will also waste time you could spend on other things restructuring what you have already written.

Hint 1: Even title choice is linked to your journal of choice. Do they like long titles? Do they appreciate a witty title to draw readers in?

Hint 2: Go through similar articles to the one you are planning to write and look at the length of different sections in order to understand where the emphasis lies. Do they have a long methods section? Do they focus on discussion?

If you get the research right you will save yourself a tonne of time later on with re-writes and rejections.

What about co-authors?

It is obviously crucial to include your co-authors but I have also learnt that it can be helpful to pick the point at which you circulate to them. If you include everyone during drafting, you can end up with too many different points of view that mean you end up with a manuscript that is unclear or meandering. I’ve learnt to include a few key people, get it to a publishable stage and then circulate. Pick your key people carefully if you are working in a multidisciplinary team so that you get the benefit of their perspective, but don’t get too distracted from the agreed paper themes.

Finally. Don’t get disheartened.

Rejection is just part of the process. Papers will become stronger for revisions and contribute more, thus having more impact. Remember that the criticisms are of the manuscript. They are not criticisms of you. My method for dealing with reviews is to open the email and read the comments. I then close the email down, go and make a double gin and tonic and wait 48 hours before responding. The memory of the comments is never as bad as I thought and once you take the emotion out of it you can just crack on.

All opinions in this blog are my own