Problems of scientific publications

For people not familiar with the scientific process, the importance of scientific articles may not be very clear. Publication is not only the dissemination of information, but also the verification of research results, and also the most important measure of the effectiveness of scientific work.



The importance of publications for scientists is described by the expression " Publish or perish " - "Publish or die." It is all kinds of bibliometric indicators that are the basis for obtaining grants, climbing the career ladder and, ultimately, scientific success. And therefore, the problems of scientific publications negatively affect the entire science as a whole.

Introduction - Why Scientific Publications?

How does the scientific publishing system work? It all starts with the fact that you have a completed study that you want to publish. You should prepare text and illustrations according to the model adopted in the magazine where you want to send your article. At the beginning, the manuscript goes to the editor, he may refuse you right away, considering the article uninteresting, not new enough or not suitable for the topic of the journal. If your manuscript passes this first filter, the editor submits it for review by 2-3 scholars in the relevant field. This stage is called peer review and is the main stage of scientific publication. Reviewers carefully review your work and provide feedback, and suggest changes and additional experiments. If all reviewers rated the manuscript negatively, the journal will not publish the article.But even with a positive assessment, changes usually need to be made. After that, the article is sent to the second round of the review. This can be repeated several times until the reviewers are satisfied with the final result.



Why is such a system needed? It ensures that experts have reviewed the manuscript and that all required standards have been met. This is exactly what makes the articles different from preprints (for example, on arxiv.org and biorxiv.org), which have not yet been reviewed, and the quality of scientific work cannot be guaranteed.



Such a detailed introduction is needed so that you understand how a scientific publication differs from, for example, an article on Habré.



What are the problems with scientific publications? There are a great many of them. And they have a tangible effect on the entire research process, because, as I said at the beginning, articles are the main measure of the effectiveness of scientists' work. Interestingly, many of the problems are inherited from ancient times, when there was no Internet and the ways of disseminating information were radically different.



So, let's begin. The most visible problem is of course

Money

At the moment, there are two main ways to "monetize" a scientific journal: subscription (paywall) and open access (open access). In the first case, you need to buy a separate article or an annual subscription to the magazine. Open access allows anyone from anywhere to download an article on the Internet.



Let's consider both options in more detail.

Paid access (paywall)

This is one of those features that emerged a hundred years ago. When paper magazines were the only way to spread information, they had to be printed and sent out. People and organizations subscribed to them like any other periodical. Some scientific journals still have paper versions, but more often than not their only use is to leaf through at lunch.



It is obvious that now most of the scientific publications are distributed via the Internet. And when working with information, automatic search systems are used (for example, Scopus or Pubmed). That is, now I am not looking for an article in a magazine, I am looking for an article by keywords or authors. But often, having found the research you need, you can run into a paywall.





^{Self-ironic article in Nature ...}



Many scientific journals now require a subscription to access the article. Since the cost of a single article can go up to several tens of dollars, and each employee needs access to hundreds of publications, publishers agree with scientific and educational institutions to subscribe to a large package of their journals for a long time. In this case, employees can access all the magazines of the publisher. However, even "with a discount" for a large volume, the cost is still very high. Exactly how much money each organization spends on subscriptions is unknown, the publishers do not disclose this data. But universities in many countries, including, for example, Germany, refuse subscriptions due to its high cost. Our institute stopped subscribing to Elsevier, and as a result we lost legal access to a large number of magazines. In general, many scientific institutions do not like Elsevier,as they want a lot of money for a subscription.



Another inconvenience is that the subscription only works on the IP addresses of the university or institute. That is, you cannot read articles from home or while traveling without a VPN. And sometimes access to magazines is limited to computers in the library only! This is very inconvenient, and that is why even those scientists who have legal access to journals often use SciHub.







The main problem is that most people cannot access the articles. But more than half of scientific research is funded by the state, that is, by our taxes. And it is often impossible to read the article with the results. Neither the school, nor the journalist, nor the ordinary interested person have a subscription, and therefore access to the article.

Open access

The problems of “monetizing” by subscription have become apparent to the scientific community for a long time. And with the development of the Internet, magazines appeared that only go online (one of the most famous such publishers is PLOS). Therefore, monetization has also changed. Now the person who publishes it, that is, the researcher, pays for the article. And the prices will "pleasantly" surprise you . An article (depending on the magazine) costs from $ 1,500 to $ 5,700. 





It is important to understand here that this is only a payment for publication. University money or grants are spent on research, and the journal receives ready-made content for free. You can compare this system with youtube, as if bloggers had to pay to upload a video. The most interesting thing is that in the end it is the same person who pays before (the institution, and therefore most often the state). But in the open access system, everyone can read the full text of the article. This change greatly simplifies the interaction with the logs. Therefore, many support the transition of all articles to open access ( program OA2020 ).



However, in this system, institutions still spend huge sums of money to publish research. For example, the UK will save 8 million euros if it puts all articles in the public domain - 45 million versus 53 million they are paying now. Only 44 million is also a rather big amount.





^{What constitutes the cost of the article. There are two important points to note in this figure. First, the reviewer's work takes up a third of the cost, but it is not paid. Secondly, the profit of scientific journals is 15-20%! And according to some estimates, it can reach 35%.}

A large number of different magazines and standards

Another important issue is that all magazines and publishers have different standards. This is especially noticeable if the article is rejected by the editor. He does this without a detailed analysis of the article, focusing only on the interest and novelty of the research, as well as on how the article fits the format and topic of the journal.



An editorial review of an article can take from a week to a month. If an editor rejects an article, the authors submit it to another journal. Typically, this will involve drastic changes to the article formatting. The number of characters in the text, the number of pictures, and even the ability to use color pictures, the number of literature sources and much more need to be changed when changing the magazine. The change can be quite time-consuming, for example, reducing the text from 6,000 words to 4,000 and even preserving the meaning and accuracy of the wording.



If the article does not fit in the next magazine, you will need to change it again. Sometimes up to 5-6 of these innings can occur. And for each journal, time is wasted changing the text and reviewing by the editor. In particularly unfortunate cases, it will take several months and quite a few useless text changes before the article even gets to review. It is important to note that these changes do not affect the quality of the article in any way, they are only needed to fit the format of the journal.



It is worth noting that, despite the lack of reasonable size limits for modern magazines - they are published online - the number of different magazines is only growing. All publishers create more and more magazines. At the same time, as noted earlier, such a variety is not justified by the emergence of new areas of knowledge - modern scientists do not read the entire journal, but look for the necessary articles in special databases (such as Scopus or PubMed).

Editor's rating

I will add a few comments on the editor's assessment of the article. The editor is one person who decides whether or not the journal will publish your work at all. He can refuse at any stage for any reason. The editor is interested in the most successful articles that will be actively cited. This means that editors select articles according to certain characteristics, while other articles, on the contrary, find themselves in a very disadvantageous position. Editors are attracted by hype topics, notable names of scientists, correspondence of the results to the current course, new methods and large amounts of data that other researchers can use. Conversely, narrow fields of science, unknown authors and simple methods can become a negative factor. It should be noted that often this has nothing to do with the quality of the research, but only reflects its topic.



The editor makes a decision after reading the article (or part of it), but you usually will not receive detailed comments from him. If the editor rejects the article, then often the reason is insufficient scientific novelty, inconsistency with the topic of the journal. The editor's work is completely opaque: you do not know whether he has read the entire article or just an abstract, on the basis of what criteria he evaluates the novelty of the research, and so on. It is also impossible to argue with the editor if the article is rejected by the editor, usually it is sent to another journal.



It can also be noted here that the personal relationship of the editor with the authors of the article can sometimes influence his decision.

Review (who, why and how much)

Peer review is the most important part of the scientific publication process. It is through peer review that we can trust the research result to some extent. Without checking the article, one could simply upload it to Habr (and, perhaps, the coverage would be even higher). Therefore, it is surprising that all the cost of publication goes to the functioning of the journal and the profit. Reviewers work for free and do not receive anything for their work. Also, reviewers do it in their free time, because they themselves are scientists, and they need to conduct their research. Such a system works due to collective responsibility: today you review the article, and tomorrow you will need the review. But the fact remains that the journal receives free of charge not only content, but also the competence of reviewers.



How does the review work? The editor, having looked at your article, selects suitable reviewers - scientists who do not collaborate with the authors of the article, but who work in the same field. Reviewers receive the text from the editor and send their feedback to him. The identity of the reviewers is not disclosed to the authors of the article, all correspondence is anonymous.





After carefully studying the article, the reviewers suggest changes that need to be made - corrections to the wording, additional controls and experiments. A good reviewer can greatly improve an article. A lot depends on who is reviewing your article. How closely will the reviewer check the calculations? How to carefully look at additional materials and databases? Will check the statistical tests and methods used? Since there are always two or three reviewers, the role of each is very important.



There is also room for abuse. The reviewer may ask you to cite their article or write negative reviews because they disagree with your position without relying on the actual quality of the research.



Authors can debate with reviewers - it is not necessary to fully accept all their changes. But it is almost impossible to publish the article in its original form, completely without corrections. In most journals, corrections are made to the text of the article, and correspondence with reviewers is not available to anyone except the authors. That is, you cannot find out what exactly was changed by the reviewers.





^{I removed almost all comics under the spoiler, but this picture perfectly explains what happens after the review.}

Waiting time

What are the authors doing while the review is in progress? But nothing, they are waiting for feedback. Of course, at this time, research continues on the following projects, but very little is done under the article on the audit of experiments.



How long does it take? The review of the article by the editor (before sending to the reviewers) takes from one week to a month. Sometimes it is worth reminding the editor about your article, then the process is accelerated. The audit can last from one month to six months. It should be understood that the work of the reviewer is not paid, so many refuse to conduct a review. In addition, it can be difficult to find a reviewer without a conflict of interest and an expert in the field of the article. Therefore, in some cases, it takes a long time to find reviewers. In exceptional situations, the editor cannot find reviewers, and this further delays the process.



Why might this be a problem? It should be noted here that articles are often necessary for reports on grants, postgraduate students need publications for defense, sometimes the author of an article soon changes his place of work - often there is one or another deadline by which the research must be published. It is clear that it is wrong to publish an unfinished article. But the spread in terms of up to six months makes it very difficult to plan work.



The review can take several rounds. After changes are made, reviewers may ask for additional experiments. As a result, the review can take several years.





^{And it's not a joke.}

Reproducibility

This is a very big problem for the scientific process in itself. The "crisis of reproducibility", especially in the medical sciences, is one of the most important obstacles to the rapid and qualitative development of science. This topic has already been analyzed in sufficient detail on Habré ( one and two ), so here we will only discuss the influence of the current publication system on this.





^{Factors making it difficult to reproduce the results. Dark arrows are something that can be corrected during review and publication. Light arrows indicate factors indirectly related to publications.}

And this influence is great. In the picture, arrows indicate the main reasons for low reproducibility, which are associated with the operation of the magazines. It is the publisher of the article who must check the presence of the code, the availability of data in the database, the description of the methods and all the necessary materials. However, many journals severely limit the scope of the article, and detailed methods simply do not fit into the main text. The authors put it in Supplementary materials, but they are much less controlled by the editor and reviewers, they are more difficult to find by reference, and they can be in any format - everything is at the discretion of the authors.





Interestingly, it is in the coolest magazines that articles are often published where methods are not described in sufficient detail, links to sources do not work, the code is not available, and the most important files are not in the databases. 





The very structure of the articles and the publishing process leads to low reproducibility - if you do not know what the authors did, then you cannot repeat it. But two other facts play an even greater role.



Publish or perish. I wrote this at the very beginning, this is how all science works now. This forces scientists to search for the most hyped results and publish them as quickly as possible. The winners are those who write a lot of articles, not those who do qualitative research. The pressure of seeking grants leads to the fact that it is more profitable to publish more often, even if the results cannot be reproduced later. It doesn't matter anymore - the article has been published. And in some cases, this can lead to fraud: both small shenanigans with pictures (like those found on Retraction Watch), and forgery of the results of entire articles.





And no less important fact - no one is interested in repeating other people's experiments. Such works are very difficult to publish and it is almost impossible to get a grant for this. Everyone wants new research. It is especially beneficial for magazines to publish articles that get as many citations as possible. Repetition of known results will generate much less interest. Interestingly, reproduction of other people's research is difficult to publish regardless of the results - you confirmed the previous article or denied it.

Using articles as a measure of coolness

A very important problem of the modern system is not directly related to articles. As I said at the beginning, publications are a measure of the success of scientific work and the main criterion for receiving grants. This is not a problem in itself. The question is implementation - how to evaluate the coolness of the article? There are many different bibliometric indicators. The most important are the number of citations of the article and the impact factor of the journal.



Article Citation Count shows how many times your research has been mentioned in other peer-reviewed articles. This is an important characteristic that assesses the relevance of research in the scientific community, a fairly objective indicator, but not without drawbacks. The main problem with this parameter is that the citation of an article accumulates only after a few months (for a very prominent work) or several years (the usual version). That is, immediately after the publication of a publication, its importance and influence cannot be estimated by the number of citations.



The impact factor of a journal is the average citation for a year of all articles that came out in the previous two years. That is, it is the average measure of citation of articles in the journal. What are the problems with this parameter? The mean is very sensitive to outliers, so the few most cited articles have the greatest impact on the impact factor.



But the main problem is that the impact factor of a journal says very little about the quality of a particular article in that journal. However, this parameter is very often used specifically for evaluating articles. Impact Factor is immediately available, and many grants use it to give an article some quality value.



What is the problem with this approach? As you already know, the decision to publish in the journal (or refuse) is made by one person - the editor. And if he finds the article uninteresting, not hype enough or inappropriate for any other parameter, the publication will be refused. This means that the article will be submitted to another journal with a lower impact factor. That is, the content of the work has not changed, but its assessment has decreased, and not because of the quality of the study. 

Information exchange and interaction

In the end, I would like to discuss not the most important, but rather indicative problem. Indeed, in the 21st century, scientific articles are not very different from what they were a hundred years ago. Yes, we can download the article from the Internet (if you have access to it), but it's still a text document with pictures. The quality of the images has improved (although in some magazines you still have to pay extra for color pictures), hyperlinks to literature sources have appeared (which for some reason do not work very conveniently), but that's all. And you read the text laid out in columns, as it was at the end of the 19th century.





^{Article from the very first issue of the journal Nature. Those familiar with this magazine will recognize the Letter format. It is still used today, although it is completely inconvenient for publishing modern scientific data.}

The main function of the magazine now is pdf hosting. This is less than what, for example, Wikipedia does. At the same time, for such a service you need to pay several thousand dollars for one document - probably the most expensive hosting in the world.



It is clear that I am exaggerating, but nowadays there are so many opportunities for reviews, ratings and aggregation of articles. You can provide a platform for scientists to directly discuss articles right on the site, and bring them there. If we take Habr as an example, then the comments are often not inferior to the article itself.



Now some magazines have the ability to comment, but almost no one uses it. On the one hand, because this function is not very convenient and is implemented worse than in the usual applications, on the other hand, because the journal does not spend energy on attracting scientists and developing such a platform.



It seems to me that modern experiences from different fields that organize a large amount of data - social networks, directory sites (like Kinopoisk), Wikipedia and many others - will change the way we exchange scientific information. Publishers are making timid steps in this direction, but they are satisfied with the current situation, they do not need to change anything.





At the end, I would like to invite everyone to a discussion. Write down if you've had any negative experiences with magazines. Or maybe you are completely satisfied with the modern system? I would be glad to hear your opinion on scientific publications in the comments.



And on how some of the problems described are being solved and what else can be done and what are the prospects for development, we will talk to you next time ( second part of the article ).

Acknowledgments

Many thanks to Olga Zolotareva for discussions and ideas for this article. Thanks to Sofya Kamalyan for help in checking the text. And also to all colleagues with whom we discussed the problems of scientific publications.