A blog from the Centre for Research Ethics & Bioethics (CRB)

Tag: research ethics (Page 2 of 8)

Predatory conferences

If you are an academic, you have probably noticed that you are getting more and more unexpected invitations via e-mail to participate as a speaker in what are presented as scientific conferences. The invitations can be confusing, as they are often not even in your subject area. But sometimes they get it right and maybe even mention your latest publication, which is praised in general terms. What is happening?

Publication ethics is one of many research areas at CRB. In recent years, we have researched (and blogged about) so-called predatory journals, which can lure academics to publish their studies in them for a considerable fee, which will make the article openly available to readers. Open access is an important trend in science, but here it is exploited for profit without regard for academic values. Predatory journals are often generously multidisciplinary and the promised “effective” peer review is just as generous, in order to capture as many paying authors as possible.

The steady stream of conference invitations to academics reflects the same dubious type of activity, but here the profit comes from conference fees and sometimes also from arranging accommodation. Within publication ethics, one therefore also speaks of predatory conferences. What do we know about these conferences? Is there any research on the phenomenon?

The first systematic scoping review of scholarly peer-reviewed literature on predatory conferences was recently published in BMJ Open. The overview was made by four researchers, Tove Godskesen and Stefan Eriksson at CRB, together with Marilyn H Oermann and Sebastian Gabrielsson.

The review showed that the literature on predatory conferences is small but growing, 20 publications could be included. Almost all of the literature in the review described characteristics that may define predatory conferences. The most cited characteristic was the spam email invitations, with flattering language that could contain grammatical errors and be non-scientific. Another distinguishing feature described was that the organization hosting the conference was unknown and used copied pictures without permission. Finally, high fees, lack of peer review, and multidisciplinary scope were also mentioned.

Why do researchers sometimes attend predatory conferences? Possible reasons cited in the literature were the focus on quantity in academic research dissemination, falling victim to misleading information, or the attractive and exotic locations where these conferences are sometimes held. The easy submission and review process and the opportunity to participate as a chair or invited speaker were also mentioned as possible attractions. Personal characteristics such as inexperience, naivety, ignorance, vanity and indifference were also mentioned.

Consequences of attending predatory conferences were described in only one of the publications, an interview study with conference participants. Their stories were marked by disappointments of various kinds. Small overcrowded conference rooms, poorly organized conference facilities, deviations from the conference program that could be reduced by a whole day, reputable keynote speakers announced in the program were absent, and the organizers were hard to reach as if the whole event was remote controlled. Participants were sometimes forced to book their accommodation through the organizers at double cost, and they could also experience that the organizers stole their identities by using their pictures and personal information as if they were part of the conference team. Many participants were so disappointed that they left the conferences early, feeling like they never wanted to attend any conferences again.

The literature also suggested various countermeasures. Among other things, education for all researchers and mentoring of junior academics, published lists of predatory conferences and their organizers, accreditation systems for conferences, and checklists to help identifying predatory conferences. It was also stated that universities and research funders should review their ways of assessing the qualifications of researchers seeking employment, promotion or funding. Attending predatory conferences should not be an asset.

For more details and discussion, read the systematic review here: Predatory conferences: a systematic scoping review.

Another countermeasure mentioned in the literature was more research on predatory conferences. This is also a conclusion of the overview: both empirical and analytical research should be encouraged by funders, journals and research institutions.

Hopefully, these staged conference rooms will soon be empty.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Godskesen T, Eriksson S, Oermann MH, et al. Predatory conferences: a systematic scoping review. BMJ Open 2022;12:e062425. doi: 10.1136/bmjopen-2022-062425

This post in Swedish

We want solid foundations

Where to publish and where not to publish in bioethics – the 2022 list

We have seen “predatory” publishing take off in a big way and noticed how colleagues start to turn up in the pages of some of these journals. While many have assumed that this phenomenon mainly is a problem for low-status universities, there are strong indications that predatory publishing is a part of a major trend towards the industrialization of misconduct and that it affects many top-flight research institutions (see Priyanka Pulla: “In India, elite institutes in shady journals”, Science 354(6319): 1511-1512).

This trend, referred to by some as the dark side of publishing, needs to be reversed. Thus we published this blog post in 2016. This is our fifth annual update (the first version can be found here). At first, we relied heavily on the work of Jeffrey Beall, a librarian at the University of Colorado, who run blacklists of “potential, possible, or probable” predatory publishers and journals. His lists have since been removed but are kept by others and they can also be found archived.

The latest effort to create a thorough list of predatory outlets comes from Cabells, who distinguish around 70 different unacceptable violations and employs a whole team reviewing journals. These lists are not, however, the final say on the matter, as it is impossible for one person or a limited group to judge reliably actors in every academic discipline.

A response of gatekeeping needs to be anchored in each discipline and the scholars who make up that discipline. As a suitable response in bioethics, we have chosen to, first, collect a few authoritative lists of recommended bioethics journals that can be consulted by anyone in bioethics to find good journals to publish with.

For our first post, we recommended a list of journals ourselves, which brought on some well-deserved questions and criticism about criteria for inclusion. Unfortunately then, our list ultimately drew attention from other parts of the message that we were more concerned to get across. Besides, there are many other parties making such lists. We, therefore, have dropped this feature. Instead, we have enlarged the collection of good journal lists to the service of our readers. They are all of great use when further exploring the reputable journals available:

It is of prime importance to list the journals that are potentially or possibly predatory or of such a low quality that it might be dishonoring to engage with them. We have listed all 50 of them alphabetically (one new entry for 2022, one has ceased operation and been removed), and provided both the homepage URL and links to any professional discussion of these journals that we have found (which most often alerted us to their existence in the first place).

Each of these journals asks scholars for manuscripts from, or claims to publish papers in bioethics or related areas (such as practical philosophy). They have been reviewed by the authors of this blog post as well as by a group of reference scholars that we have asked for advice on the list. Those journals listed have unanimously been agreed are journals that – in light of the criticism put forth and the quality we see – we would not deem acceptable for us to publish in. Typical signs as to why a journal could fall in this category, such as extensive spamming, publishing in almost any subject, or fake data being included on the website etc., are listed here:

We have started to more systematically evaluate the journals against the 25 defining characteristics we outlined in the article linked to above (with the help of science and technology PhD students). The results will be added when they exist.

We would love to hear about your views on this blog post, and be especially grateful for pointers to journals engaging in sloppy or bad publishing practices. The list is not meant as a check-list but as a starting point for any bioethics scholar to ponder for him- or herself where to publish.

Also, anyone thinking that a journal in our list should be given due reconsideration might post their reasons for this as a comment to the blog post or send an email to us. Journals might start out with some sloppy practices but shape up over time and we will be happy to hear about it. You can make an appeal against the inclusion of a journal and we will deal with it promptly and publicly.

Please spread the content of this blog as much as you can and check back for updates (we will do a major update annually and continually add any further information found).

Note to readers: The list contained on Stop Predatory Journals referred to below has been down for while. From 2022 any reference to journals/publishers being included on SPJ refers to their previous inclusion. We will gradually check for inclusion in the most prominent list presently available, Cabells’ Predatory Reports, as a alternative.

WHERE NOT TO PUBLISH IN BIOETHICS – THE 2022 LIST

  • Advanced Humanities & Social Sciences (Consortium Publisher)
    Critical remark (2018): It has been claimed that behind this journal you find OMICS, the most-ever discussed publisher of this kind, see http://ottawacitizen.com/news/local-news/predatory-publisher-expanding-empire-in-canada. The only article published in 2016 is very badly edited, all the references are lost in the text and the paper would not pass an exam at our departments.  2017 volume is again only one article. The publisher is listed on SPJ.
    Critical remark (2022). After a complaint from the publisher, we have checked the latest volume. An article like this one shows no evident editorial work on the paper at all, so we still regard the journal to be a low quality outlet for research.
  • Advances In Medical Ethics  (Longdom Publishing)
    Critical remark (2019): When asked, one editor attest to the fact that his editorship was forged. Publisher was on Beall’s list and is now listed at Cabells with 5 violations. A thorough review December 2019 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
    Update: A more recent review (2022) concludes that it exhibits about 17 such criteria. How an e-mail exchange with this publisher can turn out is shown here.
    Update (2023): A thorough review in August concludes that it exhibits at least 10 of the 25 criteria for “predatory” journals
  • American Open Ethics Journal (Research and Knowledge Publication)
    Critical remark (2019): Listed on Cabells with 7 violations.
    Update: A thorough review February 2020 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
    Update: Another thorough review December 2023 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • Annals of Bioethics & Clinical Applications (Medwin Publishers)
    Criticism 1 
    Critical remark (2019): Publisher was on Beall’s list and is on many other lists of these journals. They say that they are “accepting all type of original works that is related to the disciplines of the journal” and indeed the flow chart of manuscript handling does not have a reject route. Indexed by alternative indexes.
    Critical remark (2020): Listed on Cabells with 5 violations. A thorough review October 2020 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • Austin Journal of Genetics and Genomic Research (Austin Publishing Group)
    Criticism 1 │Criticism 2 │Criticism 3
    Critical remark (2017): Spam e-mail about special issue on bioethics; Listed by SPJ; Romanian editorial member is said to be from a university in “Europe”; Another editorial board member is just called “Michael”; APG has been sued by International Association for Dental Research and The American Association of Neurological Surgeons for infringing on their IP rights. Student reviews concludes the journal is not suitable to publish in, one finding that the journal exhibits at least 16 of the 25 criteria for “predatory” journals.
    Critical remark (2019): Listed by Cabells with 10 violations.
    Critical remark (2021): A thorough review concludes that the journals exhibits at least 13 of the 25 criteria for “predatory” journals.
  • British Open Journal of Ethics (British Open Research Publications)
    Critical remark (2019): Listed by Cabells with 6 violations.
    Critical remark (2022): A thorough review concludes that the journal exhibit many criteria for “predatory” journals, for example that no editorial board exists and the journal is not indexed, and that it is strongly recommended to avoid “publishing” with this journal.
  • Creative Education (Scientific Research Publishing – SCIRP)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Listed by SPJ; They claim misleadingly to be indexed by ISI but this relates to be among cited articles only – they are not indexed. A thorough review May 2017 concludes that it exhibits at least 5 of the 25 criteria for “predatory” journals.
    Update (2022): Listed on Cabells with 3 violations.
    Update (2023): A new review in 2023 concludes that it exhibits 11 of the 25 criteria for “predatory” journals.
  • Eastern European Scientific Journal (East European Research Alliance)
    Critical remark (2017): Listed by SPJ; Criticised by Beall for having a bogus editorial board; Claims to be indexed by ISI but that is not the well-known Institute for Scientific Information (now Thompson Reuters), but rather the so-called International Scientific Indexing.
    Update: Thorough reviews November 2018 and February 2019  conclude that it exhibits at least 13 or 14 of the 25 criteria for “predatory” journals.
    Update (2022): Listed on Cabells under its old name (“East”) with 11 violations.
  • Ethics Today Journal (Franklin Publishing)
    Critical remark (2019): Listed by Cabells with 9 violations.
    Update: “www.franklinpublishing.net expired on 06/21/2022 and is pending renewal or deletion”
  • European Academic Research (Kogaion Publishing Center, formerly Bridge Center)
    Critical remark (2017): Listed by SPJ; Uses impact factor from Universal Impact Factor (now defunct); A thorough review May 2017 concludes that it exhibits at least 15 of the 25 criteria for “predatory” journals.
    Update: A thorough review October 2022 concludes that it exhibits at least 14 of the 25 criteria for “predatory” journals.
  • European Scientific Journal (European Scientific Institute)
    Critical remark (2017): Listed by SPJ; Use of alternative indexes. A thorough review May 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
    Update: A thorough review November 2022 concludes that it exhibits at least 6 of the 25 criteria for “predatory” journals. Another review in May 2023 concludes that it exhibits at least 6 of the 25 criteria.
  • International Journal of Advances in Social Science and Humanities
    Critical remark (2017): Listed by SPJ; Impact factor given by  Global Impact Factor. A thorough review March 2019 concludes that it exhibits at least 10 of the 25 criteria for “predatory” journals.
  • International Journal of Contemporary Research & Review
    Critical remark (2017): Listed by SPJ; Indexed by Index Copernicus; Despite claims they seem not to be indexed by either Chemical Abstracts or DOAJ. 
    A thorough review June 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Current Research
    Critical remark (2017): Listed by SPJ; Uses IF from SJIF and Index Copernicus and more. It wrongly claims to be indexed by Thomson Reuters, ORCID and having a DOI among other things. A thorough review January 2018 concludes that it exhibits at least 12 of the 25 criteria for “predatory” journals.
    Update (2023): A thorough review October 2023 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • International Journal of Current Research and Academic Review (Excellent Publishers)
    Critical remark (June 2018): Listed by SPJ and Cabells because of misleading claims about credentials, metrics, and too quick review; alternative indexing; publishes in almost any field imaginable; the editor -in-chief is head of the “Excellent Education and Researh Institute” (sic) which does not seem to exist even when spelled right?
    Update: A thorough review in December 2019 concludes that it exhibits at least 12 of the 25 criteria for “predatory journals”.
    Update (2022): A thorough review in July concludes that it exhibits at least 13 of the 25 criteria for “predatory journals”.
  • International Journal of Ethics (Nova Science Publishers)
    Criticism 1
    Critical remark (2022): The article on Nova at Wikipedia notes that librarians have been critical of this publisher; A Ms. Alexandra Columbus is both the owner of, business manager and customer contact for Nova.
    Update (2023): A thorough review in May concludes that it exhibits at least 9 of the 25 criteria for “predatory journals”.
  • International Journal of Ethics & Moral Philosophy (Journal Network)
    Critical remark (2017): Listed by SPJ; Publisher was criticized by Beall when launching 350 journals at once; After several years not one associate editor has signed up and no article has been published; No editorial or contact details available. Thorough reviews in May 2019 and February 2020 conclude that it exhibits at least 10 to 12 of the 25 criteria for “predatory journals”.
    Update (2022): Does not seem to be online.
  • International Journal of Ethics in Engineering & Management Education
    Critical remark (2019): Papers from almost any field; Claims to have a 5.4 Impact factor (from IJEEE); Indexed by GJIF etc. A non-existent address in “Varginia”, US (sic!); Open access but asks for the copyright; Claims to be indexed in Scopus can’t be verified.
    Update (2020): Thorough reviews February 2018 and February 2020 conclude that it exhibits at least 16-17 of the 25 criteria for “predatory” journals. Listed by Cabells with 11 violations found.
  • International Journal of Humanities and Social Science (Centre for Promoting Ideas)
    Criticism 1Criticism 2Criticism 3 │ Criticism 4
    Critical remark (2019): The chief editor listed in April 2014  is a deceased person (2018). A thorough review in April 2019 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Science Invention
    Criticism 1 
    Critical remark (2017): Listed by SPJ and is on many other lists of blacklisted journals; An IF of 4.5 given by African Quality Centre for Journals; Open access but asks for the copyright; Publishes any subject; Says that the journal is indexed in DOAJ which it does not seem to be. 
    Update: A thorough review February 2018 concludes that it exhibits at least 13 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Science Research
    Critical remark (2017): Listed by SPJ and Cabells; Claims an IF of 5.22 (by “Research Journal Impact Factor“); Despite title from India; Alternative indexing; Thorough reviews in February 2018 and February 2020 conclude that it exhibits at least 10-13 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Sciences
    Critical remark (2017): Listed by SPJ; Has an amazing fast-track review option for $100 that guarantees “the review, editorial decision, author notification and publication” to take place “within 2 weeks”. “Editors” claim that repeated requests to be removed from the list of editors result in nothing. Thorough reviews in  February and June 2018 conclude that it seems to exhibit at least 7 to 10 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities & Social Studies
    Critical remark (2017): Listed by SPJ; IF from International Impact Factor Services; States that there “is no scope of correction after the paper publication”.
    Critical remark (2018): They write that the “review process will be completed expectedly within 3-4 days”.
    Critical remark (2020): A thorough review in October 2020 concludes that it seems to exhibit at least 7-8 of the 25 criteria for “predatory” journals.
    Update (2024): A thorough review concludes that it seems to exhibit at least 10 of the 25 criteria for “predatory” journals.
  • International Journal of Legal, Ethical and Regulatory Issues (Jacobs Publishers)
    Criticism 1
    Critical remark (2019): Spamming with invitation to publish. They are unsure of their own name; in the e-mail they call the journal “International Journal of Legal, Ethical and Regulatory Affairs“! Publisher listed on SPJ. Editor-in-chief and editorial board are missing. Claims that material is “written by leading scholars” which is obviously false.
    Update (2022): Publisher cannot be found any longer.
  • International Journal of Philosophy (SciencePG)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Listed by SPJ; Alternative indexing and also IF from Universal Impact Factor (now defunct); Promises a two-week peer review.
    Update: Thorough reviews in April and November 2018 conclude that it seems to exhibit at least 10 or 8 of the 25 criteria for “predatory” journals and also find obvious examples of pseudo-science among the published articles.
    Update (2023): A thorough review in October concludes that it seems to exhibit at least 6 of the 25 criteria for “predatory” journals.
  • International Journal of Philosophy and Theology (American Research Institute for Policy Development) Criticism 1Criticism 2 │ Criticism 3
    Critical remark: A thorough review in June 2018 concludes that “there are grounds to believe that the American Research Institute never intended to create a serious scientific periodical and that, on the contrary, its publications are out-and-out predatory journals.”
    Update (2022): A thorough review in June concludes that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals. However, the website could not be accessed on June 21.
  • International Journal of Public Health and Human Rights (Bioinfo Publications)
    Criticism 1
    Critical remark (2017): Listed on SPJ; On many other lists of predatory journals and have an IF from Index Copernicus.
    Update (2022): Not accessible in June.
  • International Journal of Research in Humanities and Social Studies (Sryahwa Publications)
    Critical remark (2018): Listed on SPJ; Open access but asks for the copyright. A thorough review in April 2018 concludes that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals.
    Update (2022): A June review again confirmed that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Social Science and Humanities Research (Research Publish Journals)
    Critical remark (2017): Listed on SPJ; On their homepage they state that in order to get a high IF their journals are “indexed in top class organisation around the world” although no major index is used.
    Update: A thorough review in 2020 concludes that it seems to exhibit at least 14 of the 25 criteria for “predatory” journals.
  • International Open Journal of Philosophy (Academic and Scientific Publishing)
    Critical remark (2017): Listed on SPJ and was heavily critized on Beall’s blog; The editorial board consists of one person from Iran; Although boosting 12 issues a year they have published only 1 article in the journal’s first four years.
    Update: A thorough review March 1 2017 concludes that it exhibits 17 of the 25 criteria for “predatory” journals and one in March 2019 that it exhibits at least 13 criteria.
  • International Researchers
    Critical remark (2017): Listed on SPJ; Indexed by e.g. Index Copernicus; Claims that it is “Monitor by Thomson Reuters” but is not part of the TR journal citation reports; Several pages are not working at time of review; A thorough review April 24 2017 concludes that it exhibits at least 6 of the 25 criteria for “predatory” journals.
  • Internet Journal of Law, Healthcare and Ethics (ISPUB)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Formerly on Beall’s list.
  • Journal of Academic and Business Ethics (Academic and Business Research Institute)
    Critical remark (2017): Listed on SPJ as well as several other; Journal seems uncertain about it’s own name, the header curiously says “Journal of ethical and legal issues”.
    Update 2021: A thorough review May 2021 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Bioethics and Applications (Sci Forschen)
    Critical remark (2018): Brand new journal with no articles yet. Publisher has been criticized for spamming , have a bad record at Scam Analyze, and is listed on SPJ.
    Critical remark (2022): A thorough review March 2022 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Clinical Research & Bioethics (OMICS/Walsh MedicalMedia)
    Criticism 1Criticism 2 │ Criticism 3 │ Criticism 4 │ Criticism 5 │ Criticism 6
    Critical remark (2017): This publisher is listed on SPJ and was taken to court for possible fraud by the Federal Trade Commission in the US (and lost). They are listed by Cabells for 8 violations.
    Update (2022): They now have a new (?) publisher, but still the same Danish editor as before. A thorough review May 2022 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Law and Ethics
    Critical remark (2017): Listed on SPJ; Claims to be on Ulrichs but is not; Claims to be in the Norwegian list and can actually be found there but under its former name (4 years earlier) and with 0 points.
    Update 2019: Seems to have moved to here. Security warnings and denied access makes it impossible to check whether it is the same journal or another one.
  • Journal of Philosophy and Ethics (Sryahwa Publications)
    Critical remark (2019): listed by Cabells for 7 violations.
    Critical remark 2020): A thorough review October 2020 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • Journal of Research in Philosophy and History (Scholink)
    Criticism 1 
    Critical remark (June 2018): Listed on several lists of predatory publishers. They only do “peer review” through their own editorial board, a flowchart states. They claim to check for plagiarism but the first 2018 article abstract run by us through a checker turned out to be self-plagiarized from a book and it looks to have been published many times over. Unfortunately, the next paper checked in the same issue was also published the previous year by another journal listed here…
    Critical remark (March 2021): A thorough review concludes that it exhibits at least 14 of the 25 criteria for “predatory” journals.
    Update 2023: A thorough review concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • Journal of Social Sciences and Humanities (AASCIT)
    Criticism 1Criticism 2Criticism 3
    Critical remark (2019): From law to religion, this journal publishes it all. Though publisher claims to be “American”, it has only two editors, both from India. The list from Cabells includes 13 journals from this publisher. The AASCIT Code of Ethics apparently plagiarizes the INCOSE Code of Ethics.
  • Journal of Studies in Social Sciences and Humanities
    Critical remark (2017): Listed on SPJ; Alternative indexing; Uses several alternative IF providers. A thorough review October 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
    Critical remark (2020): A thorough review October 2020 concludes that it exhibits at least 4 of the 25 criteria for “predatory” journals.
  • JSM Health Education and Primary Health Care
    Spamming with invitation to special issue on ‘Bioethics’. The publisher is listed on SPJ,  and criticized and exposed here. It is indexed by spoof indexer Directory of Research Journals Indexing among others (whose website is now gone, BTW).
    Update (2019): Access denied because of non-secure connection.
    Update (2022): Access now possible again.
  • Medical Ethics and Communication (Avid Science)
    Criticism 1
    Critical remarks (2017): Listed on SPJ; Spamming researchers with offer of eBook publication for $350.
    Update: In June 2022, the journal cannot be accessed online.
  • Nova Journal of Humanities and Social Sciences
    Criticism 1
    Critical remark (2018): This publisher was on Beall’s list; Uses alternative impact factors and indexing; Publishes in less than 30 days; Curiously, it says no fee is charged for publication.
    Update: In June 2022, the journal cannot be accessed online.
  • Open Journal of Philosophy (Scientific Research Publishing – SCIRP)
    Criticism 1 │ Criticism 2 │
    Critical remark (2021): A thorough review March 2021 concludes that it exhibits 6 of the 25 criteria for “predatory” journals.
  • Open Journal for Studies in Philosophy (Center for Open Access in Science)
    Critical remark (2020): Cabells found 8 violations.
    Update: Thorough reviews May-June 2022 concludes that it exhibits at least 8-9 of the 25 criteria for “predatory” journals.
  • Philosophical Papers and Review (Academic Journals)
    Critical remark (2017): Listed on SPJ and blacklisted by the Ministry of Higher Education of Malaysia.
    Update (2021): Latest article in press was accepted the same day it was sent in – and it happened back in 2018!
    Update: A thorough review April 2022 concludes that it exhibits at least 10 of the 25 criteria for “predatory” journals.
  • Philosophy Study  (David Publishing Company)
    Criticism 1Criticism 2
    Critical remark (2017): Listed on SPJ. A thorough review October 2019 concludes that it exhibits approx. 8 of the 25 criteria for “predatory” journals.
    Update: Another thorough review December 2023 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • The Recent Advances in Academic Science Journal (Swedish Scientific Publications)
    Critical remark (2018): Despite the publisher’s name it seems based in India. The only Swedish editor’s existence cannot be verified. Website quality is lacking. Listed on SPJ. A thorough review October 2017 concludes that it exhibits at least 15 of the 25 criteria for “predatory” journals.
    Update: Domain for sale in June 2022.
  • Universal Open Ethics Journal (Adyan Academic Press)
    Critical remark (2019): listed by Cabells for 7 violations.
    Update: Thorough reviews in May 2022 concludes that it exhibits 13 to 20 of the 25 criteria for “predatory” journals.
  • World Journal of Social Sciences and Humanities (Science and Education Publishing, SciEP)
    Criticism 1 │Criticism 2
    Critical remark (2017): Listed on SPJ as well as many other.
    Update: A thorough review in May 2019 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.

End remark:

In light of legal action and threats against people trying to warn others about dubious publishers and journals – see here and here, for example – we want to stress that this blog post is about where we would like our articles to show up, it is about quality, and as such it is an expression of a professional judgement intended to help authors find good journals with which to publish.

Indirectly, this may also help readers to be more discerning about the articles they read. As such it is no different from other rankings that can be found for various products and services everywhere. Our list of where not to publish implies no accusation of deception or fraud but claims to identify journals that experienced bioethicists would usually not find to be of high quality. Those criticisms linked to might be more upfront or confrontational; us linking to them does not imply an endorsement of any objectionable statement made therein. We would also like to point out that individual papers published in these journals might of course nevertheless be perfectly acceptable contributions to the scholarly literature of bioethics.

Essential resources on so-called predatory publishing and open access:

Written by…

Stefan Eriksson, Associate Professor of Research Ethics at Uppsala University, read more about his work on publication, regulation and consent.

and…

Gert Helgesson, professor of Medical Ethics at Karolinska Institute.


We like ethics

Co-authorship when not everyone’s research is included in the paper

Questions about authorship are among the most sensitive professional issues for researchers. Apart from the fact that researchers live and make careers on their publications, it is important for scientific and research ethical reasons to know who is responsible for the content of the publications.

A feature of research that can create uncertainty about who should be counted as a co-author of a scientific publication is that such publications usually report research that has mainly already been carried out when the paper is being written. Many researchers may have contributed to the research work, but only a few of them may contribute to the writing of the paper. Should everyone still be counted as an author? Or just those who contribute to the writing of the paper?

The International Committee of Medical Journal Editors (ICMJE) has formulated a recommendation that creates greater clarity. Simplified, the recommendation is the following. Authorship can be given to researchers who clearly meet four criteria. You must: (1) have made substantial contributions to the research study (e.g., designing the study, or collecting, analysing and interpreting data); (2) have contributed to drafting the paper and revising its intellectual content; (3) have approved the final version of the article; (4) have agreed to be responsible for all aspects of the work by ensuring that issues of accuracy and integrity are investigated.

Furthermore, it is recommended that researchers who meet criterion (1) should be invited to participate in the writing process, so that they can also meet criteria (2)(4) and thus be counted as co-authors.

However, research does not always go according to plan. Sometimes the plans need to be adjusted during the research process. This may mean that one of the researchers has already made a significant research effort when the group decides not to include that research in the writing of the paper. How should co-authorship be handled in such a situation, when someone’s results fall out of the publication?

The issue is discussed by Gert Helgesson, Zubin Master and William Bülow in the journal Science and Engineering Ethics. Considering, among other things, how easily disagreement about authorship can disrupt the dynamics of a research group, it is important that there is an established order concerning authorship, which handles situations such as this.

The discussion in the article is based on an imaginary, concrete case: A research group includes three younger researchers, Ann, Bo and Choi. They have all been given individual responsibility for different parts of the planning and execution of the empirical work. They work many hours in the laboratory. When the research group sees the results, they agree on the content of the article to be written. It then turns out that Ann’s and Bo’s analyses are central to the idea in the article, while Choi’s analyses are not. Choi’s results are therefore not included in the article. Should Choi be included as a co-author?

We can easily imagine Choi contributing to the writing process, but what about criterion (1)? If Choi’s results are not described in the article, has she made a significant contribution to the published research study? Helgesson, Master and Bülow point out that the criterion is ambiguous. Of course, making a significant contribution to a research study can mean contributing to the results that are described in the article. But it can also mean contributing to the research process that leads up to the article. The former interpretation excludes Choi as co-author. The latter interpretation makes co-authorship possible for Choi.

The more inclusive interpretation is not unreasonable, as research is a partially uncertain exploratory process. But do any strong reasons support that interpretation? Yes, say Helgesson, Master and Bülow, who state two types of reasons. Firstly, it is about transparency and accountability: what happened and who was involved? Excluding Choi would be misleading. Secondly, it is a matter of proper recognition of merit and of fairness. Choi worked as hard in the laboratory as Ann and Bo and contributed as much to the research that led to the article. Of course, the purpose of the article changed during the process and Choi’s contribution became irrelevant to the content of the article. But her efforts were still relevant to the research process that led up to the article. She also did a good job as a researcher in the group: it seems unfair if her good work by chance should not be recognized in the same way as the other researchers’ work.

The proposal in the article is therefore that the first criterion for authorship should be interpreted as a significant contribution to the research process leading up to the article, and that this should be clarified in the recommendation.

The article also discusses possible counter-arguments to the more inclusive interpretation of the authorship recommendation. If you want to study the reasoning more closely and form your own opinion, read the article: How to Handle Co-authorship When Not Everyone’s Research Contributions Make It into the Paper.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Helgesson, G., Master, Z. & Bülow, W. How to Handle Co-authorship When Not Everyone’s Research Contributions Make It into the Paper. Sci Eng Ethics 27, 27 (2021). https://doi.org/10.1007/s11948-021-00303-y

This post in Swedish

We recommend readings

When established treatments do not help

What should the healthcare team do when established treatments do not help the patient? Should one be allowed to test a so-called non-validated treatment on the patient, where efficacy and side effects have not yet been determined scientifically?

Gert Helgesson comments on this problem in Theoretical Medicine and Bioethics. His comment concerns suggestions from authors who in the same journal propose a specific restrictive policy. They argue that if you want to test a non-validated treatment, you should from the beginning plan this as a research project where the treatment is tested on several subjects. Only in this way do you get data that can form the basis for scientific conclusions about the treatment. Above all, the test will undergo ethical review, where the risks to the patient and the reasons for trying the treatment are carefully assessed.

Of course, it is important to be restrictive. At the same time, there are disadvantages with the specific proposal above. If the patient has a rare disease, for example, it can be difficult to gather enough patients to draw scientific conclusions from. Here it may be more reasonable to allow case reports and open storage of data, rather than requiring ethically approved clinical trials. Another problem is that clinical trials take place under conditions that differ from those of patient care. If the purpose is to treat an individual patient because established treatments do not work, then it becomes strange if the patient is included in a randomized study where the patient may end up in the control group which receives the standard treatment. A third problem is when the need for treatment is urgent and there is no time to approach an ethical review board and await their response. Moreover, is it reasonable that research ethical review boards make treatment decisions about individual patients?

Gert Helgesson is well aware of the complexity of the problem and the importance of being careful. Patients must not be used as if they were guinea pigs for clinicians who want to make quick, prestigious discoveries without undergoing proper research ethical review. At the same time, one can do a lot of good for patients by identifying new effective treatments when established treatments do not work. But who should make the decision to test a non-validated treatment if it is unreasonable to leave the decision to a research ethical board?

Gert Helgesson suggests that such decisions on non-validated treatments can reasonably be made by the head of the clinic, and that a procedure for such decisions at the clinic level should exist. For example, an advisory hospital board can be appointed, which supports discussions and decisions at the clinic level about new treatments. The fact that a treatment is non-validated does not mean that there are no empirical and theoretical reasons to believe that it might work. Making a careful assessment of these reasons is an important task in these discussions and decisions.

I hope I have done justice to Gert Helgesson’s balanced discussion of a complex question: What is a reasonable framework for new non-validated treatments? In some last-resort cases where the need for care is urgent, for example, or the disease is rare, decisions about non-validated treatments should be clinical rather than research ethical, concludes Gert Helgesson. The patient must, of course, consent and a careful assessment must be made of the available knowledge about the treatment.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Helgesson, G. What is a reasonable framework for new non-validated treatments?. Theor Med Bioeth 41, 239–245 (2020). https://doi.org/10.1007/s11017-020-09537-6

This post in Swedish

We recommend readings

Human enhancement: Time for ethical guidance!

Perhaps you also dream about being more than you are: faster, better, bolder, stronger, smarter, and maybe more attractive? Until recently, technology to improve and enhance our abilities was mostly science fiction, but today we can augment our bodies and minds in a way that challenges our notions of normal and abnormal. Blurring the lines between treatments and enhancements. Very few scientists and companies that develop medicines, prosthetics, and implants would say that they are in the human enhancement business. But the technologies they develop still manage to move from one domain to another. Our bodies allow for physical and cosmetic alterations. And there are attempts to make us live longer. Our minds can also be enhanced in several ways: our feelings and thoughts, perhaps also our morals, could be improved, or corrupted.

We recognise this tension from familiar debates about more common uses of enhancements: doping in sports, or students using ADHD medicines to study for exams. But there are other examples of technologies that can be used to enhance abilities. In the military context, altering our morals, or using cybernetic implants could give us ‘super soldiers’. Using neuroprostheses to replace or improve memory that was damaged by neurological disease would be considered a treatment. But what happens when it is repurposed for the healthy to improve memory or another cognitive function? 

There have been calls for regulation and ethical guidance, but because very few of the researchers and engineers that develop the technologies that can be used to enhance abilities would call themselves enhancers, the efforts have not been very successful. Perhaps now is a good time to develop guidelines? But what is the best approach? A set of self-contained general ethical guidelines, or is the field so disparate that it requires field- or domain-specific guidance? 

The SIENNA project (Stakeholder-Informed Ethics for New technologies with high socio-ecoNomic and human rights impAct) has been tasked with developing this kind of ethical guidance for Human Enhancement, Human Genetics, Artificial Intelligence and Robotics, three very different technological domains. Not surprising, given the challenges to delineate, human enhancement has by far proved to be the most challenging. For almost three years, the SIENNA project mapped the field, analysed the ethical implications and legal requirements, surveyed how research ethics committees address the ethical issues, and proposed ways to improve existing regulation. We have received input from stakeholders, experts, and publics. Industry representatives, academics, policymakers and ethicists have participated in workshops and reviewed documents. Focus groups in five countries and surveys with 11,000 people in 11 countries in Europe, Africa, Asia, and the Americas have also provided insight in the public’s attitudes to using different technologies to enhance abilities or performance. This resulted in an ethical framework, outlining several options for how to approach the process of translating this to practical ethical guidance. 

The framework for human enhancement is built on three case studies that can bring some clarity to what is at stake in a very diverse field; antidepressants, dementia treatment, and genetics. These case studies have shed some light on the kinds of issues that are likely to appear, and the difficulties involved with the complex task of developing ethical guidelines for human enhancement technologies. 

A lot of these technologies, their applications, and enhancement potentials are in their infancy. So perhaps this is the right time to promote ways for research ethics committees to inform researchers about the ethical challenges associated with human enhancement. And encouraging them to reflect on the potential enhancement impacts of their work in ethics self-assessments. 

And perhaps it is time for ethical guidance for human enhancement after all? At least now there is an opportunity for you and others to give input in a public consultation in mid-January 2021! If you want to give input to SIENNA’s proposals for human enhancement, human genomics, artificial intelligence, and robotics, visit the website to sign up for news www.sienna-project.eu.

The public consultation will launch on January 11, the deadline to submit a response is January 25, 2021. 

Josepine Fernow

Written by…

Josepine Fernow, Coordinator at the Centre for Research Ethics & Bioethics (CRB), and communications leader for the SIENNA project.

SIENNA project logo

This post in Swedish

Unethical research papers should be retracted

Articles that turn out to be based on fraudulent or flawed research are, of course, retracted by the journals that published them. The fact that there is a clearly stated policy for retracting fraudulent research is extremely important. Science as well as its societal applications must be able to trust that published findings are correct and not fabricated or distorted.

However, how should we handle articles that turn out to be based on unethical research? For example, research on the bodies of executed prisoners? Or research that exposes participants to unreasonable risks? Or research supported by unacceptable sources of funding?

In a new article, William Bülow, Tove E. Godskesen, Gert Helgesson and Stefan Eriksson examine whether academic journals have clearly formulated policies for retracting papers that are based on unethical research. The review shows that many journals lack such policies. This introduces arbitrariness and uncertainty into the system, the authors argue. Readers cannot trust that published research is ethical. They also do not know on what grounds articles are retracted or remain in the journal.

To motivate a clearly stated policy, the authors discuss four possible arguments for retracting unethical research papers. Two arguments are considered particularly conclusive. The first is that such a policy communicates that unethical research is unacceptable, which can deter researchers from acting unethically. The second argument is that journals that make it possible to complete unethical research by publishing it and that benefit from it become complicit in the unethical conduct.

Retraction of research papers is a serious matter and very compromising for researchers. Therefore, it is essential to clarify which forms and degrees of unethical conduct are sufficient to justify retraction. The authors cite as examples research based on serious violations of human rights, unfree research and research with unacceptable sources of funding.

The article concludes by recommending scientific journals to introduce a clearly stated policy for retracting unethical research: as clear as the policy for fraudulent research. Among other things, all retractions should be marked in the journal and the reasons behind the retractions should be specified in terms of both the kind and degree of unethical conduct.

For more details on the policy recommendation, read the article in the Journal of Medical Ethics.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Bülow, W., Godskesen, T. E., Helgesson, G., Eriksson, S. Why unethical papers should be retracted. Journal of Medical Ethics, Published Online First: 13 August 2020. doi: 10.1136/medethics-2020-106140

This post in Swedish

We care about communication

Ethical frameworks for research

The word ethical framework evokes the idea of ​​something rigid and separating, like the fence around the garden. The research that emerges within the framework is dynamic and constantly new. However, to ensure safety, it is placed in an ethical framework that sets clear boundaries for what researchers are allowed to do in their work.

That this is an oversimplified picture is clear after reading an inventive discussion of ethical frameworks in neuroscientific research projects, such as the Human Brain Project. The article is written by Arleen Salles and Michele Farisco at CRB and is published in AJOB Neuroscience.

The article questions not only the image of ethical frameworks as static boundaries for dynamic research activities. Inspired by ideas within so-called responsible research and innovation (RRI), the image that research can be separated from ethics and society is also questioned.

Researchers tend to regard research as their own concern. However, there are tendencies towards increasing collaboration not only across disciplinary boundaries, but also with stakeholders such as patients, industry and various forms of extra-scientific expertise. These tendencies make research an increasingly dispersed, common concern. Not only in retrospect in the form of applications, which presupposes that the research effort can be separated, but already when research is initiated, planned and carried out.

This could sound threatening, as if foreign powers were influencing the free search for truth. Nevertheless, there may also be something hopeful in the development. To see the hopeful aspect, however, we need to free ourselves from the image of ethical frameworks as static boundaries, separate from dynamic research.

With examples from the Human Brain Project, Arleen Salles and Michele Farisco try to show how ethical challenges in neuroscience projects cannot always be controlled in advance, through declared principles, values ​​and guidelines. Even ethical work is dynamic and requires living intelligent attention. The authors also try to show how ethical attention reaches all he way into the neuroscientific issues, concepts and working conditions.

When research on the human brain is not aware of its own cultural and societal conditions, but takes them for granted, it may mean that relevant questions are not asked and that research results do not always have the validity that one assumes they have.

We thus have good reasons to see ethical and societal reflections as living parts of neuroscience, rather than as rigid frameworks around it.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Arleen Salles & Michele Farisco (2020) Of Ethical Frameworks and Neuroethics in Big Neuroscience Projects: A View from the HBP, AJOB Neuroscience, 11:3, 167-175, DOI: 10.1080/21507740.2020.1778116

This post in Swedish

We like real-life ethics

Where to publish and where not to publish in bioethics – the 2020 list

Update: The 2022 list is available!

Allegedly, there are over 12.000 so-called predatory journals out there. Instead of supporting readers and science, these journals serve their own economic interests first and at best offer dubious merits for scholars. We believe that scholars working in any academic discipline have a professional interest and a responsibility to keep track of these journals. It is our job to warn the young or inexperienced of journals where a publication or editorship could be detrimental to their career and science is not served.

We have seen “predatory” publishing take off in a big way and noticed how colleagues start to turn up in the pages of some of these journals. While many have assumed that this phenomenon mainly is a problem for low-status universities, there are strong indications that predatory publishing is a part of a major trend towards the industrialization of misconduct and that it affects many top-flight research institutions (see Priyanka Pulla: “In India, elite institutes in shady journals”, Science 354(6319): 1511-1512).

This trend, referred to by some as the dark side of publishing, needs to be reversed. Thus we published this blog post in 2016. This is our fourth annual update (the first version can be found here). At first, we relied heavily on the work of Jeffrey Beall, a librarian at the University of Colorado, who run blacklists of “potential, possible, or probable” predatory publishers and journals. His lists have since been removed although they live on in new form (anonymous) at the Stop predatory journals site (SPJ) and they can also be found archived.

The latest effort to create a thorough blacklist comes from Cabells, who distinguish around 70 different unacceptable violations and employs a whole team reviewing journals. These lists are not, however, the final say on the matter, as it is impossible for one person or a limited group to judge reliably actors in every academic discipline. Moreover, since only questionable journals are listed, the good journals must be found elsewhere.

A response of gatekeeping needs to be anchored in each discipline and the scholars who make up that discipline. As a suitable response in bioethics, we have chosen to, first, collect a few authoritative lists of recommended bioethics journals that can be consulted by anyone in bioethics to find good journals to publish with.

For our first post, we recommended a list of journals ourselves, which brought on some well-deserved questions and criticism about criteria for inclusion. Unfortunately then, our list ultimately drew attention from other parts of the message that we were more concerned to get across. Besides, there are many other parties making such lists. We, therefore, have dropped this feature. Instead, we have enlarged the collection of good journal lists to the service of our readers. They are all of great use when further exploring the reputable journals available:

It is of prime importance to list the journals that are potentially or possibly predatory or of such a low quality that it might be dishonoring to engage with them. We have listed all 50 of them alphabetically (eleven new entries for 2019, two have ceased operation and been removed), and provided both the homepage URL and links to any professional discussion of these journals that we have found (which most often alerted us to their existence in the first place).

Each of these journals asks scholars for manuscripts from, or claims to publish papers in bioethics or related areas (such as practical philosophy). They have been reviewed by the authors of this blog post as well as by a group of reference scholars that we have asked for advice on the list. Those journals listed have unanimously been agreed are journals that – in light of the criticism put forth and the quality we see – we would not deem acceptable for us to publish in. Typical signs as to why a journal could fall in this category, such as extensive spamming, publishing in almost any subject, or fake data being included on the website etc., are listed here:

We have started to more systematically evaluate the journals against the 25 defining characteristics we outlined in the article linked to above (with the help of science and technology PhD students). The results will be added when they exist.

We would love to hear about your views on this blog post, and be especially grateful for pointers to journals engaging in sloppy or bad publishing practices. The list is not meant as a check-list but as a starting point for any bioethics scholar to ponder for him- or herself where to publish.

Also, anyone thinking that a journal in our list should be given due reconsideration might post their reasons for this as a comment to the blog post or send an email to us. Journals might start out with some sloppy practices but shape up over time and we will be happy to hear about it. You can make an appeal against the inclusion of a journal and we will deal with it promptly and publicly.

Please spread the content of this blog as much as you can and check back for updates (we will do a major update annually and continually add any further information found).

Note to readers: The list contained on Stop Predatory Journals has been down for while and it seems the domain now is for sale. From 2022 any reference to journals/publishers being included on SPJ refers to their previous inclusion. We will gradually check for inclusion in the most prominent list presently available, Cabells’ Predatory Reports, as a alternative.

WHERE NOT TO PUBLISH IN BIOETHICS – THE 2020 LIST

  • Advanced Humanities & Social Sciences (Consortium Publisher)
    Critical remark (2018): It has been claimed that behind this journal you find OMICS, the most-ever discussed publisher of this kind, see http://ottawacitizen.com/news/local-news/predatory-publisher-expanding-empire-in-canada. The only article published in 2016 is very badly edited, all the references are lost in the text and the paper would not pass an exam at our departments.  2017 volume is again only one article. The publisher is listed on SPJ.
    Critical remark (2022). After a complaint from the publisher, we have checked the latest volume. An article like this one shows no evident editorial work on the paper at all, so we still regard the journal to be a low quality outlet for research.
  • Advances In Medical Ethics  (Longdom Publishing)
    Critical remark (2019): When asked, one editor attest to the fact that his editorship was forged. Publisher was on Beall’s list and is now listed at Cabells with 5 violations. A thorough review December 2019 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals. A more recent review (2022) concludes that it exhibits about 17 such criteria.
  • American Open Ethics Journal (Research and Knowledge Publication)
    Critical remark (2019): Listed on Cabells with 7 violations. A thorough review February 2020 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • Annals of Bioethics & Clinical Applications (Medwin Publishers)
    Criticism 1 │ Criticism 2
    Critical remark (2019): Publisher was on Beall’s list and is on many other lists of these journals. They say that they are “accepting all type of original works that is related to the disciplines of the journal” and indeed the flow chart of manuscript handling does not have a reject route. Indexed by alternative indexes.
    Critical remark (2020): Listed on Cabells with 5 violations. A thorough review October 2020 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • Austin Journal of Genetics and Genomic Research (Austin Publishing Group)
    Criticism 1 │Criticism 2 │Criticism 3
    Critical remark (2017): Spam e-mail about special issue on bioethics; Listed by SPJ; Romanian editorial member is said to be from a university in “Europe”; Another editorial board member is just called “Michael”; APG has been sued by International Association for Dental Research and The American Association of Neurological Surgeons for infringing on their IP rights. Student reviews concludes the journal is not suitable to publish in, one finding that the journal exhibits at least 16 of the 25 criteria for “predatory” journals.
    Critical remark (2019): Listed by Cabells with 10 violations.
    Critical remark (2021): A thorough review concludes that the journals exhibits at least 13 of the 25 criteria for “predatory” journals.
  • British Open Journal of Ethics (British Open Research Publications)
    Critical remark (2019): Listed by Cabells with 6 violations.
    Critical remark (2022): A thorough review concludes that the journal exhibit many criteria for “predatory” journals, for example that no editorial board exists and the journal is not indexed, and that it is strongly recommended to avoid “publishing” with this journal.
  • Creative Education (Scientific Research Publishing – SCIRP)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Listed by SPJ; They claim misleadingly to be indexed by ISI but this relates to be among cited articles only – they are not indexed. A thorough review May 2017 concludes that it exhibits at least 5 of the 25 criteria for “predatory” journals.
  • East European Scientific Journal (East European Research Alliance)
    Critical remark (2017): Listed by SPJ; Criticised by Beall for having a bogus editorial board; Claims to be indexed by ISI but that is not the well-known Institute for Scientific Information (now Thompson Reuters), but rather the so-called International Scientific Indexing. Thorough reviews November 2018 and February 2019  conclude that it exhibits at least 13 or 14 of the 25 criteria for “predatory” journals.
  • Ethics Today Journal (Franklin Publishing)
    Critical remark (2019): Listed by Cabells with 9 violations.
  • European Academic Research (Kogaion Publishing Center, formerly Bridge Center)
    Critical remark (2017): Listed by SPJ; Uses impact factor from Universal Impact Factor (now defunct); A thorough review May 2017 concludes that it exhibits at least 15 of the 25 criteria for “predatory” journals.
  • European Scientific Journal (European Scientific Institute)
    Critical remark (2017): Listed by SPJ; Use of alternative indexes. A thorough review May 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Advances in Social Science and Humanities
    Critical remark (2017): Listed by SPJ; Impact factor given by  Global Impact Factor. A thorough review March 2019 concludes that it exhibits at least 10 of the 25 criteria for “predatory” journals.
  • International Journal of Contemporary Research & Review
    Critical remark (2017): Listed by SPJ; Indexed by Index Copernicus; Despite claims they seem not to be indexed by either Chemical Abstracts or DOAJ. A thorough review June 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Current Research
    Criticism 1
    Critical remark (2017): Listed by SPJ; Uses IF from SJIF and Index Copernicus and more. It wrongly claims to be indexed by Thomson Reuters, ORCID and having a DOI among other things. A thorough review January 2018 concludes that it exhibits at least 12 of the 25 criteria for “predatory” journals.
  • International Journal of Current Research and Academic Review (Excellent Publishers)
    Critical remark (June 2018): Listed by SPJ and Cabells because of misleading claims about credentials, metrics, and too quick review; alternative indexing; publishes in almost any field imaginable; the editor -in-chief is head of the “Excellent Education and Researh Institute” (sic) which does not seem to exist even when spelled right? A thorough review in December 2019 concludes that it exhibits at least 12 of the 25 criteria for “predatory journals”.
  • International Journal of Ethics (Nova Science Publishers)
    Criticism 1Criticism 2
    Critical remark (2022): The article on Nova at Wikipedia notes that librarians have been critical of this publisher, a Ms. Alexandra Columbus is both the owner of, business manager and customer contact for Nova.
  • International Journal of Ethics & Moral Philosophy (Journal Network)
    Critical remark (2017): Listed by SPJ; Publisher was criticized by Beall when launching 350 journals at once; After several years not one associate editor has signed up and no article has been published; No editorial or contact details available. Thorough reviews in May 2019 and February 2020 conclude that it exhibits at least 10 to 12 of the 25 criteria for “predatory journals”.
  • International Journal of Ethics in Engineering & Management Education
    Critical remark (2019): Papers from almost any field; Claims to have a 5.4 Impact factor (from IJEEE); Indexed by GJIF etc. A non-existent address in “Varginia”, US (sic!); Open access but asks for the copyright; Claims to be indexed in Scopus can’t be verified. Thorough reviews February 2018 and February 2020 conclude that it exhibits at least 16-17 of the 25 criteria for “predatory” journals. Listed by Cabells with 11 violations found.
  • International Journal of Humanities and Social Science (Centre for Promoting Ideas)
    Criticism 1Criticism 2Criticism 3 │ Criticism 4
    Critical remark (2019): The chief editor listed in April 2014  is a deceased person (2018). A thorough review in April 2019 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Science Invention
    Criticism 1 
    Critical remark (2017): Listed by SPJ and is on many other lists of blacklisted journals; An IF of 4.5 given by African Quality Centre for Journals; Open access but asks for the copyright; Publishes any subject; Says that the journal is indexed in DOAJ which it does not seem to be. A thorough review February 2018 concludes that it exhibits at least 13 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Science Research
    Critical remark (2017): Listed by SPJ and Cabells; Claims an IF of 5.22 (by “Research Journal Impact Factor“); Despite title from India; Alternative indexing; Thorough reviews in February 2018 and February 2020 conclude that it exhibits at least 10-13 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities and Social Sciences
    Critical remark (2017): Listed by SPJ; Has an amazing fast-track review option for $100 that guarantees “the review, editorial decision, author notification and publication” to take place “within 2 weeks”. “Editors” claim that repeated requests to be removed from the list of editors result in nothing. Thorough reviews in  February and June 2018 conclude that it seems to exhibit at least 7 to 10 of the 25 criteria for “predatory” journals.
  • International Journal of Humanities & Social Studies
    Critical remark (2017): Listed by SPJ; IF from International Impact Factor Services; States that there “is no scope of correction after the paper publication”.
    Critical remark (2018): They write that the “review process will be completed expectedly within 3-4 days”.
    Critical remark (2020): A thorough review in October 2020 concludes that it seems to exhibit at least 7-8 of the 25 criteria for “predatory” journals.
  • International Journal of Legal, Ethical and Regulatory Issues (Jacobs Publishers)
    Criticism 1
    Critical remark (2019): Spamming with invitation to publish. They are unsure of their own name; in the e-mail they call the journal “International Journal of Legal, Ethical and Regulatory Affairs“! Publisher listed on SPJ. Editor-in-chief and editorial board are missing. Claims that material is “written by leading scholars” which is obviously false.
  • International Journal of Philosophy (SciencePG)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Listed by SPJ; Alternative indexing and also IF from Universal Impact Factor (now defunct); Promises a two-week peer review. Thorough reviews in April and November 2018 conclude that it seems to exhibit at least 10 or 8 of the 25 criteria for “predatory” journals and also find obvious examples of pseudo-science among the published articles.
  • International Journal of Philosophy and Theology (American Research Institute for Policy Development) Criticism 1Criticism 2 │ Criticism 3
    Critical remark: A thorough review in June 2018 concludes that “there are grounds to believe that the American Research Institute never intended to create a serious scientific periodical and that, on the contrary, its publications are out-and-out predatory journals.”
    Update (2022): A thorough review in June concludes that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals. However, the website could not be accessed on June 21.
  • International Journal of Public Health and Human Rights (Bioinfo Publications)
    Criticism 1
    Critical remark (2017): Listed on SPJ; On many other blacklists and IF from Index Copernicus.
  • International Journal of Research in Humanities and Social Studies (Sryahwa Publications)
    Critical remark (2017): Listed on SPJ; Open access but asks for the copyright. A thorough review in April 2018 concludes that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals.
    Update (2022): A June review again confirmed that it seems to exhibit at least 9 of the 25 criteria for “predatory” journals.
  • International Journal of Social Science and Humanities Research (Research Publish Journals)
    Critical remark (2017): Listed on SPJ; On their homepage they state that in order to get a high IF their journals are “indexed in top class organisation around the world” although no major index is used. A thorough review in 2020 concludes that it seems to exhibit at least 14 of the 25 criteria for “predatory” journals.
  • International Open Journal of Philosophy (Academic and Scientific Publishing)
    Critical remark (2017): Listed on SPJ and was heavily critized on Beall’s blog; The editorial board consists of one person from Iran; Although boosting 12 issues a year they have published only 1 article in the journal’s first four years; A thorough review March 1 2017 concludes that it exhibits 17 of the 25 criteria for “predatory” journals and one in March 2019 that it exhibits at least 13 criteria.
  • International Researchers
    Critical remark (2017): Listed on SPJ; Indexed by e.g. Index Copernicus; Claims that it is “Monitor by Thomson Reuters” but is not part of the TR journal citation reports; Several pages are not working at time of review; A thorough review April 24 2017 concludes that it exhibits at least 6 of the 25 criteria for “predatory” journals.
  • Internet Journal of Law, Healthcare and Ethics (ISPUB)
    Criticism 1 │ Criticism 2
    Critical remark (2017): Formerly on Beall’s list.
  • Journal of Academic and Business Ethics (Academic and Business Research Institute)
    Critical remark (2017): Listed on SPJ as well as several other blacklists; Journal seems uncertain about it’s own name, the header curiously says “Journal of ethical and legal issues”.
    Update 2021: A thorough review May 2021 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Bioethics and Applications (Sci Forschen)
    Critical remark (2018): Brand new journal with no articles yet. Publisher has been criticized for spamming more than once, have a bad record at Scam Analyze, and is listed on SPJ.
    Critical remark (2022): A thorough review March 2022 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Clinical Research & Bioethics (OMICS/Walsh MedicalMedia)
    Criticism 1Criticism 2 │ Criticism 3 │ Criticism 4 │ Criticism 5 │ Criticism 6
    Critical remark (2017): This publisher is listed on SPJ and was taken to court for possible fraud by the Federal Trade Commission in the US (and lost). They are listed by Cabells for 8 violations.
    Update (2022): They now have a new (?) publisher, but still the same Danish editor as before. A thorough review May 2022 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.
  • Journal of Law and Ethics
    Critical remark (2017): Listed on SPJ; Claims to be on Ulrichs but is not; Claims to be in the Norwegian list and can actually be found there but under its former name (4 years earlier) and with 0 points.
    Update 2019: Seems to have moved to here. Security warnings and denied access makes it impossible to check whether it is the same journal or another one.
  • Journal of Philosophy and Ethics (Sryahwa Publications)
    Critical remark (2019): listed by Cabells for 7 violations.
    Critical remark 2020): A thorough review October 2020 concludes that it exhibits at least 11 of the 25 criteria for “predatory” journals.
  • Journal of Research in Philosophy and History (Scholink)
    Criticism 1 
    Critical remark (June 2018): Listed on several lists of predatory publishers. They only do “peer review” through their own editorial board, a flowchart states. They claim to check for plagiarism but the first 2018 article abstract run by us through a checker turned out to be self-plagiarized from a book and it looks to have been published many times over. Unfortunately, the next paper checked in the same issue was also published the previous year by another journal listed here…
    Critical remark (March 2021): A thorough review concludes that it exhibits at least 14 of the 25 criteria for “predatory” journals.
  • Journal of Social Sciences and Humanities (AASCIT)
    Criticism 1Criticism 2Criticism 3
    Critical remark (2019): From law to religion, this journal publishes it all. Though publisher claims to be “American”, it has only two editors, both from India. The list from Cabells includes 13 journals from this publisher. The AASCIT Code of Ethics apparently plagiarizes the INCOSE Code of Ethics.
  • Journal of Studies in Social Sciences and Humanities
    Critical remark (2017): Listed on SPJ; Alternative indexing; Uses several alternative IF providers. A thorough review October 2017 concludes that it exhibits at least 9 of the 25 criteria for “predatory” journals.
    Critical remark (2020): A thorough review October 2020 concludes that it exhibits at least 4 of the 25 criteria for “predatory” journals.
  • JSM Health Education and Primary Health Care
    Spamming with invitation to special issue on ‘Bioethics’. The publisher is listed on SPJ,  and criticized and exposed here. It is indexed by spoof indexer Directory of Research Journals Indexing among others (whose website is now gone, BTW).
    Update 2019: Access denied because of non-secure connection.
  • Medical Ethics and Communication (Avid Science)
    Criticism 1
    Critical remarks (2017): Listed on SPJ; Spamming researchers with offer of eBook publication for $350.
    Update: In June 2022, the journal cannot be accessed online.
  • Nova Journal of Humanities and Social Sciences
    Criticism 1
    Critical remark (2018): This publisher was on Beall’s list; Uses alternative impact factors and indexing; Publishes in less than 30 days; Curiously, it says no fee is charged for publication.
    Update: In June 2022, the journal cannot be accessed online.
  • Open Journal of Philosophy (Scientific Research Publishing – SCIRP)
    Criticism 1 │ Criticism 2 │ Criticism 3 │
    Critical remark (2021): A thorough review March 2021 concludes that it exhibits 6 of the 25 criteria for “predatory” journals.
  • Open Journal for Studies in Philosophy (Center for Open Access in Science)
    Critical remark (2020): Cabells found 8 violations.
    Update: Thorough reviews May-June 2022 concludes that it exhibits at least 8-9 of the 25 criteria for “predatory” journals.
  • Philosophical Papers and Review (Academic Journals)
    Critical remark (2017): Listed on SPJ and blacklisted by the Ministry of Higher Education of Malaysia.
    Update (2021): Latest article in press was accepted the same day it was sent in – and it happened back in 2018!
    Update: A thorough review April 2022 concludes that it exhibits at least 10 of the 25 criteria for “predatory” journals.
  • Philosophy Study  (David Publishing Company)
    Criticism 1Criticism 2
    Critical remark (2017): Listed on SPJ. A thorough review October 2019 concludes that it exhibits approx. 8 of the 25 criteria for “predatory” journals.
  • The Recent Advances in Academic Science Journal (Swedish Scientific Publications)
    Critical remark (2018): Despite the publisher’s name it seems based in India. The only Swedish editor’s existence cannot be verified. Website quality is lacking. Listed on SPJ. A thorough review October 2017 concludes that it exhibits at least 15 of the 25 criteria for “predatory” journals.
  • Universal Open Ethics Journal (Adyan Academic Press)
    Critical remark (2019): listed by Cabells for 7 violations.
    Update: Thorough reviews in May 2022 concludes that it exhibits 13 to 20 of the 25 criteria for “predatory” journals.
  • World Journal of Social Sciences and Humanities (Science and Education Publishing, SciEP)
    Criticism 1 │Criticism 2
    Critical remark (2017): Listed on SPJ as well as many other blacklists. A thorough review in May 2019 concludes that it exhibits at least 7 of the 25 criteria for “predatory” journals.

End remark:

In light of recent legal action taken against people trying to warn others about dubious publishers and journals – see here and here, for example – we want to stress that this blog post is about where we would like our articles to show up, it is about quality, and as such it is an expression of a professional judgement intended to help authors find good journals with which to publish.

Indirectly, this may also help readers to be more discerning about the articles they read. As such it is no different from other rankings that can be found for various products and services everywhere. Our list of where not to publish implies no accusation of deception or fraud but claims to identify journals that experienced bioethicists would usually not find to be of high quality. Those criticisms linked to might be more upfront or confrontational; us linking to them does not imply an endorsement of any objectionable statement made therein. We would also like to point out that individual papers published in these journals might of course nevertheless be perfectly acceptable contributions to the scholarly literature of bioethics.

Essential resources on so-called predatory publishing and open access:

Written by…

Stefan Eriksson, Associate Professor of Research Ethics at Uppsala University, read more about his work on publication, regulation and consent.

Gert Helgesson, professor of Medical Ethics at Karolinska Institute.

and…

We like ethics

Responsibly planned research communication

Academic research is driven by dissemination of results to peers at conferences and through publication in scientific journals. However, research results belong not only to the research community. They also belong to society. Therefore, results should reach not only your colleagues in the field or the specialists in adjacent fields. They should also reach outside the academy.

Who is out there? A homogeneous public? No, it is not that simple. Communicating research is not two activities: first communicating the science to peers and then telling the popular scientific story to the public. Outside the academy, we find engineers, entrepreneurs, politicians, government officials, teachers, students, research funders, taxpayers, healthcare professionals… We are all out there with our different experiences, functions and skills.

Research communication is therefore a strategically more complicated task than just “reaching the public.” Why do you want to communicate your results; why are they important? Who will find your results important? How do you want to communicate them? When is the best time to communicate? There is not just one task here. You have to think through what the task is in each particular case. For the task varies with the answers to these questions. Only when you can think strategically about the task can you communicate research responsibly.

Josepine Fernow is a skilled and experienced research communications officer at CRB. She works with communication in several research projects, including the Human Brain Project and STARBIOS2. In the latter project, about Responsible Research and Innovation (RRI), she contributes in a new book with arguments for responsibly planned research communication: Achieving impact: some arguments for designing a communications strategy.

Josepine Fernow’s contribution is, in my view, more than a convincing argument. It is an eye-opening text that helps researchers see more clearly their diverse relationships to society, and thereby their responsibilities. The academy is not a rock of knowledge in a sea of ​​ignorant lay people. Society consists of experienced people who, because of what they know, can benefit from your research. It is easier to think strategically about research communication when you survey your relations to a diversified society that is already knowledgeable. Josepine Fernow’s argumentation helps and motivates you to do that.

Josepine Fernow also warns against exaggerating the significance of your results. Bioscience has potential to give us effective treatments for serious diseases, new crops that meet specific demands, and much more. Since we are all potential beneficiaries of such research, as future patients and consumers, we may want to believe the excessively wishful stories that some excessively ambitious researchers want to tell. We participate in a dangerous game of increasingly unrealistic hopes.

The name of this dangerous game is hype. Research hype can make it difficult for you to continue your research in the future, because of eroded trust. It can also make you prone to take unethical shortcuts. The “huge potential benefit” obscures your judgment as a responsible researcher.

In some research fields, it is extra difficult to avoid research hype, as exaggerated hopes seem inscribed in the very language of the field. An example is artificial intelligence (AI), where the use of psychological and neuroscientific vocabulary about machines can create the impression that one has already fulfilled the hopes. Anthropomorphic language can make it sound as if some machines already thought like humans and functioned like brains.

Responsible research communication is as important as difficult. Therefore, these tasks deserve our greatest attention. Read Josepine Fernow’s argumentation for carefully planned communication strategies. It will help you see more clearly your responsibility.

Finally, a reminder for those interested: the STARBIOS2 project organizes its final event via Zoom on Friday, May 29, 2020.

Pär Segerdahl

Written by…

Pär Segerdahl, Associate Professor at the Centre for Research Ethics & Bioethics and editor of the Ethics Blog.

Fernow, J. (2019). Note #11: Achieving impact: Some arguments for designing a communications strategy, In A. Declich (Ed.), RRI implementation in bioscience organisations: Guidelines from the STARBIOS2 project, (pp. 177-180). Uppsala University. ISBN: 978-91-506-2811-1

We care about communication

This post in Swedish

Science and society: a changing framework and the role of RRI (by Daniele Mezzana)

The STARBIOS2 project has carried out its activities in a context of the profound transformations that affect contemporary societies, and now we are all facing the Covid-19 pandemic. Science and society have always coevolved, they are interconnected entities, but their relationship is changing and it has been for some time. This shift from modern to so-called postmodern society affects all social institutions in similar ways, whether their work is in politics, religion, family, state administration, or bioscience.

We can find a wide range of phenomena connected to this trend in the literature, for instance: globalization; weakening of previous social “structures” (rules, models of action, values and beliefs); more capacity and power of individuals to think and act more freely (thanks also to new communication technologies); exposure to risks of different kinds (climate change, weakening of welfare, etc.); great social and cultural diversification; and weakening of traditional boundaries and spheres of life, etc.

In this context, we are witnessing the diminishing authority and prestige of all political, religious, even scientific institutions, together with a decline in people’s trust towards these institutions. One example would be the anti-vaccination movement.

Meanwhile, scientific research is also undergoing profound transformations, experiencing a transition that has been examined in various ways and called various names. At the heart of this transformation is the relationship between research and the society it belongs to. We can observe a set of global trends in science.

Such trends include the increasing relationship between universities, governments and industries; the emergence of approaches aimed at “opening” science to society, such as citizen science; the diffusion of cooperative practices in scientific production; the increasing relevance of transdisciplinarity; the increasing expectation that scientific results have economic, social, and environmental impacts; the increasingly competitive access to public funds for research; the growing importance attached to quantitative evaluation systems based on publications, often with distorting effects and questionable results; and the emergence on the international economic and technological scene of actors such as India, China, Brazil, South Africa and others. These trends produce risks and opportunities for both science and society.

Critical concerns for science includes career difficulties for young researchers and women in the scientific sector; the cost of publishing and the difficulties to publish open access; and the protection of intellectual property rights.

Of course, these trends and issues manifest in different ways and intensities according to the different political, social and cultural contexts they exist in.

After the so-called “biological revolution” and within the context of the “fourth industrial revolution” and with “converging technologies” like genetics, robotics, info-digital, neurosciences, nanotechnologies, biotechnologies, and artificial intelligence, the biosciences are at a crossroads in its relationship to society.

In this new context, more and more knowledge is produced and technological solutions developed require a deeper understanding of their status, limits, and ethical and social acceptability (take organoids, to name one example). Moreover, food security, clean energy transition, climate change, and pandemics are all challenges where bioscience can play a crucial role, while new legal, ethical, and social questions that need to be dealt with arise.

These processes have been running for years, albeit in different ways, and national and international decision-makers have been paying attention. Various forms of governance have been developed and implemented over time, to re-establish and harmonize the relationship between scientific and technological research and the rest of society, including more general European strategies and approaches such as Smart Specialization, Open Innovation, Open Science and Responsible Research and Innovation as well as strategies related to specific social aspects of science (such as ethics or gender).

Taking on an approach such as RRI is not simply morally recommendable, but indispensable for attempting a re-alignment between scientific research and the needs of society. Starting from the areas of the life of the scientific communities that are most crucial to science-society relations (The 5+1 RRI keys: Science education, Gender equality, Public engagement, Ethics, Open access, and the cross-cutting sixth key: Governance) and taking the four RRI dimensions into account (anticipation, inclusiveness, responsiveness, and reflexivity) can provide useful guidance for how to activate and drive change in research organisations and research systems.

We elaborate and experiment, in search of the most effective and most relevant solution. While at the same time, there is a need to encourage mainstreaming of the most substantial solutions, to root them more deeply and sustainably in the complex fabric of scientific organisations and networks. Which leads us to ask ourselves: in this context, how can we mainstream RRI and its application in the field of bioscience?

Based on what we know, and on experiences from the STARBIOS2 project, RRI and similar approaches need to be promoted and supported by specific policies and contextualised on at least four levels.

  • Organizational contextualization
    Where mainstreaming takes place through the promotion of a greater embedment of RRI, or similar approaches, within the individual research organizations such as universities, national institutes, private centres, etc.
  • Disciplinary or sectoral contextualization
    Where mainstreaming consists of adapting the responsible research and innovation approach to a specific discipline − for example, biotechnology − or to an entire “sector” in a broad sense, such as bioscience.
  • Geopolitical and cultural contextualization
    Where mainstreaming aims to identify forms of adaptation, or rather reshaping, RRI or similar approaches, in various geopolitical and cultural contexts, taking into account elements such as the features of the national research systems, the economy, territorial dynamics, local philosophy and traditions, etc.
  • Historical contextualization
    Where RRI mainstreaming is related to the ability of science to respond to the challenges that history poses from time to time − and of which the COVID-19 pandemic is only the last, serious example − and to prevent them as much as possible.

During the course of the STARBIOS2 project, we have developed a set of guidelines and a sustainable model for RRI implementation in bioscience research institutions. Over the course of 4 years, 6 bioscience research institutions in Europe, and 3 outside Europe, worked together to achieve structural change towards RRO in their own research institutions with the goal of achieving responsible biosciences. We were looking forward to revealing and discussing our results in April, but with the Covid-19 outbreak, neither that event nor our Cape Town workshop was a possibility. Luckily, we have adapted and will now share our findings online, at our final event on 29 May. We hope to see you there.

For our final remark, as the Covid-19 pandemic is challenging our societies, our political and economic systems, we recognise that scientists are also being challenged. By the corona virus as well as by contextual challenges. The virus is testing their ability to play a key role to the public, to share information and to produce relevant knowledge. But when we go back to “normal”, the challenge of changing science-society relations will persist. And we will remain convinced that RRI and similar approaches will be a valuable contribution to addressing these challenges, now and in the future.

Daniele Mezzana

Written by…

Daniele Mezzana, a social researcher working in the STARBIOS2 project (Structural Transformation to Attain Responsible BIOSciences) as part of the coordination team at University of Rome – Tor Vergata.

This text is based on the Discussion Note for the STARBIOS2 final event on 29 May 2020. 

STARBIOS2 logo

The STARBIOS2 project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 709517. The contents of this text and the view expressed are the sole responsibility of the author and under no circumstances can be regarded as reflecting the position of the European Union.

We recommend readings

« Older posts Newer posts »