Large expert-curated database for benchmarking document similarity detection in biomedical literature search.

Journal article


Brown P, RELISH Consortium, Zhou Y and Garip, G. 2019. Large expert-curated database for benchmarking document similarity detection in biomedical literature search. Database : the journal of biological databases and curation. Vol 2019, pp. 1 - 67. https://doi.org/10.1093/database/baz085
AuthorsBrown P, RELISH Consortium, Zhou Y and Garip, G.
Abstract

Abstract
Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.

Keywordsbiomedical; research; techniques; systematic bias
Year2019
JournalDatabase : the journal of biological databases and curation
Journal citationVol 2019, pp. 1 - 67
PublisherOxford University Press
ISSN17580463
Digital Object Identifier (DOI)https://doi.org/10.1093/database/baz085
Web address (URL)http://europepmc.org/abstract/med/33326193
Output statusPublished
Publication dates01 Jan 2019
Online29 Oct 2019
Publication process dates
Accepted31 May 2019
Deposited01 Jun 2023
Permalink -

https://repository.derby.ac.uk/item/9z045/large-expert-curated-database-for-benchmarking-document-similarity-detection-in-biomedical-literature-search

  • 49
    total views
  • 0
    total downloads
  • 8
    views this month
  • 0
    downloads this month

Export as

Related outputs

Tested Instructions for the 1-Minute Laughie Laughter Prescription
Kuru Alici, N., Gonot-Schoupinsky, F. and Garip, G. 2024. Tested Instructions for the 1-Minute Laughie Laughter Prescription. The Journal for Nurse Practitioners . 20 (8), pp. 1-4. https://doi.org/10.1016/j.nurpra.2024.105132
University makes me angry: Investigating stimulus-response (S-R) and cognitive-mediation (C-M) emotion beliefs in undergraduate students
Turner, M., Garip, G., Boatwright, D., Chandler, C., Chadha, N. and Wood, A. 2024. University makes me angry: Investigating stimulus-response (S-R) and cognitive-mediation (C-M) emotion beliefs in undergraduate students. PLos ONE. 19 (2), pp. 1-16. https://doi.org/10.1371/journal.pone.0294777
“Turbulence even in a bubble”: A reflexive thematic analysis with family caregivers of people with dementia in the United Kingdom during and beyond the COVID-19 pandemic
Danby, A., Benson, T. and Garip, G. 2023. “Turbulence even in a bubble”: A reflexive thematic analysis with family caregivers of people with dementia in the United Kingdom during and beyond the COVID-19 pandemic. Psychology and Health. pp. 1-26. https://doi.org/10.1080/08870446.2023.2289460
The Hidden Pandemic: A Qualitative Study on How Middle-Aged Women Make Sense Of Managing Their Long Covid Symptoms
Collier, D. and Garip, G. 2023. The Hidden Pandemic: A Qualitative Study on How Middle-Aged Women Make Sense Of Managing Their Long Covid Symptoms. Health Psychology Report. pp. 1-13.
Investigation of the Caregivers’ Quality of Life in Patients with Renal Failure
Stavrou, K. and Garip, G. 2023. Investigation of the Caregivers’ Quality of Life in Patients with Renal Failure. EC PSYCHOLOGY AND PSYCHIATRY. 12 (3), pp. 13-25.
The influence of self‐management, anxiety and depression on chronic eczema‐related quality of life
Standen, L. and Garip, G. 2022. The influence of self‐management, anxiety and depression on chronic eczema‐related quality of life. Skin Health and Disease. 2 (4), pp. 1-8. https://doi.org/10.1002/ski2.106
Facilitating the planning and evaluation of narrative intervention reviews: Systematic Transparency in All Intervention Reviews (STAIR)
Gonot-Schoupinsky, Freda, Garip, Gulcan and Sheffield, David 2021. Facilitating the planning and evaluation of narrative intervention reviews: Systematic Transparency in All Intervention Reviews (STAIR). Evaluation and Program Planning. https://doi.org/10.1016/j.evalprogplan.2021.102043
Teach Public Health With a Sense of Humor: Why (and How to) Be a Funnier and More Effective Public Health Professor and Laugh All the Way to Your Classroom
Gonot-Schoupinsky, Freda and Garip, Gulcan 2021. Teach Public Health With a Sense of Humor: Why (and How to) Be a Funnier and More Effective Public Health Professor and Laugh All the Way to Your Classroom. in: The Curious Academic Publishing.
The Covid-19 Pandemic as an Opportunity for Positive Psychology to Promote a Wider-Ranging Definition of Humour and Laughter
Gonot-Schoupinsky, Freda and Garip, Gülcan 2021. The Covid-19 Pandemic as an Opportunity for Positive Psychology to Promote a Wider-Ranging Definition of Humour and Laughter. in: The Palgrave Handbook of Humour Research Palgrave Macmillan/ Springer.
The Engage-Disengage Model as an Inclusive Model for the Promotion of Healthy and Successful Aging in the Oldest-old
Gonot-Schoupinsky, Freda, Garip, Gulcan and Sheffield, David 2021. The Engage-Disengage Model as an Inclusive Model for the Promotion of Healthy and Successful Aging in the Oldest-old. Activities, Adaptation & Aging. https://doi.org/10.1080/01924788.2021.1970892
“We All Need Purpose and Reason to Be Here.”: A Qualitative Investigation of How Members of Alcoholics Anonymous with Long-term Recovery Experience Aging
mcinerney, Kevin, Garip, Gulcan and benson, tony 2021. “We All Need Purpose and Reason to Be Here.”: A Qualitative Investigation of How Members of Alcoholics Anonymous with Long-term Recovery Experience Aging. Alcoholism Treatment Quarterly. https://doi.org/10.1080/07347324.2021.1947165
The Effect of Self-Compassion on Job Burnout and Hours Worked in Employees’ Working from Home
Cotterill, Matthew, Kotera, Y. and Garip, Gulcan 2021. The Effect of Self-Compassion on Job Burnout and Hours Worked in Employees’ Working from Home. Journal of Concurrent Disorders.
Health Benefits of Ikigai: A Review of Literature
Kotera, Y., Kaluzeviciute, G., Garip, Gulcan, McEwan, Kirsten and Chamberlain, Katy 2021. Health Benefits of Ikigai: A Review of Literature. in: Concurrent Disorders Society Publishing.
Development and implementation of evaluation resources for a green outdoor educational program
Garip, Gulcan, Richardson, Miles, Tinkler, Abigail, Glover, Susannah and Rees, Alice 2020. Development and implementation of evaluation resources for a green outdoor educational program. Journal of Environmental Education. https://doi.org/10.1080/00958964.2020.1845588
Learners’ perceptions and experiences of studying psychology online
Garip G, Seneviratne S and Iacovou S 2020. Learners’ perceptions and experiences of studying psychology online. Journal of Computers in Education. https://doi.org/10.1007/s40692-020-00167-4
Laughter and humour for personal development: A systematic scoping review of the evidence
Gonot-Schoupinsky, Freda N., Garip, Gulcan and Sheffield, David 2020. Laughter and humour for personal development: A systematic scoping review of the evidence. European Journal of Integrative Medicine. https://doi.org/10.1016/j.eujim.2020.101144
“Thumb Exercise”: an interpretative phenomenological analysis of psychosocial factors encouraging inactive adults to engage with their smartphones rather than physical activity
Hedges, Tallulah and Garip, Gulcan 2020. “Thumb Exercise”: an interpretative phenomenological analysis of psychosocial factors encouraging inactive adults to engage with their smartphones rather than physical activity. Physical Activity and Health. https://doi.org/10.5334/paah.50
Prescribing laughter to ameliorate mental health, sleep, and wellbeing in university students: A protocol for a feasibility study of a randomised controlled trial.
Gonot-Schoupinsky FN, Garip G, Sheffield D, Omar OM and Arora T 2020. Prescribing laughter to ameliorate mental health, sleep, and wellbeing in university students: A protocol for a feasibility study of a randomised controlled trial. Contemporary clinical trials communications. Vol 20 (Dec 2020), p. 100676. https://doi.org/10.1016/j.conctc.2020.100676
An integrative systematic review of creative arts interventions for older informal caregivers of people with neurological conditions
Irons, J Yoon, Garip, Gulcan, Cross, Ainslea J, Sheffield, David and Bird, Jamie 2020. An integrative systematic review of creative arts interventions for older informal caregivers of people with neurological conditions. PLos ONE. 15 (12), p. e0243461. https://doi.org/10.1371/journal.pone.0243461
Differential qualitative analysis: a pragmatic qualitative methodology to support personalised healthcare research in heterogenous samples
Gonot-Schoupinsky, Freda and Garip, Gulcan 2019. Differential qualitative analysis: a pragmatic qualitative methodology to support personalised healthcare research in heterogenous samples. The Qualitative Report.
Systematic review and meta-synthesis of coping with retinitis pigmentosa: implications for improving quality of life
Garip, Gulcan and Kamal, Atiya 2019. Systematic review and meta-synthesis of coping with retinitis pigmentosa: implications for improving quality of life. BMC Ophthalmology. 19 (1). https://doi.org/10.1186/s12886-019-1169-z
A flexible framework for planning and evaluating early-stage health interventions: FRAME-IT
Gonot-Schoupinsky, Freda N. and Garip, Gulcan 2019. A flexible framework for planning and evaluating early-stage health interventions: FRAME-IT. Evaluation and Program Planning. 77, p. 101685. https://doi.org/10.1016/j.evalprogplan.2019.101685
Acceptance and identity change: An interpretative phenomenological analysis of carers’ experiences in myalgic encephalopathy/chronic fatigue syndrome
Catchpole, Sarah and Garip, Gulcan 2019. Acceptance and identity change: An interpretative phenomenological analysis of carers’ experiences in myalgic encephalopathy/chronic fatigue syndrome. Journal of Health Psychology. https://doi.org/10.1177/1359105319834678
Prescribing laughter to increase well-being in healthy adults: an exploratory mixed methods feasibility study of the Laughie
Gonot-Schoupinsky, Freda and Garip, Gulcan 2019. Prescribing laughter to increase well-being in healthy adults: an exploratory mixed methods feasibility study of the Laughie. European Journal of Integrative Medicine. 26, pp. 56-64. https://doi.org/10.1016/j.eujim.2019.01.005
The lived experiences of experienced Vipassana Mahasi meditators: an interpretative phenomenological analysis.
Ekici, Cimen, Garip, Gulcan and Van Gordon, William 2018. The lived experiences of experienced Vipassana Mahasi meditators: an interpretative phenomenological analysis. Mindfulness. https://doi.org/10.1007/s12671-018-1063-4
The psychosocial impact of caregiving in dementia and quality of life: a systematic review and meta-synthesis of qualitative research.
Cross, Ainslea, Garip, Gulcan and Sheffield, David 2018. The psychosocial impact of caregiving in dementia and quality of life: a systematic review and meta-synthesis of qualitative research. psychology and health. https://doi.org/10.1080/08870446.2018.1496250
Caregiving in multiple sclerosis and quality of life: A meta-synthesis of qualitative research.
Topcu, Gogem, Buchanan, Heather, Aubeeluck, Aimee and Garip, Gulcan 2016. Caregiving in multiple sclerosis and quality of life: A meta-synthesis of qualitative research. psychology and health. https://doi.org/10.1080/08870446.2016.1139112
Laughter and humour interventions for well-being in older adults: A systematic review and intervention classification.
Gonot-Schoupinsky, Freda N. and Garip, Gulcan 2018. Laughter and humour interventions for well-being in older adults: A systematic review and intervention classification. Complementary therapies in medicine. https://doi.org/10.1016/j.ctim.2018.04.009
Coping strategies, vision-related quality of life, and emotional health in managing retinitis pigmentosa: a survey study.
Anil, Krithika and Garip, Gulcan 2018. Coping strategies, vision-related quality of life, and emotional health in managing retinitis pigmentosa: a survey study. BMC Ophthalmology. https://doi.org/10.1186/s12886-018-0689-2
Evaluating the feasibility of a web-based weight loss programme for naval service personnel with excess body weight.
Garip, Gulcan, Morton, Kate, Bridger, Robert and Yardley, Lucy 2017. Evaluating the feasibility of a web-based weight loss programme for naval service personnel with excess body weight. Pilot and Feasibility Studies. https://doi.org/10.1186/s40814-017-0122-2