Belter 2017

Appraisal of: "Belter, C. W. A relevance ranking method for citation-based search results." Scientometrics 2017 112(2): 731-746."

Reviewer(s):

Elke Hausner

Ruth Wong

Full Reference:
Belter CW. "A relevance ranking method for citation-based search results." Scientometrics 2017 112(2): 731-746.

Short description:

This study tests a method which was modified from a previous study (Belter 2016). In this paper, the author proposed a method of ranking the relevance of citation-based search results to a set of key, or seed, papers by measuring the number of citation relationships they share with those key papers.

The method was tested against 23 published systematic reviews and it was found that the method retrieved 87% of the studies included in these reviews. Additional testing suggested that the method may be less appropriate for reviews that combine literature in ways that are not reflected in the literature itself.

Limitations stated by the author(s):

• The systematic reviews included in this analysis included fewer conference and articles published in journals not indexed by WOS, so this finding should be interpreted with caution.

• These results also suggests that the opposite is also true: that the method is less successful, and therefore less appropriate for literature retrieval, in reviews that combine topics in ways that the literature does not.

• It may be that this method is not appropriate for literature retrieval in reviews that seek to combine the results of multiple interventions, multiple treatment populations, or multiple disorders.

• Although statistical testing indicates that the coherence of the included studies is associated with the success of the method, the correlations are low enough to suggest that other factors may also play a role.

• Much additional work is also needed to further refine the methodological choices involved in implementing this method.

• Since unpublished works and conference proceedings are rarely cited in the medical literature, relying solely on citation methods of literature retrieval could reinforce publication bias in systematic review results.

Limitations stated by the reviewer(s):

• The impact of the missed publications on the findings of the systematic reviews are unknown and outside the scope of the study.

• Six out of 23 reviews retrieved citations that are greater than the number of reported retrieved values.

Study Type:

Single study

Related Chapters:

Tags:

• Citation searching