What is peer review?
Peer review is, together with research indicators, self-evaluation report of a unit, bibliometric analysis and site visit, one of the evaluation tools used to assess the quality research at Charles University. It consists in a review of the selected outputs by independent internationally renowned experts, i.e. reviewers, with the objective of stating the extent to which they comply with the international and national standards of the field and what are their scientific or social benefits.
How many outputs are subject to peer review?
Maximum number of outputs that may be, due to operational and organisational reasons, reviewed within the given evaluation cycle is stipulated by the Rector's measure that sets technical parameters of the evaluation.
In which research areas is peer review applied?
The decision on the research areas in which peer review is applied is within the competence the Research Evaluation Board as one the academic parameters of the evaluation. Once the evaluation commences the Board makes a decision on which research areas to be subject to bibliometric analysis and research areas to be subject to peer review where it is possible to use both tools simultaneously for the assessment of a single research area. The Board sets this parameter based on the preliminary structure of outputs achieved during the evaluation period and proposals of the assessed units.
How many outputs are to be reviewed within individual fields?
The evaluation regulations stipulate a mechanism for distributing the maximum number of outputs among individual fields and units with the aim of ensuring that a well-founded assessment of their overall quality is carried out. Quotas for peer reviewed outputs for each unit and field will be determined by careful calculation based on the number of results produced by that unit in the given field over the perion of evaluation. The quotas ar distributed in such a way so that they provide representation of the research activity of the unit/field, could provide some information even on the smaller units however do not represent too much of a burden for the whole evaluation process.
Which outputs are reviewed?
Based on the distribution of the maximum number of outputs among individual fields and units as described above, each unit selects for every field the appropriate number of outputs to be submitted for peer review using the information system. In addition to the basic information automatically processed by the information system, the unit enters the following information, for every selected output:
Abstract;
Description of the contribution of the unit to the output, in case it was created in cooperation with other units or institutions;
Justification of its importance; and
Expert reviews issued so far, if any.
Outputs that will be peer reviewed will be adequately evidenced in the information system of the University and fall within the evaluation period. In addition, in case of a publication output, access will be provided to full text of the output by uploading its electronic version or an address from which it is free to download from the internet to the relevant section of the information system.
How are outputs reviewed?
Every output is reviewed by two reviewers (if the same output is submitted by two or more units it is reviewed for all of them jointly). Each reviewer is asked to classify the output and to add a brief justification of his or her grade. The classification is as follows:
Excellent output, A. An output representing global top quality in terms of originality, importance, academic correctness or practical application. Belongs to the top 10 % outputs (1st Decile) in your field (Europe/World-wide). The fields with a prevailing national audience may also achieve excellent international quality via the application of state-of-the-art internationally respected methods.;
Very good output, B. An output that is very good on the international scale in terms of originality, importance and academic correctness. Belongs to the top 25 % outputs (1st Quartile) in your field (Europe/World-wide). The fields with a prevailing national audience may also achieve high international quality via the application of state-of-the-art internationally respected methods;
Average output, C. An output that is standard in terms of originality, importance and academic correctness. Belongs to the top 50 % outputs (2nd Quartile) in your field (Europe/World-wide).
Below average output, D. An output that is irrelevant or weak or does not comply with the essential methodological requirements. Bottom 50 % (3rd or even 4th Quartile).
The final grade is decided by the Expert Panel which may for this purpose request a review to be produced by a third reviewer. The final grade of the output is saved in the information system together with the reviews; any discrepancy between the final grade and the grades awarded in the reviews is justified.
What is the outcome of peer review and how it can be accessed?
The outcome of peer review consists in overviews of final grades of the outputs structured by fields, research areas, and units. This outcome is used, together with outcomes of other evaluation tools, by the Expert Panels and the Research Evaluation Board to assess the quality of research within invidivual fields, research areas and units.
During the course of the evaluation the overviews are restricted to the Expert Panels members and Research Evaluation Board members, who can access it via the IS Veda system. To do that, please see the respective manual.
Who is responsible for conducting peer review?
The Expert Panels ensure the course of peer review and prepare its outcomes.