Preamble

1. Charles University (“the University”) in accordance with its aim to spread learning and to protect knowledge; to cultivate free thinking, independent scholarly research, and original artistic creativity; and to fully promote the creative spirit of human society[1], continuously places emphasis on increasing the quality of scientific, research, development, innovative, artistic or other creative activities (“creative activities”).

2. In order to support the development of a wide range of fields at faculties and higher education institutes (“the units”), to contribute to achieving an international standard of quality, and to strengthen the standing of the University as a prestigious European research institution, the University hereby introduces the evaluation of creative activities (“the evaluation”) in accordance with Article 7 of the Rules for the System of Internal Evaluation and Quality Assurance (“the Rules”).

3. It is the purpose of this measure to set out the details of the evaluation in accordance with Article 7 (19) of the Rules.

Part I • Evaluation Concept

Article 1 • Fundamental Principles

1. The objective of the evaluation is to obtain precise and verifiable information on the quality of creative activities at the University through national and international benchmarking, and to recommend the course of future development of the University based on the information obtained.

2. The purpose of the evaluation is to contribute to ensuring the quality of the fields developed at the University comparable to that achieved by leading European higher education institutions and other research institutions, and to further improve their quality.

3. The evaluation is governed by the principles stated in Article 3 of the Rules.

4. The evaluation is carried out in accordance with the Methodology for Evaluating Research Organisations approved by the government to maximise the use of existing sources of data and to prevent the duplication of work on collection and preparation of documents.

5. The commencement of the relevant evaluation cycle, and the determination of its timelines and further elements stated in this measure, is set out in a Rector’s measure under Article 10.

6. The evaluation process uses to the fullest possible extent the Věda information system for registration and evaluation of creative activities (“information system”) which forms part of the University information system (“CU IS”).

Article 2 • Definition of the Evaluation

1. The period consisting of entire calendar years (“the evaluation period”) extending over no less than three and no more than five years for which the evaluation is performed, is set out in the Rector’s measure under Article 10.

2. The basic organisational and content elements of the evaluation are as follows:

   • a) A field, according to the internal classification of academic fields at the University (“internal classification”) described in Article 2 (1) of the Rector’s measure No. 24/2018 Registration of Creative Activities (“registration rules”);

   • b) A research area, according to the internal classification;[2] and

   • c) A unit.

3. The object of the evaluation is the academic quality of individual fields and research areas offered during the evaluation period at the University in comparison to international standards and the capacity for and the quality of creative activities at individual units.

4. The following sources are used for the purposes of the evaluation:

   • a) Data on creative activities registered in the information system in accordance with the registration rules;

   • b) Additional data registered in the CU IS;

   • c) Documents of the University and the units; and

   • d) External sources of information, in particular international bibliographic and citation databases.

Article 3 • Tools and Mode of Evaluation

1. The following evaluation tools, which are described in more detail in Part II of this Measure, are used as the basis of evaluation within the meaning of Article 4 of the Rules:

   • a) Indicators of creative activities;

   • b) Self-evaluation report of a unit;

   • c) Bibliometric analysis;

   • d) Peer review of selected outputs (“peer review”); and

   • e) Site visit.

2. The quality of a field is assessed by a Field-specific Expert Panel (“Expert Panel”) based on the outcomes of individual evaluation tools.

3. The quality of a field depends primarily on the quality of its outputs, contribution to outputs produced by other fields through interdisciplinary cooperation and national and international standing based on auxiliary criteria listed in Appendix No. 6 to this Measure.

4. If a field was developed over the course of the evaluation period in more than one unit of the University, its quality is assessed also individually for the relevant units based on identical evaluation tools.

5. The quality of a research area is assessed by the Expert Panel based on the quality of the relevant fields and aggregate data on the given area obtained using individual evaluation tools.

6. The quality of a unit is assessed by the Creative Activities Evaluation Board in particular based on the assessment of the quality of individual fields consistently developed by the unit taking into account the indicators of creative activities and the self-evaluation report of the unit, both of which provide information on the concept, strategy, and organisation of creative activities, their relevance for the society, potential for development (viability), and based on how the given unit contributes through its creative activities to the fulfilment of priorities of the University set in the valid strategic plan where auxiliary criteria provided in Appendix No. 7 are used.

Article 4 • Evaluation Outcomes

1. Research area evaluation reports, which are prepared for each area within which at least one field was assessed based on a bibliometric report or peer review, constitute the interim evaluation outcomes.

2. The research area evaluation report includes also an assessment of its relative quality within the relevant units.

3. The final outcome of the evaluation is the Final Report describing and evaluating the quality of individual fields, research areas, and units which contains the recommendations for further development of the creative activities at the University.

4. The following appendices form part of the Final Report:

   • a) Academic evaluation parameters under Article 11 (1);

   • b) Information on the course of evaluation;

   • c) Summary of creative activities indicators under Article 5;

   • d) Outcomes of individual evaluation tools;

   • e) Opinion of the units on the research area evaluation reports;

   • f) Opinion of the units on the draft Final Report.

5. The Final Report and its appendices are drafted in the English language.

6. The Final Report is discussed upon the advice of the Rector by the International Board, Research Board, and Internal Evaluation Board. The Academic Senate is informed of the Final Report.

7. After discussion of the Report under paragraph 6, the Final Report is made available to the bodies of the University and the members of the academic community, its principal findings are published in the publicly accessible section of the University’s website.

8. Evaluation outcomes are used in particular as a basis for the following:

   • a) preparation of strategic documents;

   • b) development of fields and research areas;

   • c) internal administration of the University and its units;

   • d) preparation of programmes for the support of science under Article 7 (2) of the Rules;

   • e) decision-making on the funding of creative activities from the resources of institutional support;[3]

   • f) support of excellence;

   • g) institutional accreditation of areas of study;[4]

   • h) accreditation and evaluation of study programmes;[5] and

   • i) development of educational activities of the University.

Part II • Evaluation Tools

Article 5 • Creative Activities Indicators

1. The creative activities indicators include overviews of:

   • a) Persons involved in creative activities;

   • b) Outputs of creative activities;

   • c) Grants, projects, licenses, and other components of creative activities; and

   • d) Study programmes and numbers of students and graduates of the programmes.

2. The processing of creative activities indicators for an evaluation period is automated; it is based on the data contained in CU IS and carried out in accordance with the methodology stated in Appendix No. 2 to this Measure.

3. The relevant creative activities indicators are used in the course of evaluation as follows:

   • a) As the basis for deciding whether the quality of a research area will be assessed using bibliometric analysis or peer review, or a combination of both methods;

   • b) As input for processing the bibliometric reports and the basic overview of peer review;

   • c) As the basis for the distribution of the outputs reviewed by means of peer review among the fields and units; and

   • d) In the form of numeric overviews as the basis for assessment of the quality of fields, research areas, and units.

4. Before using the creative activities indicators for the purposes stated in paragraph 3, they are submitted to the units for review and for expressing their opinion.

Article 6 • Self-evaluation Report of a Unit

1. The self-evaluation report of a unit describes and critically evaluates, where appropriate, the following:

   • a) The mission, vision, and goals of creative activities;

   • b) The principal measures adopted in the evaluation period to support development of the creative activities and the impact of the measures;

   • c) The research policy of the unit with particular emphasis on doctoral students, post-doctoral staff, creating new research groups or schools, and attracting international students and staff;

   • d) Funding of creative activities including the most important grants and projects investigated;

   • e) Personnel policy and qualification growth;

   • f) Interconnection of creative activities and educational activities;

   • g) importance of creative activities including the most important outputs, and their impact and application;

   • h) The most prestigious prizes won;

   • i) The national and international standing of the research areas developed in particular in comparison to the benchmark institutions;[6]

   • j) Strengths, weaknesses, opportunities, and risks in the area of creative activities; and

   • k) Plan for the further development of creative activities.

2. The self-evaluation report of a unit is drafted in the English language.

3. The report is discussed by the Research Board of the relevant unit before it is uploaded into the information system and thus provided to the Expert Panels and Creative Activities Evaluation Board in accordance with Article 7 (12) of the Rules.

Article 7 • Bibliometric Analysis

1. The bibliometric analyses of individual fields, research areas, and units of the University are prepared on the basis of:

   • a) Data obtained from the information system in accordance with the definition stated in Appendix No. 3 to this Measure; and

   • b) Data contained in the Web of Science and Journal Citation Reports databases.

2. The analysis is prepared based on the bibliometric indicators of the frequency of creative activities and their quality derived from the reputation of the journals in which the outputs were published and from their citation indexes.

3. The data used as input for bibliometric analysis are first provided to the relevant units for review and for expressing their opinion before they are used in the analysis.

4. The outcome of the bibliometric analysis is a bibliometric report which is drafted in the English language in a standardised structure in the form of analytical reports and comments, if any, for individual fields and research areas and as a summary for individual units.

5. The following also forms part of the bibliometric report for each field:

   • a) Internal comparison of the quality of the field at the University, if it was developed during the evaluation period in more than one unit;

   • b) A comparison of the quality of the field at the University with the quality achieved by the benchmark institutions specified by the Creative Activities Evaluation Board under Article 11 (1) (f).

6. The Rectorate is in charge of the methodology and completion of the bibliometric analysis.

Article 8 • Peer Review

1. Peer review consists in a review of the selected outputs by independent internationally renowned experts (“reviewers") with the objective of stating the extent to which the outputs comply with the international and national standards of the field and what are the scientific or social benefits of the outputs.

2. Peer review may be used to review only such outputs which:

   • a) Comply with the conditions for assignment to the indicator under Article 5 (1) (b);

   • b) In case of a publication output, access is provided to full text of the output by uploading its electronic version or an address from which it is free to download from the internet to the relevant section of the information system; and

   • c) It was selected at least by one unit for review using this evaluation tool.

3. The maximum number of outputs that are reviewed within the given evaluation cycle using peer review is stipulated by the Rector’s measure under Article 10, ensuring that a well-founded assessment of the overall quality of all fields using this evaluation tool is carried out.

4. The maximum number of outputs that may be submitted for review is distributed among the fields and units by means of the information system using the procedure described in Appendix No. 4. Based on this distribution, each unit selects for every field the outputs to be submitted for peer review using the information system.[7]

5. The process of selection of the outputs, among other things, takes into account the principal outputs of the author under Article 11 (3) (f) of the registration rules.

6. In addition to the basic information automatically processed by the information system, it is necessary to enter, in the relevant section of the information system and in the English language, the following, for every selected output:

   • a) Abstract;

   • b) Description of the contribution of the unit, which submits the output, to creating the output, in case it was created in cooperation with other units of the University or other national or international research institutions;

   • c) Justification of the importance; and

   • d) Expert reviews issued so far, if any.

7. If the same output is submitted by two or more units of the University which contributed to its creation, it is reviewed for all of them jointly.

8. Every output is reviewed by two reviewers.

9. It is the task of the reviewer to assess the output using the grades listed in Appendix No. 5 to this Measure and to add a brief justification of the grade in the English language.

10. The final grade is decided by the Expert Panel which may for this purpose request a review to be produced by a third reviewer. The final grade of the output is saved in the information system together with the reviews; any discrepancy between the final grade and the grades awarded in the reviews is justified in the English language.

11. If the Expert Panel in the course of reviewing reaches the conclusion that the reviews carried out so far are sufficient to evaluate the overall quality of the field as well as the quality of the field at individual units, the Expert Panel may submit to the Creative Activities Evaluation Board a proposal to terminate peer reviewing in this field.

12. The outcome of peer review consists in overviews of final grades of the outputs structured by fields, research areas, and units and created in the English language.

Article 9 • Site Visit

1. In order to complement the basis required for precise assessment of the quality of a field, research area, or unit the Expert Panel or Creative Activities Evaluation Board may request a site visit.

2. The site visit for the purpose of complementing the basis for assessment takes place after the outcomes of individual parts of evaluation listed in Article 3 (1) (a) to (d) of this Measure are ready.

3. The request for site visit includes the information on the components and content of the meeting and additional required materials, if any, and is sent with sufficient advance, usually one month before the meeting is to take place, so that the relevant unit may prepare for the meeting.

4. The minutes of the meeting are drafted in the English language, and after approval of the minutes by both parties involved they are saved in the information system.

Part III • Course of Evaluation

Article 10 • Commencement and Technical Parameters of Evaluation

1. The Rector stipulates in a Rector’s measure the timelines of the evaluation which include the dates of the following:

   • a) Commencement of the relevant evaluation cycle;

   • b) Setting of the academic evaluation parameters under Article 11;

   • c) Assignment of persons to fields, selection of outputs to be peer reviewed, submission of proposals for the assignment of persons to the list of reviewers, and establishment of Expert Panels;

   • d) Checking of the material correctness of the input data;

   • e) Consideration of the self-evaluation report of units, production of bibliometric report, peer review completion, and submission of the documents to the Expert Panels and the Creative Activities Evaluation Board;

   • f) Processing of research area evaluation reports;

   • g) Expressing of opinion on research area reports by the units;

   • h) Preparation of the Final Report;

   • i) Expressing of opinion on the Final Report by the units; and

   • j) Submission of the Final Report for discussion to the University bodies.

2. The same Rector’s measure also stipulates the technical parameters of evaluation including:

   • a) Evaluation period;

   • b) Conditions for including a person as a member of key staff;[8] and

   • c) Maximum number of outputs to be subject to peer review.

Article 11 • Setting of Academic Parameters

1. The Creative Activities Evaluation Board sets, no later than on the date under Article 10 (1) (b), the academic evaluation parameters, which include:

   • a) Research areas to be subject to bibliometric analysis and research areas to be subject to peer review; it is also possible to use both tools simultaneously for the assessment of a single research area;

   • b) Combinations of fields and units where in particular due to a very low number of outputs the relevant unit is not assessed in the given field and its outputs in the field are not included in the calculation under Appendix No. 4;

   • c) The lower and upper limit on the number of outputs that are subject to peer review in the field for which it was determined that this evaluation tool would be used;

   • d) The number of total outputs per field which forms a decisive limit for determining whether the field is subject to peer review based on the lower limit on the number of outputs or on a higher number of outputs distributed among fields and units using the procedure described in Appendix No. 4;

   • e) The list of operational areas of registration of components included in the indicator under Article 5 (1) (c); and

   • f) The list and elements of the use of selected benchmark institutions primarily for the purposes of comparison within the framework of bibliometric analysis.

2. The Creative Activities Evaluation Board sets the academic evaluation parameters based on preliminary values of indicators under Article 5 (1) (b) and (c) prepared as of the date of commencement of evaluation under Article 10 (1) (a) using the stipulated technical parameters and made available in the information system, and also takes into consideration the proposals of the assessed units.

3. If necessary, the Creative Activities Evaluation Board may convene an academic consultation in order to reach a consensus on academic evaluation parameters.

4. If a research area comprising fields with a high level of interdisciplinarity is created within the internal classification, where the fields go beyond the limits of one panel, the Creative Activities Evaluation Board determines which panel will coordinate their assessment.

Article 12 • Preparation of Input Data and Academic Background

1. The units must ensure, no later than on the date stipulated under Article 10 (1) (c), the:

   • a) Assignment of persons to fields under Article 4 of the registration rules;

   • b) Selection of outputs for peer review in accordance with the rules stated in Article 8; and

   • c) Nomination of reviewers for the fields subject to peer review.

2. In the period between the date under Article 10 (1) (c) and the date under Article 10 (1) (d) the units must check the material correctness of creative activities indicators and data to be used as the basis for producing bibliometric reports. The fact that the check was completed must be indicated by the designated persons for individual units in the information system.

3. If the check is not carried out within the stipulated deadline, the relevant indicators and data are presumed to be correct and it is stated in the information system that the relevant check was not carried out.

4. The basis and the data under paragraph 1 and 2 are closed no later than on the date under Article 10 (1) (d). They may be corrected only based on a decision of the Creative Activities Evaluation Board if otherwise the evaluation results would be substantially distorted.

5. The Rector establishes the Expert Panels no later than on the date under Article 10 (1) (c).

Article 13 • Evaluation Implementation and Preparation of Final Report

1. The following is saved in the information system within the deadline under Article 10 (1) (e):

   • a) Creative activities indicators;

   • b) Self-evaluation reports of units;

   • c) Bibliometric reports;

   • d) Peer review outcomes.

2. Based on the available source documents, the Expert Panels decide whether it is necessary to carry out any site visits. After the site visits they draft the evaluation reports of research areas and make them available in the information system to the designated persons for units and to the members of Creative Activities Evaluation Board and of all Expert Panels within the deadline under Article 10 (1) (f).

3. No later than within the deadline under Article 10 (1) (g) the units may, via the information system, add to the research areas evaluation their opinions written in the English language.

4. After making the evaluation reports on the research areas available, the Creative Activities Evaluation Board decides whether it is necessary to organise further site visits in order to supplement the data required for assessing the quality of individual units.

5. After carrying out the site visits, if any, the Creative Activities Evaluation Board performs a complex assessment of the quality of the units and no later than on the date under Article 10 (1) (h) makes available in the information system the draft Final Report to the designated persons of units and members of all Expert Panels.

6. No later than on the date under Article 10 (1) (i) the units may, via the information system, supplement the Final Report with their opinion written in the English language.

7. Within the deadline under Article 10 (1) (j) the Creative Activities Evaluation Board finalises the Final Report and hands it over to the Rector who submits it for discussion to the University bodies under Article 4 (6).

Part IV • Organisation

Article 14 • Creative Activities Evaluation Board

1. The Creative Activities Evaluation Board is responsible for the management of the evaluation.

2. The Creative Activities Evaluation Board in particular:

   • a) Sets the academic evaluation parameters under Article 11;

   • b) Supervises the course of evaluation from the perspective of compliance with principles, rules, and organisation;

   • c) If required, brings to the attention of the chairs of Expert Panels any deficiencies, especially materially incorrect facts, internal discrepancies or differences in requirements applied in assessment of quality of fields within individual research areas;

   • d) Decides any disputed matters;

   • e) Visits the units in accordance with the conditions set out in Article 9;

   • f) In cooperation with the chairs of Expert Panels, carries out the assessment of quality of units; and

   • g) Produces the Final Report and refers it to the Rector under Article 4.

3. The following applies to establishment of the Creative Activities Evaluation Board:

   • a) The composition of its members guarantees the academic competence for the assessment of fields;

   • b) The Board members are outstanding scientists primarily from abroad or having significant international experience;

   • c) The chairs of Expert Panels are members of the Board;

   • d) The following are members of the Board without the right to vote:

     • i) Vice-Rector of the University vested with the powers of evaluating creative activities;

     • ii) A representative of the Academic Senate;

     • iii) A representative of the Research Board;

     • iv) A representative of the Internal Evaluation Board; and

     • v) A representative appointed by the Rector.

4. The chair, deputy chair, and other members of the Creative Activities Evaluation Board are appointed by the Rector after discussion of the nomination by the Research Board and the International Board no later than on the date of commencement of evaluation.

5. The meetings of the Board are governed by the code of procedure approved by the Creative Activities Evaluation Board.

6. The Creative Activities Evaluation Board may decide on organisational matters remotely; if it makes decisions concerning the content of the evaluation such decisions are made in a formal meeting which usually takes place in the premises of the University.

7. The secretary to the Creative Activities Evaluation Board participates in its meetings and is also responsible for ensuring the required support and administrative activities.

Article 15 • Expert Panels

1. The content part of evaluation dealing with the assessment of the quality of fields and research areas is the responsibility of the Expert Panels for the following fields:

   • a) Arts and humanities;

   • b) Social sciences;

   • c) Medical and health sciences; and

   • d) Natural sciences.

2. The distribution of fields and research areas among the Expert Panels reflects the internal classification.

3. The Expert Panels in particular:

   • a) Ensure the course of peer review and prepare the outcomes;

   • b) Visit the units in accordance with the conditions stipulated in Article 9;

   • c) Carry out the assessment of the quality of fields and research areas on the basis of documents and outcomes of individual evaluation tools;

   • d) Submit the outcomes of their activities to the Creative Activities Evaluation Board.

4. The following applies to the establishment of Expert Panels:

   • a) The number of its members is based on the size of research areas and fields which fall under the responsibility of the Panel;

   • b) The Panel members are outstanding scientists from abroad or scientists having significant international experience;

   • c) An executive deputy chair (without the right to vote) who is a renowned scientist knowledgeable of Czech higher education is always a member of the Panel;

   • d) The members of the Panel are not in conflict of interest with the assessed units.

5. The chairs, deputy chairs, and other members of the Expert Panels are appointed by the Rector on the advice of the Creative Activities Evaluation Board based on the nominations submitted by the units, members of the Rector’s Board, the Research Board and the International Board.

6. The chair of the Expert Panel:

   • a) Ensures that it is functional;

   • b) Manages its activities;

   • c) Submits for discussion any issues with a significant impact on the course, correctness, or outcomes of evaluation to the Creative Activities Evaluation Board together with the proposed solution; and

   • d) Cooperates with the chairs of other Expert Panels.

7. An Expert Panel may decide on organisational matters remotely; if it makes decisions concerning the content of evaluation such decisions are made in a formal meeting which usually takes place on the premises of the University.

8. The secretary to an Expert Panel participates in its meetings and is also responsible for ensuring the required support and administrative activities.

Article 16 • Reviewers

1. It is the task of the reviewer to provide to the Expert Panel, within the given time limit via the relevant section of the information system, the reviews of the outputs the reviewer was asked to review by the Panel in the form stated in Article 8 (9).

2. The reviewers are, within the deadline under Article 10 (1) (c), and further as required, nominated by the units and by the members of the Creative Activities Evaluation Board, Expert Panels, Research Board, Internal Evaluation Board, and International Board together with a statement of the field in which they are qualified to review the outputs.

3. The nominated reviewers are approved by the Expert Panel.

4. The list of approved reviewers is registered in the information system on the basis of an employment relationship; in addition to the activities listed in paragraph 1, the signature of the agreement binds the reviewer:

   • a) To keep confidential the facts the reviewer becomes aware of in the course of evaluation;

   • b) To protect the intellectual property in relation to the outputs submitted to the reviewer for the purpose of review; and

   • c) To the duty to disclose any conflict of interest defined in Appendix No. 8 to this Measure.

5. The reviewer reviews only those outputs where a conflict of interest as described in Appendix No. 8 to this Measure was ruled out.

6. The reviews of other reviewers are not made available to the reviewer.

7. The reviewer is entitled to receive a remuneration for every review produced.

Article 17 • Units

1. A unit has a right to:

   • a) Express its opinion on matters that are the subject of the Rector’s measure under Article 10;

   • b) Propose the academic evaluation parameters;

   • c) Nominate the members of the Expert Panels and reviewers;

   • d) Submit to the Creative Activities Evaluation Board motions concerning the failure to fulfil or a violation of the evaluation rules;

   • e) Carry out a check of the material correctness of data serving as input for the calculation of indicators and data sets submitted for the purpose of bibliometric analysis;

   • f) Express its opinion on interim and final evaluation outcomes.

2. A unit has a duty to:

   • a) Cooperate in ensuring the course of the evaluation in accordance with the requisites stipulated in this Measure;

   • b) Ensure the maximum possible correctness and completeness of information based on which the evaluation is carried out;

   • c) Appoint a person that is designated to coordinate the administrative and organisational activities related to evaluation (“evaluation coordinator”);

   • d) Ensure the participation of the evaluation coordinator and, in the case of a faculty, also of the relevant vice-dean at consultation meetings convened by the Vice-Rector vested with the power of evaluating creative activities.

Article 18 • Administrative and Technical Support

1. Administrative and technical support for evaluation at the level of the University is provided by:

   • a) The Research Support Office of the Rectorate; and

   • b) The Computer Science Centre.

2. The Research Support Office ensures:

   • a) Methodological support for the participants in the evaluation;

   • b) The background for the activities of Creative Activities Evaluation Board and the Expert Panels;

   • c) The preparation of employment agreements with the reviewers and administration of their list under Article 16 (4) of this Measure; and

   • d) In cooperation with the Computer Science Centre, the correct setting of parameters of the information system and administration of access rights.

3. The Computer Science Centre ensures:

   • a) The operation and development of information system; and

   • b) In cooperation with the Research Support Office, the technical support for the users of the information system.

4. The participants of evaluation have preferential access to the tool for remote user support which forms part of the information system.

Part V • Common and Final Provisions

Article 19 • Final Provisions

1. The following are the appendices to this Measure:

   • No. 1 – Diagram of the Course of Evaluation;

   • No. 2 – Methodology for Producing Creative Activities Indicators;

   • No. 3 – Data Sets for Producing Bibliometric Analysis;

   • No. 4 – Determination of the Number and Structure of Outputs for Peer Review;

   • No. 5 – Determination of Quality of an Output Subject to Peer Review;

   • No. 6 – Assessment of the Quality of Research Areas and Fields;

   • No. 7 – Assessment of the Quality of a Unit;

   • No. 8 – Conflict of Interest of Participants in Evaluation.

2. This Measure becomes effective on 1 November 2018.

In Prague on 29 October 2018

Prof. MUDr. Tomáš Zima, DrSc. (Rector)

Appendix No. 2 • Methodology for Producing Creative Activities Indicators

Persons Involved in Creative Activities

1. The source data for the indicator is retrieved from the register of authors maintained in accordance with the registration rules.

2. The indicator uses data on persons involved in creative activities during the evaluation period, structured into categories. The categories are created automatically using the rules arising from compliance with the requirements stated in the relevant paragraphs of Article 4 of the registration rules:

   • a) A person that complies with the requirements under paragraph 2 (a) is included in the category “core staff” for the given unit;

   • b) A person that complies with the requirements under paragraph 2 (b) and at the same time does not comply with the requirements for being included in the core staff is included in the category “doctoral students” for the given unit;

   • c) A person included in the register of author’s under paragraph 2 (c) or (d) is included in the category “others” for the given unit.

3. The outcome of the indicator consists in overviews of physical numbers of persons and numbers of persons converted using the involvement rate[9] of persons involved in creative activities structured by units, fields, and research areas.

4. The overviews may also be displayed in the following ways:

   • a) In the case of core staff, sorted by pay bands, age groups, and nationality;

   • b) In the case of doctoral students, sorted by nationality and the language of study; and

   • c) In the case of “others”, sorted by the type of relation to the University.

5. The processing of personal data is carried out in a way that complies with the principles and rules of personal data protection stipulated in the relevant Rector’s measure.

Outputs of Creative Activities

1. The source of data for the indicator is the register of outputs maintained in accordance with the registration rules.

2. The indicator uses data on outputs of creative activities produced during the evaluation period; the following aspects are decisive for an output to be included in evaluation:

   • a) The year of publication falling within the evaluation period; and

   • b) Achievement of correct and complete data as a result of compliance with the registration rules, which is evidenced in the information system in the appropriate record status[10].

3. The outcome of the indicator consists in absolute numbers of outputs and converted numbers after the deduction of contributions of authors who in relation to this output were neither employed by the University nor studied at the University (“external authors”), where the contribution of external authors is determined proportionally to their representation in the overall number of an output authors.

4. The overviews may be sorted by the following:

   • a) Individual years of evaluation period;

   • b) Units;

   • c) Research areas and fields;

   • d) Individual types and subtypes of outputs including the attribute of the usually valuable type of research area;

   • e) Statement of Web of Science identifier;

   • f) Language of the output.

Components of Creative Activities

1. The source of data for the indicator is the register of components of creative activities maintained in accordance with the registration rules.

2. The indicator uses data on components of creative activities produced during the evaluation period; the following aspects are decisive for a component to be included in evaluation:

   • a) Operational area within which the component falls;[11]

   • b) The period of investigation falling at least partially within the evaluation period; and

   • c) Achievement of correct and complete data as a result of compliance with the registration rules, which is evidenced in the information system by an appropriate record status[12].

3. The outcome of the indicator consists in lists of components of creative activities which may be sorted by:

   • a) Individual years of evaluation period;

   • b) Units;

   • c) Academic fields;

   • d) Operational areas; and

   • e) Provider and programme.

Study Programmes, Students and Graduates

1. The source of data for the indicator are the data stored in the Student Information System and data transferred in batches to the national register of students.

2. The indicator uses data on study programmes, students, and graduates.

3. The outcome of the indicator consists in overviews of related study programmes and numbers of their students and graduates which may be sorted by:

   • a) Individual years of evaluation period;

   • b) Units;

   • c) Related academic fields;

   • d) Study programme type.

Appendix No. 3 • Data Sets for Producing Bibliometric Analysis

Data Set Containing Persons

1. The set of persons included in the data set is identical to the set of persons listed in the indicator under Article 5 (1) (a) of this Measure.

2. Every person is listed in the data set separately for every unit for which the person was included in the register of authors.

3. The data set contains the following data for each person:

   • a) Name of the unit to which the person is assigned;

   • b) University identification number;

   • c) Date of birth;

   • d) Gender;

   • e) Nationality;

   • f) Assignment to a field or fields;

   • g) Category with respect to the evaluation period;

   • h) Involvement rate;

   • i) Full time equivalent as of the last day of the evaluation period;

   • j) Category with respect to the last day of the evaluation period;

   • k) The list of codes of PROGRES programmes in which the person was involved for at least one day of the evaluation period.

4. The processing of personal data is carried out in a way that complies with the principles and rules of personal data protection stipulated in the relevant Rector’s measure.

Data Set Containing the Outputs

1. The set of outputs included in the data set is identical to the set of outputs listed in the indicator under Article 5 (1) (b) of this Measure.

2. The data set of outputs lists every output for every unit whose workplace is recorded in the record for any of the authors.

3. The data set contains the following data:

   • a) Name of the unit which contributed to creation of the output;

   • b) Unique internal identifier of the output in information system;

   • c) Title of the output in the language of the original;

   • d) Year of publication;

   • e) ISSN;

   • f) Type of output according to the information system;

   • g) Type of output according to the Register of Information on Outputs (RIV);

   • h) Field of the output according to internal classification;

   • i) Information whether the output is or is not selected for peer review;

   • j) Number of authors according to the RIV definition;

   • k) Field according to the code list of CEP/RIV,

   • l) Field according to the code list of RIV/FORD,

   • m) List of codes of PROGRES programmes in which the output was created;

   • n) Unique identifier of output in Web of Science database if provided in the information system;

   • o) Type of output according to the classification of Web of Science, if provided in the information system; and

   • p) List of the University identification numbers of all internal authors.

Supplementary Data Sets of Code Lists

1. The supplementary data sets include:

   • a) Code list of fields according to internal classification;

   • b) Code list of units;

   • c) Code list of types and subtypes from the information system; and

   • d) Code list of PROGRES programmes.

Appendix No. 4 • Determination of the Number and Structure of Outputs for Peer Review[13]

1. The Rector’s measure under Article 10 sets out the maximum number of outputs that may be reviewed due to operational and organisational reasons within the given evaluation cycle using peer review (“maximum number of outputs”).

2. The Creative Activities Evaluation Board sets out:

   • a) The lower and upper limit on number of outputs that are reviewed in the field (“the lower limit" and "the upper limit”); and

   • b) The limit for the field size determined on the basis of the number of outputs in the field in the evaluation period, that is decisive for determination whether the lower limit on the number of outputs is sufficient to assess the quality of the field.

3. Based on the above parameters, the number of outputs to be submitted for peer review is determined for individual fields and units as follows:

   • a) After allocation of the lower-limit number of outputs to each peer reviewed field the remaining part of the maximum number of outputs is distributed among the fields that exceeded the limit for the field size in proportion reflecting their contribution to all outputs of research areas that are peer reviewed;

   • b) If the number of outputs determined in this way exceeds in a given field the upper limit, the outputs exceeding the limit are released for subsequent allocation using the procedure described in paragraph 5;

   • c) The number of outputs determined in this way that are reviewed in the given field is distributed among units[14] in proportion to their contribution to the total of outputs achieved in the given field.

4. In the calculations under paragraph 3 the numbers are rounded down.

5. If the procedure under paragraph 3 does not use up the available number of outputs, the remaining outputs are distributed among individual units based on the ranking of units by total number of out-puts which are subject to evaluation[15], for all fields from the highest number to the lowest number where each unit starting from the top of the list is allocated one output until the max-imum number of outputs is used up.

Appendix No. 5 • Determination of Quality of an Output Subject to Peer Review

Excellent output, A • An output representing global top quality in terms of originality, importance, academic correctness, or practical application.[16]

Very good output, B • An output that is excellent on the international scale in terms of originality, importance, and academic correctness.

Average output, C • An output that is standard in terms of originality, importance, and academic correctness.

Below average output, D • An output that is irrelevant or weak or does not comply with the essential methodological requirements.

Appendix No. 6 • Assessment of the Quality of Research Areas and Fields

The quality of a field is assessed by the Expert Panel where, among others, the below criteria are followed:

Excellent field, A • The field produces in all evaluated criteria excellent outputs whose quality is fully comparable to the standard of selected benchmark institutions. It includes a sufficient number of internationally reputed persons, the production of high-quality outputs is in line with its size and there are no cases of problematic outputs or components of creative activities. The majority, i.e., for example 60% of outputs reviewed using bibliometric analysis falls within the first two quartiles, or category A and B in the case of peer review, and a significant minority (approximately less than 10%) falls within the fourth quartile or category D. An excellent field reflects in terms of quality the corresponding fields at high-quality quality European universities predetermined as the quality benchmarks.

Very good field, B • The field produces in all evaluated criteria very good outputs whose quality is close to the standard of selected benchmark institutions. The production of high-quality outputs is in line with its size and there are almost no cases, or no cases, of problematic outputs or components of creative activities. In the bibliometric analysis or peer review of selected outputs, the majority of outputs falls within the first three quartiles, or categories A, B, and C where for example less than 20% of outputs falls within the fourth quartile, or category D. A very good field approximates in terms of quality the corresponding fields at high-quality European universities predetermined as the quality benchmarks.

Average field, C • The field produces in all evaluated criteria satisfactory or good outputs. Outputs evaluated using bibliometric analysis that fall within the first two quartiles or peer reviewed as A or B are rather exceptional, similarly to outputs evaluated as D (fourth quartile). An average field approximates through its best outputs the quality of corresponding fields at very high-quality European universities predetermined as the quality benchmark.

Below average field, D • The field produces in all evaluated criteria average or below average outputs. More than half of the peer reviewed outputs fall in category C or lower, majority of outputs in bibliometric analysis is in the third or fourth quartile. In an international comparison, the field distinctly lags behind the quality of corresponding fields at very high-quality European universities predetermined as the quality benchmark.

The above criteria are used with the necessary modifications for the evaluation of research areas.

Appendix No. 7 • Assessment of the Quality of a Unit

The quality of a unit is assessed by the Creative Activities Evaluation Board with respect to the quality of selected benchmark institutions and following the below criteria, among others:

Excellent unit, A • An excellent unit is an internationally competitive unit in terms of the research parameters of global fields. More than half of the fields of the unit, representing the majority of its academic and research staff, was graded as A, a small minority, e.g., less than 10% of the fields (representing approximately ten or less percent of academic and research staff) is graded as C and none of the fields are graded as D. The unit is successful in the competition for top international grants of the ERC type, it has a functioning system for attracting and supporting junior researchers, it is significantly involved in international research networks, it is clearly internationalised in terms of both the members of the academic staff and students, it is successful in competing for national grant funds. It makes a substantial contribution to the profiling of the relevant academic fields on the national and European levels and in these criteria is comparable to the relevant institutions of high-quality European universities that were predetermined as the quality benchmarks.

Very good unit, B • A unit of well balanced and very good quality fields with very good or excellent research outputs. For example, more than half of the fields of the unit fall in category A or B, and none or in justified cases only one field falls in category D (for example a new field without history). The unit is seldom successful in the competition for top international grants of the ERC type, it has a system for attracting and supporting junior researchers, it is involved in international research networks, it has international members of academic staff and students, it is successful in competing for national grant funds. It contributes to the profiling of the relevant academic fields on the national and European levels. In the criteria monitored, the unit is comparable to the relevant institutions of high-quality European universities that were predetermined as the quality benchmarks.

Average unit, C • A unit whose fields are not so well balanced in terms of quality (ranging from below average to excellent). Excellent fields are rather rare, the majority of fields fall in category B or C, fields graded as A or B represent a small proportion (e.g., less than 20 %) of the academic and research staff of the unit. The unit is rarely successful in the competition for top international grants of the ERC type, it is to a limited extent involved in international research networks, it has very few or no international members of academic staff and students, and it obtains national grant funds. It contributes to the profiling of the relevant academic fields on the national and European levels. The unit is above average in national comparison, in the criteria monitored it lags behind the relevant institutions of high-quality European universities that were predetermined as the quality benchmarks.

Below average unit, D • A unit that is below average in the vast majority of parameters of its fields. More than a half of its fields are graded as C or D, no field is graded as A. The unit is not successful in the competition for top international grants of the ERC type, it is to a limited extent involved in international research networks, it has very few or no international members of academic staff and students, it obtains only a limited amount of national grant funds. It contributes to the profiling of the relevant academic fields on the national and European levels. The unit is average in the national comparison, in the criteria monitored it distinctly lags behind the relevant institutions of high-quality European universities that were predetermined as the quality benchmarks.

Appendix No. 8 • Conflict of Interest of Participants in Evaluation

Definition of Conflict of Interest

For the purposes of the evaluation of creative activities, a reviewer or a member of an Expert Panel is deemed to be in conflict of interest in particular if he:

• 1. Was involved in the preparation of, or is an author or co-author of, outputs that he should review;

  2. Has close family relations (e.g., a spouse or partner) or other close personal relations to a person that is author or co-author of the reviewed outputs, or holds a management position in the relevant unit;

  3. Is in any manner involved in the management of the relevant unit;

  4. Has an employment contract at the unit he should evaluate or an agreement to produce creative activities at the unit he should evaluate;

  5. Has or had a relationship of research rivalry or professional hostility with any of the authors of outputs that are reviewed or with a person in a management position in the unit that is assessed;

  6. Is or was in the past the mentor of the author of outputs that are being reviewed or was or is mentored by him.

Procedure to Resolve Conflict of Interest

• 1. Based on notification from the reviewer, Expert Panel member, or from a unit, the Creative Activities Evaluation Board decides whether there is a situation that could challenge the capacity of the reviewer or member of Expert Panel to independently assess the unit or review its outputs.

  2. If it turns out during the evaluation that a reviewer or a member of an Expert Panel intentionally failed to disclose a conflict of interest, he will be immediately excluded from the evaluation by the Creative Activities Evaluation Board and the recommendations or opinions in which he was involved shall be subject to review.