ExOrdo's "The No-Panic Guide to Organising a Great Research Conference"
ExOrdo's "How to rescue your research conference by going virtual"
When we converted from the legacy software to Ex Ordo, we updated all the codes. We did so by obtaining the content/methods codes from JCP, ACR, JCR, JM and JMR and creating an expanded list. It’s important to have enough codes because the Ex Ordo software uses them to allocate to reviewers and AEs/chairs. We encouraged reviewers, AEs and Chairs to specify as many codes as they could, since this provides a better resource to get good reviews done. Authors of submissions also must choose at least 1 and up to 4 in total (no more).
We also learned that the list is a bit political. We invited reviewers who turned us down because when they went to enter their own codes, they could not find the ones they thought best represented their work. In fact, in follow up communications, we always found they had misread the list and their preferred codes were in there. The larger point is that some people who you’ll interact with already feel annoyed with or antagonized by ACR (e.g. ACR doesn’t respect my research area, ACR does too little to serve international members, ACR caters to or is run by elites etc…).
One of the important methods codes is “Films (only for Film Festival Format)”. Not many people want to review Films, so you need to keep this to identify reviewers interested in that content.
(We shared this list with SCP, which is also using Ex Ordo for their next conference, so don’t be surprised to learn the codes are identical across conferences.)
We had not intended to use all these categories as criteria to accept or reject submissions. For example, “Novelty” was added due to an initiative between ACR and Marketing Letters to help the journal’s editors identify possible submissions for their journal and “Awards” was intended to help us identify truly outstanding submissions as award candidates.
The way the system is set up now, it treats these seven criteria as equal and uses all to create an overall score for each submission. Our goal was to have AEs/Chairs focus on the Topic, Execution, Contribution, Confidence and Acceptance criteria (+ written comments). We did not want the Awards and Novelty criteria to be included (both categories drastically reduce the overall score, leading us to be concerned about the possibility of false negatives – rejecting a paper that is actually decent) but we didn’t figure out how to configure the criteria to only use a subset to calculate the overall score. Hopefully you can fix this.
Our work-around was to go into “Excel Exports” and download the “Review” file. Then, in Excel, you can rank and sort the evaluations and play around with weighting (e.g. average of Topic, Contribution, Execution and Acceptance weighted by Confidence etc…). Once reviews were all in, we provided each AE and Working Paper chair an excel file with all their submissions plus the criteria scores for each so they could use them to help make their decisions.
Finally, we opted to not provide criteria scores to authors. We worried about getting into discussions about averages and distributions etc…
Before you open up the system to allow people to submit, you should do many dry-runs (fake submissions) with different types of papers, try to upload files, make sure content/methods codes work etc… Ensure all the instructions, questions, fields, processes etc.. work. We did loads of testing and we still had issues we had to iron out in the first few days.
Here are the submission instructions we provided authors. You’ll have a chance to design your own of course, but perhaps this is helpful.
Welcome! Here are some of the basics:
By submitting, you are agreeing that if applicable, your research has received an approval or a waiver from your Institutional Review Board (IRB); that your research paper, session, or poster represents accurately the data that you collected for this research project; that the research submitted is your own original work; that you understand that the research submitted may be run through anti-plagiarism software; that you have not submitted any paper to more than one track (e.g. competitive sessions and special session); that no author may be listed as a presenter for more than two submissions; that you are not submitting a paper that has been published, or accepted for publication, at the time of submission (including online publications and full papers published in ACR Proceedings); that you are not submitting content that has been presented at any earlier ACR conferences; and that your submission has adhered to word and page limits.
Do not submit a paper that has been published or accepted for publication elsewhere. Submissions should not include content that has been presented at any earlier ACR conference.
No author may be listed as a presenter for more than two submissions (i.e. competitive papers, special sessions). An author who is not presenting may be listed on more than two submissions.
Submissions should be in Word format and should adhere to the word and page limits.
Be sure to use consistent author/co-author information in every submission. Please check with your co-authors about how they want to be listed for official publication purposes. PLEASE BE CAREFUL WITH THIS. The database will consider Drew Blank, Drew A. Blank, and Drew Avery Blank as three different authors and may result in a program that has Drew presenting at the same time.
Please ensure you list all authors involved in a submission. This means any person who is an author on the submission (e.g. Competitive Paper) or on any part of the submission (e.g. a paper included as part of a Special Session) must be listed even if that person is not a presenter. This is critical to ensure researchers do not review their own submissions.
All submissions must indicate the type of submission (e.g. Competitive Paper, Knowledge Forum etc…) and the submitting author’s name and email.
During the submission process, the submitting authors will be asked to select topics from two lists: the first list contains Research Codes and the second list contains Methodology Codes. IN TOTAL (i.e. across both lists), you may select up to 4 codes. Please review both lists before making selections. These are critical for assigning reviewers.
After you have selected your topics, proceed to the track that corresponds to your submission type. The tracks are arranged in the following order: Competitive Paper, Special Sessions, Working Paper, Knowledge Forums, Film Festival, Globe Trotting. Using the button at the bottom of each page, proceed to the location that corresponds to your submission type and provide the information asked. Once done, again use the button at the bottom to click through to the end of the submission process. That is, simply skip all the tracks that do not match your submission type.
The submitting author will automatically receive an email acknowledging the submission. If you do not receive an acknowledgment within 48 hours after submission (please also check your spam folder), send an email inquiry to ACRSeattle2021@gmail.com
There are not separate tracks for each type of submission. A submitting author simply clicks through until they find the kind of submission they want (e.g. Special Session, Working Paper), answers the questions there, then clicks past the other submission types that are not relevant to that submission. It seemed to not present any issues for submitters (we certainly didn’t hear a lot of problems with it).
Once the call for papers is up and the submission system is announced, people will go and ‘try it out’. They’ll start a submission then abandon it. Sometimes they will ‘complete’ a submission but forget to upload the Word file. Often authors will contact you saying they did not get the automatic email confirming receipt (ask them to check their junk email folder – the system is quite reliable). Some authors assume that if they upload a document, they can go back before the deadline and upload a new version (they cannot - you could manage the process for them by manually uploading a new version for them, but it’s best to have authors withdraw their old/wrong submission and submit a new one, or else you may inadvertently send out both versions for review or screw up the submission somehow. Because you’re dealing with thousands of files and authors, it’s probably best to put the work back on the submitting authors).
The point is, there is all sorts of messiness. You can withdraw an incomplete file but you should email the corresponding author first to make sure it wasn’t an honest mistake. DO NOT DELETE submissions. They’re gone for good then. If you withdraw a submission (because of a missing file, for example), but the authors email it to you, you can reinstate a withdrawn file and manually upload the missing file. You cannot do that for a deleted file.
It’s a good idea, at the submission deadline, to go through the Ex Ordo list of incomplete submissions and check – does it look like someone tried to submit something and made a mistake? Just send the corresponding author an email through Ex Ordo saying “hey we noticed your submission (#ABC) is not complete – is this on purpose?” or whatever. You want to avoid a person months later contacting you and saying “what happened to my submission!?” when it was just a simple mistake. We found people were very responsive when contacted.
Everyone for all kinds of submissions need to upload a final version. Competitive Paper authors need to select Option 1 or Option 2, which correspond to publishing the full paper or just an abstract in the Proceedings. Most authors will make changes to their 50 word abstract. Author orders and titles change etc…Many people think that “I’m not planning to make changes, so I won’t do anything”. They assume the original submission carries over. It will not. So, you’ll have to chase down a bunch of people and explain (even though you’ll have reminded them in writing many times at this point) that all submissions must be uploaded a final time and that all submissions must come with a copyright form, even if the person is only uploading a working paper 50-word abstract etc… You’ll have to manually upload a bunch of stuff too – once you close out the system, people who are late cannot upload their content.
Below is an example of a reminder email with this information:
This is a reminder that ALL FINAL SUBMISSIONS for ACR 2021 are due July 15.
EVEN IF YOU PLAN TO MAKE NO CHANGES to your original, all submissions must be uploaded as a final submission by 11:59 EST.
Thank you.
Tonya, Anat and Matt
________________________________
Instructions:
These are instructions for submitting your final documents for publication in the ACR 2021 conference proceedings and/or program.
Even if you plan to make no changes to your original, all submissions must be uploaded as a FINAL SUBMISSION by Thursday, July 15, 2021, 11:59 pm Eastern Standard Time (New York Time).
To do so, log into acr2021.exordo.com, find “My Final Submissions” on your author card and follow the prompts, which correspond to the information below.
GENERAL GUIDELINES
Final submissions to be included in ACR Proceedings should be revised to be responsive to reviewer/AE comments.
All submissions must be in Microsoft Word (12-point Times New Roman and 1-inch margins) and include the title (Initial Caps and bolded), author names and affiliations.
If an author has multiple submissions, be sure to use consistent author information (e.g. name spelling) across submissions. If you are submitting on behalf of co-authors, check with them about how they want to be listed for official publication purposes. Please be careful with this: the database will treat Drew Blank, Drew A. Blank, and Drew Avery Blank as three different authors and may result in a program with Drew presenting at the same time in different locations.
The corresponding author will be required to electronically sign and submit a copyright release during the submission process.
Successfully submitting a session will be confirmed with an email from Ex Ordo within 24 hours. If you do not receive a confirmation, contact us at ACRSeattle2021@gmail.com.
Below, we provide specific instructions for each type of submission.
COMPETITIVE PAPERS
Option 1: Publish the full paper in the ACR Proceedings. Authors choosing this option will upload a 50-word abstract that concentrates on the big picture contribution(s) of the paper, a 2500-word paper, references, one table summarizing the results, and one figure (optional). References, table and optional figure are not included in the word limit.
Option 2: If authors wish to submit an accepted paper elsewhere for publication, they will upload a 50-word abstract that concentrates on the big picture contribution(s) of the paper, a 750 to 1000-word extended abstract to appear in the ACR Proceedings, and references. Tables and figures are not permitted. References are not included in the word limit.
Competitive papers will be presented using pre-recorded videos of about 10 minutes. These will be due September 1. Instructions on how to upload those presentations will follow. However, all written portions (abstracts/extended abstracts/full papers – as above) are due July 15.
SPECIAL SESSIONS
Special Sessions will be published in the ACR Proceedings with the session overview and each of the papers comprising the session. Session organizers must upload a 50-word abstract that concentrates on the big picture of the session, and the 500-word overview describing the session (i.e. intent, why the topic is important, and how the papers fit together). This is to be followed by the extended abstracts (750-1000 words each) and references for each paper.
With the exception of the 50-word abstract, which can be pasted into Ex Ordo during final submission, all the content above should be uploaded as a single Word file. Organizers will also be asked to indicate the name and affiliation of the Session Discussant and/or Chair.
WORKING PAPERS
The final version of the 50-word abstract must be uploaded with the title and authors’ names and affiliations for publication in the ACR Proceedings.
KNOWLEDGE FORUMS
Knowledge Forums will have a 50-word abstract published. Accepted Forums will need to be uploaded with all presenters’ names and affiliations for publication in the ACR Proceedings.
FILM FESTIVAL
Films will have an abstract of up to 300 words (1,000 words in the films with commentary category) published in the ACR Proceedings. Abstracts should include intended contribution to knowledge, literature foundation, research method, findings and implications, and references. References are not included in the word limit.
At the end of the structured abstract (or commentary where applicable), which will be published on the ACR website, a Vimeo link to the film should be included so each film can be viewed without using a password, OR an email address should be supplied so that the person wanting to view a film can contact the lead filmmaker and request a link.
While abstracts are due July 15, authors of films will also provide the Films Co-Chairs a trailer by August 19 (using a Vimeo link) and access to the final film by September 1. Instructions on how to upload final films will follow.
GLOBE TROTTING
Globe Trotting sessions will have a 50-word abstract published in the conference program.
Questions? Please email ACRSeattle2021@gmail.com