This workshop will consist of a series of invited talks and panels covering the following topics.
- Ethics of Experiments for Scholarly Knowledge versus Business Interest
- Should the ethical questions asked of experiments conducted for scholarly knowledge be different from those asked of experiments conducted for business interest?
- What are the ethics of experiments or analysis that have both objectives?
- Review Processes
- What are existing experiment review practices for online experiments in academia and industry?
- In an industrial setting, what are the differences between review of experiments for product development and experiments for academic research?
- How do review processes scale in the number of users or experiments?
- Informed Consent
- What are the best mechanism for ensuring informed consent at scale?
- When is informed consent required for online experiments?
- Pedagogy
- How should the training of current and future data scientists and researchers be changed, if at all?
- Ethics in Automated Experimentation (e.g. bandit algorithms/explore-exploit)
- How do we audit automated experimentation systems to detect unethical experimentation?
- How do we optimize automated experimentation systems to avoid unethical experimentation?
- Ethical Obligation to Experiment
- In what situations do researchers or practitioners have an ethical obligation to experiment?
- Case Studies
- information retrieval/web search
- news filtering/recommendation
- advertising/product recommendation
- health
- online dating
- job search