1st workshop on critical evaluation of 

generative models and their impact on society

September 29, 2024 from 9:00 to 13:00

at ECCV 2024, Milan, Italy

Call for papers

Format. We call for novel work and work in progress that has not been published elsewhere. Submissions must follow the ECCV 2024 template and will be peer-reviewed in a single-blind fashion. We welcome the following formats:

• Full papers: 14 pages (excluding references).  By default, accepted papers will be included in the ECCV workshop proceedings, unless authors choose to opt out of inclusion in the proceedings, in which case the paper should be made available on arXiv.

• Extended abstracts: 4 pages (excluding references). Accepted extended abstracts will not be published in the proceedings but should be made available on arXiv. 

Additionally, we also accept recently published papers elsewhere for poster presentation; e.g., CVPR, NeurIPS, EMNLP, ECCV main conference. We do not review these papers again and it is not necessary to change the format to the ECCV 2024 template. A jury of organizers will select these papers.

Topics. The workshop revolves around two main topics, visual quality assessment and impact on society:

• Visual generation quality assessment 

  • Image-quality evaluation metrics aligned with human perception. 

  • Language and vision alignment metrics in text-to-image generation. 

  • Protocols for reproducible human evaluation.

  • New benchmarks for visual generative models. 

• Visual generation impact on society

  • Data audition and analysis. 

  • Social bias evaluation. 

  • Privacy threads detection. 

  • Intellectual property violation detection. 

  • Impact on the informational environment.

  • Impact on the cultural environment. 

  • Impact on the natural environment. 

Submission site: https://cmt3.research.microsoft.com/CEGIS2024

Important Dates

• Paper submission: July 15th, July 25th, July 29th 2024 (final deadline, no more extensions!).

• Notification to authors: August 5th August 12th, 2024.

• Camera-ready submission: August 15th August 21th, 2024.

• Workshop: September 29th, 2024.

All dates are 11:59PM, AoE.

Related work

• Bianchi et al. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. ACM FAccT 2023.

• Carlini et al. Extracting training data from diffusion models. USENIX Security Symposium 2023.

• Ding et al. Cogview2: Faster and better text-to-image generation via hierarchical transformers. NeurIPS 2022.

• Hessel et al. Clipscore: A reference-free evaluation metric for image captioning. EMNLP 2021.

• Heusel et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium. NeurIPS 2017.

• Jiang et al. AI art and its impact on artists. AAAI/ACM AIES  2023.

• Katirai et al. Situating the social issues of image generation models in the model life cycle: a sociotechnical approach. arXiv 2023. 

• Luccioni et al. Stable bias: Analyzing societal representations in diffusion models. NeurIPS D&B 2023. 

• Otani et al. Toward verifiable and reproducible human evaluation for text-to-image generation. CVPR 2023. 

• Somepalli et al. Diffusion art or digital forgery? Investigating data replication in diffusion models. CVPR 2023.