Project Results‎ > ‎

Dimensioning of Crowdsourcing Platforms

Crowdsourcing platforms provide an easy and scalable access to human workforce that can, e.g., provide subjective judgements, tagging information, or even generate knowledge. In conjunction with machine clouds offering scalable access to computing resources, these human cloud provider offer numerous possibilities for creating new applications which would not have been possible a few years ago. However, in order to build sustainable services on top of this inter-cloud environment, scalability considerations have to be made. While cloud computing systems are already well studied in terms of dimensioning of the hardware resources, there still exists little work on the appropriate scaling of crowdsourcing platforms. This is especially challenging, as the complex interaction between all involved stakeholders, platform providers, workers and employers have to be considered.

In [1] we addressed these challenges. First we develop a model for common crowdsourcing platforms and implement the model using a simulative approach, which is validated with a comparison to an analytic M[X]/M/c − ∞ system. In a second step we evaluate inter-arrival times as well as campaign size distributions based on a dataset of a large commercial crowdsourcing platform to derive realistic model parameters and illustrate the differences to the analytic approximation. Finally, we perform a parameter study using the simulation model to derive guidelines for dimensioning crowdsourcing platforms, while considering relevant parameters for the involved stakeholders, i.e., the delay before work on a task begins and the work load of the workers.


https://sites.google.com/site/dfgcrowdsourcing/project-results/dimensioning-of-crowdsourcing-platforms/Tradeoff%20worker%20utilization%20and%20tas%20pre-processing%20delay.png

    References

    [1] Christian Schwartz, Kathrin Borchert, Matthias Hirth und Phuoc Tran-Gia. „Modeling Crowdsourcing Platforms to Enable Workforce Dimensioning“. In: Proceedings of Telecommunication Networks and Applications Conference. Sydney, New South Wales, Nov. 2015.