Deakin University
Browse

File(s) under permanent embargo

Bee colony based worker reliability estimation algorithm in microtask crowdsourcing

conference contribution
posted on 2017-01-31, 00:00 authored by Alireza Moayedikia, K L Ong, Y L Boo, William YeohWilliam Yeoh
© 2016 IEEE. Estimation of worker reliability on microtask crowdsourcing platforms has gained attention from many researchers. On microtask platforms no worker is fully reliable for a task and it is likely that some workers are spammers, in the sense that they provide a random answer to collect the financial reward. Existence of spammers is harmful as they increase the cost of microtasking and will negatively affect the answer aggregation process. Hence, to discriminate spammers and non-spammers one needs to measure worker reliability to predict how likely that a worker put an effort in solving a task. In this paper we introduce a new reliability estimation algorithm works based on bee colony algorithm called REBECO. This algorithm relies on Gaussian process model to estimate reliability of workers dynamically. With bees that go in search of pollen, some are more successful than the others. This maps well to our problem, where some workers (i.e., bees) are more successful than other workers for a given task thus, giving rise to a reliability measure. Answer aggregation with respect to worker reliability rates has been considered as a suitable replacement for conventional majority voting. We compared REBECO with majority voting using two real world datasets. The results indicate that REBECO is able to outperform MV significantly.

History

Event

Machine Learning and Applications. Conference (2016 : 15th : Anaheim, California)

Pagination

713 - 717

Publisher

IEEE

Location

Anaheim, California

Place of publication

Piscataway, N.J.

Start date

2016-12-18

End date

2015-12-20

ISBN-13

9781509061662

Language

eng

Publication classification

E Conference publication; E1 Full written paper - refereed

Copyright notice

2016, IEEE

Title of proceedings

Proceedings - 2016 15th IEEE International Conference on Machine Learning and Applications, ICMLA 2016