Рус Eng During last 365 days Approved articles: 1924,   Articles in work: 305 Declined articles: 811 
Articles and journals | Tariffs | Payments | Your profile

Back to contents

Ontology for describing applications that use elements of crowd calculations
Ponomarev Andrei

PhD in Technical Science

Senior Researcher, St. Petersburg Institute of Informatics and Automation of the Russian Academy of Sciences.

199178, Russia, g. Saint Petersburg, ul. 14 Liniya, 39


The purpose of the work is the development of a machine-readable dictionary (ontology) for describing the scope and features of implementing software systems that use elements of crowd computing (information processing systems that include operations performed by people interacting with them via the Internet). Ontology will allow us to apply elements of semantic search to work with scientific and technical information in this relatively new but actively developing field of research, which, ultimately, should help to improve the level of organization of research in it. The ontology is constructed by the "top-down" method on the basis of analysis of the existing conceptualizations in the field of human-machine calculations described in the most frequently cited survey publications indexed in the bibliographic database Scopus. As a result, the ontology CROSS-ODF is formed, which allows describing four main characteristics of applications using the elements of crowd calculations: 1) the features of the problem for which the application was created; 2) the characteristics of the tasks formed by the system to the participants; 3) the contribution properties of the participants; 4) specific mechanisms for attracting participants and processing results. The generated ontology is written in the language OWL 2 and is published in the public domain. The developed ontology can be used in research support systems to simplify the search for crawl computing systems that have certain characteristics and experimental results of using such systems.

Keywords: OWL, Semantic Web, Research automation, Human-machine systems, Semantic technologies, Ontologies, Crowdsourcing, Crowd computing, Collective intelligence, Semantic search



Article was received:


Review date:


Publish date:


This article written in Russian. You can find full text of article in Russian here .

Wechsler D. Crowdsourcing as a method of transdisciplinary research-Tapping the full potential of participants // Futures. Elsevier Ltd, 2014. Vol. 60. P. 14–22.
Baev V., Sablok G., Minkov I. Next generation sequencing crowd sourcing at BIOCOMP: What promises it holds for us in future? // J. Comput. Sci. Elsevier B.V., 2014. Vol. 5, №
P. 325–326. 3.Ye H. (Jonathan), Kankanhalli A. Investigating the antecedents of organizational task crowdsourcing // Inf. Manag. Elsevier B.V., 2015. Vol. 52, № 1. P. 98–110.
Fraternali P. et al. Putting humans in the loop: Social computing for Water Resources Management // Environ. Model. Softw. Elsevier Ltd, 2012. Vol. 37. P. 68–77.
Nunes A. a., Galvão T., Cunha J.F.E. Urban Public Transport Service Co-creation: Leveraging Passenger’s Knowledge to Enhance Travel Experience // Procedia-Soc. Behav. Sci. Elsevier B.V., 2014. Vol. 111. P. 577–585.
Hetmank L. A Lightweight Ontology for Enterprise Crowdsourcing // Ecis. 2014. № section 4. P. Paper 886.
Thuan N.H. et al. Building an Enterprise Ontology of Business Process Crowdsourcing: a Design Science Approach // Proceedings of PACIS. 2015. № August.
Alabduljabbar R., Al-Dossari H. Ontology for Task and Quality Management in Crowdsourcing // Int. J. Comput. 2016. Vol. 22, № 1. P. 90–102.
Document Ontology [Electronic resource]. URL: (accessed: 08.06.2018).
Ontology of Rhetorical Blocks [Electronic resource]. URL: (accessed: 08.06.2018).
SPAR [Electronic resource]. URL:
Ruiz-Iniesta A., Corcho O. A review of ontologies for describing scholarly and scientific documents // CEUR Workshop Proc. 2014. Vol. 1155.
Soldatova L., Liakata M. An ontology methodology and CISP-the proposed Core Information about Scientific Papers. 2007. № December.
Soldatova L.N., King R.D. An ontology of scientific experiments // J. R. Soc. Interface. 2006. Vol. 3, № 11. P. 795–803.
Liakata M. et al. Corpora for the conceptualisation and zoning of scientific papers // Proc. Lr. 2010. P. 2054–2061.
Ashburner M. et al. Gene Ontology: tool for the unification of biology // Nat. Genet. 2000. Vol. 25, № 1. P. 25–29.
Natale D.A. et al. The Protein Ontology: a structured representation of protein forms and complexes // Nucleic Acids Res. 2011. Vol. 39, № Database. P. D539–D545.
Arp R., Smith B., Spear A.D. Building Ontologies with Basic Formal Ontology. 2015. 248 p.
Yuen M.C., King I., Leung K.S. A survey of crowdsourcing systems // Proceedings-2011 IEEE International Conference on Privacy, Security, Risk and Trust and IEEE International Conference on Social Computing, PASSAT/SocialCom 2011. 2011. P. 766–773.
Yuen M.-C., Chen L.-J., King I. A Survey of Human Computation Systems // 2009 International Conference on Computational Science and Engineering. 2009. № July 2014. P. 723–728.
King I., Li J., Chan K.T. A brief survey of computational approaches in Social Computing // 2009 International Joint Conference on Neural Networks. 2009. P. 1625–1632.
Pedersen J. et al. Conceptual foundations of crowdsourcing: A review of IS research // Proc. Annu. Hawaii Int. Conf. Syst. Sci. 2013. P. 579–588.
Quinn A.J., Bederson B.B. Human Computation: A Survey and Taxonomy of a Growing Field // Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). 2011. P. 1403–1412.
Morschheuser B., Hamari J., Koivisto J. Gamification in crowdsourcing: A review // Proceedings of the Annual Hawaii International Conference on System Sciences. 2016. Vol. 2016–March. P. 4375–4384.
Hossain M., Kauranen I. Crowdsourcing: a comprehensive literature review // Strateg. Outsourcing An Int. J. 2015. Vol. 8, № 1. P. 2–22.
Senaratne H. et al. A review of volunteered geographic information quality assessment methods // Int. J. Geogr. Inf. Sci. 2016. Vol. 8816, № May. P. 1–29.
Li G. et al. Crowdsourced Data Management: A Survey // IEEE Trans. Knowl. Data Eng. 2016. Vol. 28, № 9. P. 2296–2319.
Buettner R. A Systematic Literature Review of Crowdsourcing Research from a Human Resource Management Perspective // 48th Hawaii International Conference on System Sciences (HICSS). 2015. P. 4609–4618.
Restuccia F. et al. Quality of Information in Mobile Crowdsensing: Survey and Research Challenges. 2017. Vol. 13, № 4.
Feng W. et al. A Survey on Security, Privacy and Trust in Mobile Crowdsourcing // IEEE Internet Things J. 2017. Vol. 4662, № c. P. 1–24.
Vergara-Laurens I.J., Jaimes L.G., Labrador M.A. Privacy-Preserving Mechanisms for Crowdsensing: Survey and Research Challenges // IEEE Internet Things J. 2017. Vol. 4, № 4. P. 855–869.
Tsvetkova M. et al. Understanding Human-Machine Networks: A Cross-Disciplinary Survey // ACM Comput. Surv. 2015. Vol. 50, № 1. P. Article 12.
Durward D., Blohm I., Leimeister J.M. Is There PAPA in Crowd Work? A Literature Review on Ethical Dimensions in Crowdsourcing // The IEEE International Conference on Internet of People,. 2016.
Ghezzi A. et al. Crowdsourcing: A Review and Suggestions for Future Research // Int. J. Manag. Rev. 2017. Vol. 0. P. 1–21.
Assis Neto F.R., Santos C.A.S. Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management // Inf. Process. Manag. 2018. Vol. 54, № 4. P. 490–506.
Daniel F. et al. Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions. 2018. Vol. 0, № 0.
Ramírez-Montoya M.S., García-Peñalvo F.-J. Co-creation and open innovation: Systematic literature review // Comunicar. 2018. Vol. 26, № 54. P. 9–18.
Wazny K. Crowdsourcing’s ten years in: A review // J. Glob. Health. 2017. Vol. 7, № 2. P. 1–13.
Morschheuser B. et al. Gamified crowdsourcing: Conceptualization, literature review, and future agenda // Int. J. Hum. Comput. Stud. Elsevier Ltd, 2017. Vol. 106, № March 2016. P. 26–43.
Liang X., Yan Z. A survey on game theoretical methods in Human-Machine Networks // Futur. Gener. Comput. Syst. Elsevier B.V., 2017.
Sandkuhl K., Smirnov A., Ponomarev A. Crowdsourcing in business process outsourcing: An exploratory study on factors influencing decision making // Lecture Notes in Business Information Processing. 2016. Vol. 261.
Smirnov A., Ponomarev A. Exploring Requirements for Multipurpose Crowd Computing Framework // ESOCC 2015: Advances in Service-Oriented and Cloud Computing. Springer, 2016. P. 299–307.
Ponomarev A.V. Metody obespecheniya kachestva v sistemakh kraud-vychislenii: analiticheskii obzor // Trudy SPIIRAN. 2017. Vol. 54, № 5. P. 152–184.
Ponomarev A.V. Razmetka izobrazhenii massovogo meropriyatiya ego uchastnikami na osnove nemonetarnogo stimulirovaniya // Informatsionno-upravlyayushchie sistemy. 2017. № 3. P. 104–114.
Zagorul'ko Yu.A. Postroenie portalov nauchnykh znanii na osnove ontologii // Vychislitel'nye tekhnologii. 2007. Vol. 12, № SV 2.
Zagorul'ko Yu.A. et al. Kontseptsiya i arkhitektura tematicheskogo intellektual'nogo nauchnogo internet-resursa // Trudy XV Vserossiiskoi nauchnoi konferentsii RCDL’2013. Yaroslavl': YarGU, 2013. P. 57–62.