Scheduling Data-Driven Workflows in Multi-Cloud Environment

Document Type : Parallel and Distributed Processing-Deldari-Abrishami

Authors

• Nafise Sooezi
• Saeid Abrishami
• Majid Lotfian

Ferdowsi University of Mashhad

10.22067/cke.v1i2.53554

Abstract

Nowadays, cloud computing and other distributed computing systems have been developed to support various types of workflows in applications. Due to the restrictions onthe use ofone cloud provider, the concept of multiple clouds as been proposed.Inmultipleclouds, schedulingworkflowswithlarge amounts ofdata is a well-knownNP-Hard problem. The existing scheduling algorithms have not paid attention to the data dependency issues and their importance in scheduling criteria such as time and cost. In this paper, we propose a communication-based algorithm for workflows with huge volumes of data in a multi-cloud environment. The proposed algorithm changes the definition of the Partial Critical Paths(PCP) to minimize the cost of workflow executionwhile meeting a user defined deadline.

Keywords

• Cloud Computing
• Multi-cloud
• Workflow Scheduling
• Data Dependency
• Communication

---

References

[1] Petcu, D.: "Consuming resources and services from multiple clouds", Journal of Grid Computing, 12, (2), pp. 321-345, 2014.

[2] Yu, J., and Buyya, R.: "A taxonomy of scientific workflow systems for grid computing", ACM Sigmod Record, 34, (3), pp. 44-49,2005.

[3] Chen, R., Yang, M., Weng, X., Choi, B., He, B., and Li, X.: "Improving large graph processing on partitioned graphs in the cloud", in Editor (Ed.)^(Eds.): "Book Improving large graph processing on partitioned graphs in the cloud", (ACM, 2012, edn.), pp. 3.

[4] Li, A., Yang, X., Kandula, S., and Zhang, M.: "CloudCmp: comparing public cloud providers", in Editor (Ed.)^(Eds.): "Book CloudCmp: comparing public cloud providers" (ACM, edn.), pp. 1-14, 2010.

[5] Wang, G., and Ng, T.E.: "The impact of virtualization on network performance of amazon ec2 data center", in Editor (Ed.)^(Eds.): "Book The impact of virtualization on network performance of amazon ec2 data center" (IEEE, edn.), pp. 1-9, 2010.

[6] Abrishami, S., Naghibzadeh, M., and Epema, D.H.: "Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds", Future Generation Computer Systems, 29, (1), pp. 158-169, 2013.

[7] Wu, Z., Ni, Z., Gu, L., and Liu, X.: "A revised discrete particle swarm optimization for cloud workflow scheduling", in Editor (Ed.)^(Eds.): "Book A revised discrete particle swarm optimization for cloud workflow scheduling" (IEEE, edn.), pp. 184-188, 2010.

[8] Pandey, S., Wu, L., Guru, S.M., and Buyya, R.: "A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments", in Editor (Ed.)^(Eds.): "Book A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments" (IEEE, edn.), pp. 400-407, 2010.

[9] Yu, J., and Buyya, R.: "Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms", Scientific Programming, 14, (3-4), pp. 217-230, 2006.

[10] Genez, T.A., Bittencourt, L.F., and Madeira, E.R.: "Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels", in Editor (Ed.)^(Eds.): "Book Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels" (IEEE, edn.), pp. 906-912, 2012.

[11] Bittencourt, L.F., and Madeira, E.R.M.: "HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds", Journal of Internet Services and Applications, 2, (3), pp. 207-227, 2011.

[12] Sakellariou, R., Zhao, H., Tsiakkouri, E., and Dikaiakos, M.D.: "Scheduling workflows with budget constraints","Integrated research in GRID computing", pp. 189-202, Springer, 2007.

[13] Topcuoglu, H., Hariri, S., and Wu, M.-y.,"Performance-effective and low-complexity task scheduling for heterogeneous computing", Parallel and Distributed Systems, IEEE Transactions on, 13, (3), pp. 260-274, 2002.

[14] Van den Bossche, R., Vanmechelen, K., and Broeckhove, J.,"Cost-optimal scheduling in hybrid iaas clouds for deadline constrained workloads", in Editor (Ed.)^(Eds.): "Book Cost-optimal scheduling in hybrid iaas clouds for deadline constrained workloads" (IEEE, edn.), pp. 228-235, 2010.

[15] Houidi, I., Mechtri, M., Louati, W., and Zeghlache, D.,"Cloud service delivery across multiple cloud platforms", in Editor (Ed.)^(Eds.),"Book Cloud service delivery across multiple cloud platforms" (IEEE, edn.), pp. 741-742, 2011.

[16] Li, W., Tordsson, J., and Elmroth, E.,"Modeling for dynamic cloud scheduling via migration of virtual machines", in Editor (Ed.)^(Eds.),"Book Modeling for dynamic cloud scheduling via migration of virtual machines" (IEEE, edn.), pp. 163-171, 2011.

[17] Fard, H.M., Prodan, R., and Fahringer, T.,"A truthful dynamic workflow scheduling mechanism for commercial multicloud environments", Parallel and Distributed Systems, IEEE Transactions on, 24, (6), pp. 1203-1212, 2013.

[18] Fard, H.M., Prodan, R., Barrionuevo, J.J.D., and Fahringer, T.,"A multi-objective approach for workflow scheduling in heterogeneous environments", in Editor (Ed.)^(Eds.),"Book A multi-objective approach for workflow scheduling in heterogeneous environments" (IEEE Computer Society, edn.), pp. 300-309, 2012.

[19] Duan, R., Prodan, R., and Li, X.,"Multi-objective game theoretic schedulingof bag-of-tasks workflows on hybrid clouds", Cloud Computing, IEEE Transactions on, 2, (1), pp. 29-42, 2014.

[20] Montes, J.D., Zou, M., Singh, R., Tao, S., and Parashar, M.,"Data-driven workflows in multi-cloud marketplaces", in Editor (Ed.)^(Eds.),"Book Data-driven workflows in multi-cloud marketplaces" (IEEE, edn.), pp. 168-175, 2014.