Schema-Evolution-Safe Polyglot Persistence for Containerized Microservices Using Event-Carried State Transfer A Literature Synthesis
Article Sidebar
Main Article Content
Abstract
Polyglot persistence is the new approach for container-orchestrated microservices, where each microservice chooses a data-store technology that best fits its workload; and event-carried state transfer, where state changes are carried as immutable events through a shared log, instead of directly through synchronous calls between services. This literature survey investigates the interplay of three research lines: schema evolution in heterogeneous NoSQL and polyglot data stores, event-driven integration patterns to minimize synchronous coupling and distributed transaction management by the saga pattern. The review is based on the distributed-systems theory, empirical microservice-migration studies, and schema-evolution measurement studies, to synthesize a conceptual framework that enables safe, independently evolvable microservice data layers through schema-registry-governed compatibility checking, event-carried state transfer, and compensation-based transaction management. The synthesis emphasizes the classic dilemma of consistency vs availability, as summarized by the CAP theorem, as well as the operational rigidity of production database schemas, and the flexibility that NoSQL stores with schema-on-read are offered but not guaranteed. Four conceptual diagrams and three comparative tables are used to summarize the reviewed material as an integrated architectural viewpoint, whereas two formal notations are used to summarize the consistency and transaction-compensation reasons that underlie the framework. The review finds that schema governance, compatibility contracts and compensating transactions are complementary mechanisms that do not compete and finds open challenges around tooling maturity, consistency on cross-store data and schema-level lifecycle management at the container-orchestration level.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
[1] J. Lewis and M. Fowler, “Microservices: a definition of this new architectural term,” MartinFowler. com, vol. 25, no. 14–26, p. 12, 2014, [Online]. Available: https://martinfowler.com/articles/microservices.html
[2] S. Newman, Building microservices: designing fine-grained systems. “ O’Reilly Media, Inc.,” 2021.
[3] P. J. Sadalage and M. Fowler, NoSQL distilled: a brief guide to the emerging world of polyglot persistence. Pearson Education, 2013. doi: 10.5555/2381014.
[4] N. Dragoni et al., “Microservices: yesterday, today, and tomorrow,” Present ulterior Softw. Eng., pp. 195–216, 2017, doi: 10.1007/978-3-319-67425-4_12.
[5] E. Casalicchio, “Container orchestration: A survey,” Syst. Model. Methodol. tools, pp. 221–235, 2018, doi: 10.1007/978-3-319-92378-9_14.
[6] C. ASHWIN, “Exploring event-driven architecture in microservices-patterns, pitfalls and best practices,” Int. J., vol. 4, no. 1, pp. 229–249, 2021, doi: 10.30574/ijsra.2021.4.1.0166.
[7] M. Kleppmann, Designing data-intensive applications. O’Reilly Beijing, 2017.
[8] P. Vassiliadis, A. V Zarras, and I. Skoulis, “Gravitating to rigidity: Patterns of schema evolution–and its absence–in the lives of tables,” Inf. Syst., vol. 63, pp. 24–46, 2017, doi: 10.1016/j.is.2016.06.010.
[9] S. Scherzinger and S. Sidortschuck, “An empirical study on the design and evolution of NoSQL database schemas,” in International Conference on Conceptual Modeling, Springer, 2020, pp. 441–455. doi: 10.1007/978-3-030-62522-1_33.
[10] P. Bailis and A. Ghodsi, “Eventual consistency today: Limitations, extensions, and beyond,” Commun. ACM, vol. 56, no. 5, pp. 55–63, 2013, doi: 10.1145/2447976.2447992.
[11] S. Gilbert and N. Lynch, “Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services,” Acm Sigact News, vol. 33, no. 2, pp. 51–59, 2002, doi: 10.1145/564585.564601.
[12] P. Di Francesco, P. Lago, and I. Malavolta, “Architecting with microservices: A systematic mapping study,” J. Syst. Softw., vol. 150, pp. 77–97, 2019, doi: 10.1016/j.jss.2019.01.001.
[13] A. Davoudian, L. Chen, and M. Liu, “A survey on NoSQL stores,” ACM Comput. Surv., vol. 51, no. 2, pp. 1–43, 2018, doi: 10.1145/3158661.
[14] J. Kazanavičius, D. Mažeika, and D. Kalibatienė, “An approach to migrate a monolith database into multi-model polyglot persistence based on microservice architecture: A case study for mainframe database,” Appl. Sci., vol. 12, no. 12, p. 6189, 2022, doi: 10.3390/app12126189.
[15] H. Singhal, A. Saxena, N. Mittal, C. Dabas, and P. Kaur, “PolyGlot persistence for microservices-based applications,” Int. J. Inf. Technol. Syst. Approach, vol. 14, no. 1, pp. 17–32, 2021, doi: 10.4018/IJITSA.2021010102.
[16] M. Klettke, U. Störl, M. Shenavai, and S. Scherzinger, “NoSQL schema evolution and big data migration at scale,” in 2016 IEEE International Conference on Big Data (Big Data), IEEE, 2016, pp. 2764–2774. doi: 10.1109/BigData.2016.7840924.
[17] A. Hillenbrand, U. Störl, M. Levchenko, S. Nabiyev, and M. Klettke, “Towards self-adapting data migration in the context of schema evolution in NoSQL databases,” in 2020 IEEE 36th International Conference on Data Engineering Workshops (ICDEW), IEEE, 2020, pp. 133–138. doi: 10.1109/ICDEW49219.2020.00013.
[18] A. Chavan, “Eventual consistency vs. strong consistency: Making the right choice in microservices,” Int. J. Softw. Appl., vol. 14, no. 3, pp. 45–56, 2021, doi: 10.30574/ijsra.2021.1.2.0036.
[19] H. Garcia-Molina and K. Salem, “Sagas,” ACM Sigmod Rec., vol. 16, no. 3, pp. 249–259, 1987, doi: 10.1145/38713.38742.
[20] C. Richardson, Microservices patterns: with examples in Java. Simon and Schuster, 2018.
[21] E. Daraghmi, C.-P. Zhang, and S.-M. Yuan, “Enhancing saga pattern for distributed transactions within a microservices architecture,” Appl. Sci., vol. 12, no. 12, p. 6242, 2022, doi: 10.3390/app12126242.
[22] K. Reza and N. Rahman, “Explication and Extension of Saga and Microservice Patterns to Enable Resilient Distributed Transaction,” in Inventive Communication and Computational Technologies: Proceedings of ICICCT 2022, Springer, 2022, pp. 209–220. doi: 10.1007/978-981-19-4960-9_18.
[23] D. Taibi, V. Lenarduzzi, and C. Pahl, “Processes, motivations, and issues for migrating to microservices architectures: An empirical investigation,” IEEE Cloud Comput., vol. 4, no. 5, pp. 22–32, 2017, doi: 10.1109/MCC.2017.4250931.