Cross-Protocol Data-Integrity Validation During Live Migration Between NVMe and Fibre Channel Storage Paths
Article Sidebar
Main Article Content
Abstract
Any data corruption that happens between the two storage paths while they are both active can go unnoticed. The 20 records supplied were used to conduct a systematic review of how data-integrity validation can be organized as the volumes move between Fibre Channel (FC) and Non-Volatile Memory Express (NVMe) paths. Evidence of the effects of storage errors, flash media, fabric performance, protocol digests, and migration mechanics was aggregated and treatment was added to an analytical model of validation cost and residual risk. Field data revealed 0.86% disks with checksum mismatches on nearline disks and 0.065% FC enterprise disks with checksum mismatches; 3.45% disks with latent sector errors. The application throughput was 20% lower with remote flash access over the Internet Small Computer System Interface (iSCSI) than without it, while NVMe over Fabrics introduced 11.7 µs of latency. With a corruption probability of 10⁻⁷ per block, the model predicted that about 6.25×10⁻⁹ blocks would be corrupted each time 1 TiB was migrated, but, as the 4 GiB/s full inline validation showed, pre-copy migration was not convergent. A compromise was reached by sampling the digest on the fly during the copying process and then comparing the final digest with the on disk digest. The main research gap identified is having no direct empirical study on FC-to-NVMe migration.
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
References
[1] L. N. Bairavasundaram, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, G. R. Goodson, and B. Schroeder, “An analysis of data corruption in the storage stack,” ACM Trans. Storage, vol. 4, no. 3, pp. 1–28, 2008, doi: https://doi.org/10.1145/1416944.1416947.
[2] W. Voorsluys, J. Broberg, S. Venugopal, and R. Buyya, “Cost of virtual machine live migration in clouds: A performance evaluation,” in IEEE international conference on cloud computing, Springer, 2009, pp. 254–265. doi: https://doi.org/10.1007/978-3-642-10665-1_23.
[3] A. Kashyap, S. Gugnani, and X. Lu, “Impact of commodity networks on storage disaggregation with nvme-of,” in International Symposium on Benchmarking, Measuring and Optimization, Springer, 2020, pp. 41–56. doi: https://doi.org/10.1007/978-3-030-71058-3_3.
[4] L. N. Bairavasundaram, G. R. Goodson, S. Pasupathy, and J. Schindler, “An analysis of latent sector errors in disk drives,” in Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, 2007, pp. 289–300. doi: https://doi.org/10.1145/1254882.1254917.
[5] B. Schroeder, S. Damouras, and P. Gill, “Understanding latent sector errors and how to protect against them,” ACM Trans. storage, vol. 6, no. 3, pp. 1–23, 2010, doi: https://doi.org/10.1145/1837915.1837917.
[6] V. Prabhakaran, L. N. Bairavasundaram, N. Agrawal, H. S. Gunawi, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, “IRON file systems,” in Proceedings of the Twentieth ACM Symposium on Operating Systems Principles, in SOSP ’05. New York, NY, USA: Association for Computing Machinery, 2005, pp. 206–220. doi: 10.1145/1095810.1095830.
[7] J. Meza, Q. Wu, S. Kumar, and O. Mutlu, “A Large-Scale Study of Flash Memory Failures in the Field,” in Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, in SIGMETRICS ’15. New York, NY, USA: Association for Computing Machinery, 2015, pp. 177–190. doi: 10.1145/2745844.2745848.
[8] Q. Xu et al., “Performance analysis of NVMe SSDs and their implication on real world databases,” in Proceedings of the 8th ACM International Systems and Storage Conference, in SYSTOR ’15. New York, NY, USA: Association for Computing Machinery, 2015. doi: 10.1145/2757667.2757684.
[9] A. Klimovic, C. Kozyrakis, E. Thereska, B. John, and S. Kumar, “Flash storage disaggregation,” in Proceedings of the Eleventh European Conference on Computer Systems, in EuroSys ’16. New York, NY, USA: Association for Computing Machinery, 2016. doi: 10.1145/2901318.2901337.
[10] M. R. Hines, U. Deshpande, and K. Gopalan, “Post-copy live migration of virtual machines,” SIGOPS Oper. Syst. Rev., vol. 43, no. 3, pp. 14–26, 2009, doi: 10.1145/1618525.1618528.
[11] F. Zhang, G. Liu, X. Fu, and R. Yahyapour, “A Survey on Virtual Machine Migration: Challenges, Techniques, and Open Issues,” IEEE Commun. Surv. Tutorials, vol. 20, no. 2, pp. 1206–1243, 2018, doi: 10.1109/COMST.2018.2794881.
[12] A. Strunk, “Costs of Virtual Machine Live Migration: A Survey,” in 2012 IEEE Eighth World Congress on Services, 2012, pp. 323–329. doi: 10.1109/SERVICES.2012.23.
[13] Y. Cai, S. Ghose, E. F. Haratsch, Y. Luo, and O. Mutlu, “Error Characterization, Mitigation, and Recovery in Flash-Memory-Based Solid-State Drives,” Proc. IEEE, vol. 105, no. 9, pp. 1666–1704, 2017, doi: 10.1109/JPROC.2017.2713127.
[14] Z. Guz, H. (Huan) Li, A. Shayesteh, and V. Balakrishnan, “NVMe-over-fabrics performance characterization and the path to low-overhead flash disaggregation,” in Proceedings of the 10th ACM International Systems and Storage Conference, in SYSTOR ’17. New York, NY, USA: Association for Computing Machinery, 2017. doi: 10.1145/3078468.3078483.
[15] Z. Guz, H. (Huan) Li, A. Shayesteh, and V. Balakrishnan, “Performance Characterization of NVMe-over-Fabrics Storage Disaggregation,” ACM Trans. Storage, vol. 14, no. 4, pp. 1–18, 2018, doi: 10.1145/3239563.
[16] H. S. Gunawi et al., “Fail-Slow at Scale: Evidence of Hardware Performance Faults in Large Production Systems,” ACM Trans. Storage, vol. 14, no. 3, pp. 1–26, 2018, doi: 10.1145/3242086.
[17] D. Sheinwald, J. Satran, P. Thaler, and V. Cavanna, “Internet protocol small computer system interface (iSCSI) cyclic redundancy check (CRC)/checksum considerations (RFC 3385),” 2002, doi: https://doi.org/10.17487/RFC3385.
[18] M. Chadalapaka, J. Satran, K. Meth, and D. Black, “Internet small computer system interface (iSCSI) protocol (consolidated) (RFC 7143),” 2014, doi: https://doi.org/10.17487/RFC7143.
[19] M. R. Hines and K. Gopalan, “Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning,” in Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, in VEE ’09. New York, NY, USA: Association for Computing Machinery, 2009, pp. 51–60. doi: 10.1145/1508293.1508301.
[20] Z. Lei, E. Sun, S. Chen, J. Wu, and W. Shen, “A novel hybrid-copy algorithm for live migration of virtual machine,” Futur. Internet, vol. 9, no. 3, p. 37, 2017, doi: https://doi.org/10.3390/fi9030037.