题名: | 多尺度通用型三维高分辨率光场重建算法研究 |
作者: | |
学号: | SX2202078 |
保密级别: | 公开 |
语种: | chi |
学科代码: | 080700 |
学科: | 工学 - 动力工程及工程热物理 |
学生类型: | 硕士 |
学位: | 工学硕士 |
入学年份: | 2022 |
学校: | 南京航空航天大学 |
院系: | |
专业: | |
研究方向: | 实验流体力学及图像处理 |
导师姓名: | |
导师单位: | |
第二导师姓名: | |
完成日期: | 2025-03-10 |
答辩日期: | 2025-03-11 |
外文题名: |
Multi-scale universal three-dimensional high-resolution light field reconstruction algorithm research |
关键词: | |
外文关键词: | Light Field Imaging ; 3D Reconstruction ; Deep Learning ; Micro-Reconstruction ; Mesoscopic Reconstruction ; Macro-Reconstruction ; Particle Image Velocimetry |
摘要: |
光场成像是近年来新兴的单帧三维成像方式,在多个领域得到了广泛关注与应用。然而,当前主流的光场成像方法普遍存在成像分辨率较低的问题,从而制约了光场成像的发展和应用。通过有效算法提高光场图像分辨率是当前的热点和难点。近年来,基于深度学习的超分辨率光场重建方法已经有效提高了光场重建的分辨率,然而现有方法无法充分利用视角间的互补信息,导致重建结果与预期仍有差距。此外,当前的算法均是针对某一尺度的光场图像进行重建,尚未存在一种可以对不同尺度光场图像均能进行高精度重建的算法。针对这些问题,本研究提出了一种精度高、重建速度快、泛化能力强且多尺度通用型光场重建算法,为高精度的多尺度光场成像提供了新技术。 |
外摘要要: |
Light-field imaging, by capturing the multidimensional information of light, is an emerging single-frame three-dimensional imaging method in recent years, which has gained wide attention and application in several fields. However, the current mainstream light-field imaging methods generally have the problem of low imaging resolution, which restricts the development and application of light-field imaging technology. Improving the resolution of light-field images through effective algorithms is currently an important method to improve the resolution of light-field imaging. In recent years, the super-resolution reconstruction method of light field images based on deep learning has effectively improved the resolution of light field reconstruction, however, the existing methods still have certain limitations, which can not make full use of the complementary information between the viewpoints, and it is difficult to effectively model the non-local attributes of the four-dimensional light field image, which leads to the reconstruction results can not reach the expected results. In addition, the current algorithms are all aimed at reconstructing light-field images at a certain scale, and there is not yet an algorithm that can reconstruct light-field images at different scales with high accuracy. Aiming at these problems, this study proposes a high-precision, multi-scale, fast reconstruction speed and strong model generalization ability of the light field reconstruction algorithm for high-precision microscopic three-dimensional sample imaging, mesoscopic three-dimensional neural imaging and three-dimensional flow field measurement to provide a new technical means. This thesis firstly introduces the development and research status of light field imaging technology in detail, and analyzes the advantages and disadvantages of the current light field acquisition technologies, including multi-sensor light field acquisition, time series light field acquisition and multiplexed light field acquisition and other types of technologies, as well as the current status of light field reconstruction technology. And then, the basic principle of the deep learning-based light field 3D reconstruction algorithm is discussed in depth, and the light field 3D reconstruction problem is analyzed. On this basis, the single-frame super-resolution image reconstruction technology and the basic principle of generative adversarial network (GAN) are introduced, and a multi-scale real-time light field volume reconstruction algorithm based on conditional adversarial is proposed, which elaborates in depth on the model structure, the design idea of the loss function, as well as the training and prediction link of the model, and the reconstruction speed of the algorithm is tested, and the results show that the reconstruction speed of this algorithm for different sizes of light field images The results show that the reconstruction speed of the algorithm for different sizes of light field images is as high as about 250 Hz, which provides a solid foundation for the subsequent experiments. Then, the performance of the algorithm in the reconstruction of microscopic light-field images is investigated in this thesis, and the reconstruction performance of the model for microscopic structures of different complexity levels is evaluated. In terms of simple structures, the algorithm was used to reconstruct light field images of 1 μm- and 10 μm-diameter microtubulin, and the average PSNRs of the reconstructed 1 μm- and 10 μm-diameter microtubulin images were 41.22 dB and 40 dB in terms of the peak signal-to-noise ratio (PSNR), respectively, which were higher than those of other existing methods by more than 5 dB; and in terms of structural similarity (SSIM), the reconstructed 1 μm-diameter microtubulin image was 41.22 dB and 40 dB, respectively. In terms of structural similarity (SSIM), the Fourier frequency domain image of 1 μm microtubulin image reconstructed by the algorithm has an SSIM of 0.8455 to the true value, and the SSIM of 10 μm microtubulin reconstructed by the algorithm has an SSIM of close to 1 to the true value, and the algorithm is tested on mitochondrial cell membranes in terms of complex structures, and the results show that the algorithm reconstructed by the algorithm can greatly reduce the artifacts and the imaging bias in the small field of view of mitochondrial cell membranes. The SSIM value of the imaging results on mitochondrial cell membranes in small field of view reaches 0.8. It proves the superior performance of the algorithm in image reconstruction in the microscopic light field in the microscopic light field. After that, the performance of the algorithm in mesoscopic light field image reconstruction is investigated in this thesis, which mainly includes the performance test of mesoscopic static samples and mesoscopic dynamic samples. The static sample is the light field image of mouse cerebral cortex structure, which was reconstructed by using the algorithm with the traditional light field image reconstruction algorithm, and the results show that the signal-to-noise ratio (SNR) value of the algorithm reaches 0.136 dB, and the cutoff frequency kc reaches 0.5741, while the traditional algorithm's SNR only reaches 0.062 dB, and the cutoff frequency kc only reaches 0.4946. The dynamic sample is the NAOMi simulation of the light field image of visual area 1 of mouse cerebral cortex over time, the image was reconstructed using the algorithm, and the results showed that the percentage change of fluorescence signal obtained from the reconstruction results using the algorithm was almost the same as the true value signal. The above study proved the superior performance of the algorithm in mesopic light field image reconstruction. Finally, the performance of the algorithm in macroscopic light field PIV image reconstruction is investigated in this thesis. Compared with the traditional light field PIV reconstruction algorithm, the algorithm can reduce the particle reconstruction error in the x-y direction by 4 times, and reduce the particle reconstruction error in the z direction by 4-7 times. And then the algorithm was used to reconstruct the 3D particle field of the light-field PIV particle image of the lid-driven flow, and the reconstruction results were combined with the optical flow method for the 3D flow field reconstruction, and the results show that the particle image obtained by the algorithm can reconstruct the 3D vortex structure better, and the velocity vector error in the z-direction is only 0.1934 ± 0.001 mm/s, whereas the traditional refocusing algorithm can't restore the The traditional refocusing algorithm cannot restore the 3D vortex structure, and the obtained velocity vector error reaches 0.3603 ± 0.0041 mm/s which is 86.3% higher than that of the algorithm. The above study demonstrates the superior performance of this algorithm in macroscopic light field PIV reconstruction. |
参考文献: |
[1] GOLDSTEIN R. Fluid mechanics measurements[J]. 2017. [2] 赵洲, 丁俊飞, 实验流体力学 施 J. 基于单相机光场 PIV 的逆压湍流边界层测量[J]. 2019, 33(2): 66-71. [7] 丁俊飞, 许晟明, 实验流体力学 施 J. 光场单相机三维流场测试技术[J]. 2016, 30(6): 50-8. [9] 吴治安, 朱效宇, 李健, et al. 基于体标定追迹法的光场 PIV 权系数计算方法[J]. 2021, 41(20): 2010001. [14] 化希耀. 光场图像超分辨率重建方法研究[D]. City, 2023. [15] 王炜豪. 微透镜阵列光场成像技术的空间采样率及散射场景成像问题研究[D]. City, 2022. [18] ZHANG C, CHEN T. The Light Field[J]. 2006. [20] MCMILLAN L. Plenoptic Modeling: An Image-Based Rendering System[J]. Proc SIGGRAPH'96, 1995. [21] LEVOY M S, HANRAHAN P M. Light field rendering[J]. 1996. [22] ZHOU S, ZHU T, SHI K, et al. Review of light field technologies[J]. 2021, 4(1): 29. [23] 方璐, SINICA 戴 J A O. 计算光场成像[J]. 2020, 40(1): 0111001. [24] 聂云峰, 相里斌, 中国科学院大学学报 周 J. 光场成像技术进展[J]. 2011, 28(5): 563. [29] CHAN S-C, NG K-T, GAN Z-F, et al. The plenoptic video[J]. 2005, 15(12): 1650-9. [32] LIN X, WU J, ZHENG G, et al. Camera array based light field microscopy[J]. 2015, 6(9): 3179-89. [40] LIPPMANN G J C-R. La photographie integrale[J]. 1908, 146: 446-51. [52] 孙超, 吕俊伟, 李健伟, et al. 基于去卷积的快速图像超分辨率方法[J]. 2017, 37(12): 1210004. [86] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[J]. 2014, 27. [91] MIRZA M, OSINDERO S J A P A. Conditional generative adversarial nets[J]. 2014. [97] HORN B K, SCHUNCK B G J A I. Determining optical flow[J]. 1981, 17(1-3): 185-203. |
中图分类号: | V211.76 |
馆藏号: | 2025-002-0102 |
开放日期: | 2025-09-27 |