A Bridge Vibration Measurement Method by UAVs based on CNNs ‎and Bayesian Optimization‎

Document Type : Research Paper


1 School of Civil and Transportation Engineering, Guangdong University of Technology, Guangzhou, 510006, China

2 Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore 138632, Republic of Singapore

3 UTBM, IRAMAT UMR 7065-CNRS, Rue de Leupe, 90010 Belfort Cedex, France

4 Centre Borelli, ENS - Université Paris-Saclay, 4 avenue des Sciences, 91190 Gif-sur-Yvette, France


A bridge vibration measurement method by Unmanned Aerial Vehicles (UAVs) based on a Convolutional Neural Network (CNN) and Bayesian Optimization (BO) is proposed. In the proposed method, the video of the bridge structure is collected by a UAV, then the reference points in the background of the bridge and the target points on the bridge in the video are tracked by the Kanade-Lucas-Tomasi (KLT) optical flow method, so that their coordinates can be obtained. The BO is used to find the optimal hyper-parameter combination of a CNN, and the CNN based on BO is used to correct the bridge displacement signal collected by the UAV. Finally, the natural frequency of the bridge is extracted by processing the corrected displacement signals with Operational Modal Analysis (OMA). Moreover, a steel truss is used as the experimental model. The number of reference points and the shooting time of the UAV with the optimal correction effect of the BO-based CNN are obtained by two groups of comparative experiments, and the influence of the distance between structure and reference points on the correction effect of the BO-based CNN is determined by another group of comparative experiment. The static reference points are not required for the proposed method, which evidently enhances the applicability of UAVs; the conclusion of this paper has great guiding significance for the actual bridge vibration measurement.


Main Subjects

Publisher’s Note Shahid Chamran University of Ahvaz remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 

[1]‎ Doebling, S.W., Farrar, C.R., Prime, M.B., Digest, V., A Summary Review of Vibration-Based Damage Identification Methods, ‎The Shock and Vibration Digest, 30, 1998, 91-105.‎
‎[2]‎ Kovačič, B., Kamnik, R., Štrukelj, A., Vatin, N., Processing of Signals Produced by Strain Gauges in Testing Measurements ‎of ‎the ‎Bridges, Procedia Engineering, 117, 2015, 795-801.‎
‎[3] Soyoz, S., Feng, M.Q., Long-Term Monitoring and Identification of Bridge Structural Parameters, Computer‐aided Civil & Infrastructure Engineering, 24, 2010, 82–92.‎
‎[4]‎ Siringoringo, D.M., Fujino, Y., Noncontact Operational Modal Analysis of Structural Members by Laser Doppler Vibrometer, ‎Computer-aided Civil and Infrastructure Engineering, 24, 2010, 249-265.‎
‎[5] Psimoulis, P., Pytharouli, S., Karambalis, D., Stiros, S., Potential of Global Positioning System (GPS) to measure ‎frequencies ‎of ‎oscillations of engineering structures, Journal of Sound & Vibration, 318, 2008, 606-623.‎
‎[6]‎ Shi, J., Tomasi, C., Good Features to Track, In Proceedings of Proceedings / CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 6000, 2002, 593-600.‎
‎[7]‎ Lucas, B.D., An Iterative Image Registration Technique with an Application to Stereo Vision (DARPA), Proceedings of the ‎‎7th International Joint Conference on Artificial Intelligence, 81, 1981, 674-679.‎
‎[8]‎ Liu, F., Yundong, W.U., Cai, G., Chen, S., Science, S.O., Research of Feature Point Tracking Algorithm for ‎UAV ‎Video ‎Image Based on KLT, Journal of Jimei University, 22, 2017, 73-80.‎
‎[9]‎ Zhu, J., Lu, Z., Zhang, C., A marker-free method for structural dynamic displacement measurement based on optical ‎flow, ‎Structure and Infrastructure Engineering, 18, 2020, 1-13.‎
‎[10]‎ Ellenberg, A., Kontsos, A., Moon, F., Bartoli, I., Bridge related damage quantification using unmanned aerial ‎vehicle ‎imagery, ‎Structural Control & Health Monitoring, 23, 2016, 1168-1179.‎
‎[11]‎ Hoskere, V., Park, J.W., Yoon, H., Spencer, B.F., Vision-Based Modal Survey of Civil Infrastructure Using Unmanned ‎Aerial ‎Vehicles, Journal of Structural Engineering, 145, 2019, 04019062.‎
‎[12]‎ Rodríguez-Canosa, G., Stephen, T., Jaime, D.C., Antonio, B., Bruce, M., A Real-Time Method to Detect and Track ‎Moving ‎Objects ‎‎(DATMO) from Unmanned Aerial Vehicles (UAVs) Using a Single Camera, Remote Sensing, 4, 2012, 1090-1111.‎
‎[13]‎ Zhai, Y., Shah, M., Visual attention detection in video sequences using spatiotemporal cues, In Proceedings of Proceedings of the ‎‎14th ACM International Conference on Multimedia, Santa Barbara, CA, USA, October 23-27, 2006.‎
‎[14] Zhu, X., Chen, Z., Tang, C., Mi, Q., Yan, X., Application of two oriented partial differential equation filtering models ‎on ‎speckle ‎fringes with poor quality and their numerically fast algorithms, Applied Optics, 52, 2013, 1814-1823.‎
‎[15]‎ Zhang, J., Wu, Z., Chen, G., Liang, Q., Comparisons of Differential Filtering and Homography Transformation in ‎Modal ‎Parameter ‎Identification from UAV Measurement, Sensors, 21, 2021, 5664.‎
‎[16]‎ Wang, G., Chen, X., Qiao, F., Wu, Z., Huang, N.E., On Intrinsic Mode Function, Advances in Adaptive Data Analysis, 2, 2010, 277-293.‎
‎[17]‎ Bochner, S., Fourier transforms, Annals of Mathematics Studies, 7, 1949, 145–151.‎
‎[18]‎ Lazaro, J.A., Wessel, R., Koppenborg, J., Dudziak, G., Blewett, I.J., Inverse Fourier transform method for characterizing arrayed-‎‎‎waveguide gratings, IEEE Photonics Technology Letters, 15, 2003, 93-95.‎
‎[19]‎ Yan, Z., Teng, S., Luo, W., Bassir, D., Chen, G., Bridge Modal Parameter Identification from UAV Measurement Based ‎on ‎Empirical ‎Mode Decomposition and Fourier Transform, Applied Sciences, 12, 2022, 8689.‎
‎[20]‎ Wu, Z., Chen, G., Ding, Q., Yuan, B., Yang, X., Three-Dimensional Reconstruction-Based Vibration Measurement of ‎Bridge ‎Model ‎Using UAVs, Applied Sciences, 11, 2021, 5111.‎
‎[21]‎ Lirong, D., Riuan, L., Tingting, C., Geometric Correction Algorithm for UAV Remote Sensing Image Based on Neural ‎Network, ‎In Proceedings of 2016 International Conference on Artificial Intelligence and Engineering Applications (AIEA 2016), 63, 2016, 59-‎‎63.‎
‎[22]‎ Ruian, L., Nan, L., Beibei, Z., Tingting, C., Ninghao, Y., Geometry correction Algorithm for UAV Remote Sensing Image ‎Based ‎on ‎Improved Neural Network, IOP Conference, 322, 2018, 072002-.‎
‎[23]‎ Weng, Y., Shan, J., Lu, Z., Lu, X., Spencer, B.F., Homography‐based structural displacement measurement for large ‎structures ‎using ‎unmanned aerial vehicles, Computer‐Aided Civil and Infrastructure Engineering, 36, 2021, 1114–1128.‎
‎[24]‎ Hao, X., Zhang, G., Ma, S., Deep Learning, International Journal of Semantic Computing, 10, 2016, 417-439.‎
‎[25]‎ Hubel, D.H., Wiesel, T.N., Receptive fields, binocular interaction and functional architecture in the cat's visual cortex, Journal of Physiology, 160, 1962, 106-154.‎
‎[26]‎ Luo, X., Tian, X., Zhang, H., Hou, W., Zhang, J., Fast Automatic Vehicle Detection in UAV Images Using ‎Convolutional ‎Neural ‎Networks, Remote Sensing, 12, 2020, 1994.‎
‎[27]‎ Aliyari, M., Droguett, E.L., Ayele, Y.Z., UAV-Based Bridge Inspection via Transfer Learning, Sustainability, 13, 2021, 11359.‎
‎v28.‎ Yan, Z., Jin, Z., Teng, S., Chen, G., Bassir, D., Measurement of Bridge Vibration by UAVs Combined with CNN and KLT Optical-‎‎Flow ‎Method, Applied Sciences, 12, 2022, 5181.‎
‎[29]‎ Seunghye, L., Jingwan, H., Mehriniso, Z., Hyeonjoon, M., Jaehong, L.J., Background ‎Information ‎of ‎Deep Learning for Structural Engineering, Archives of Computational Methods in Engineering, 25, 2017, 1-9.‎
‎[30]‎ Boughorbel, S., Tarel, J.P., Boujemaa, N., The LCCP for Optimizing Kernel Parameters for SVM, In Proceedings of International Conference on Artificial Neural Networks, 3697, 2005, 589-594.‎
‎[31]‎ Sun, Y., Bing, X., Zhang, M., Yen, G.G., An Experimental Study on Hyper-parameter Optimization for Stacked Auto-‎Encoders, ‎In Proceedings of 2018 IEEE Congress on Evolutionary Computation (CEC), 2018, 1-8.‎
‎[32]‎ Bergstra, J., Bengio, Y., Random Search for Hyper-Parameter Optimization, Journal of Machine Learning Research, 13, 2012, 281-305.‎
‎[33]‎ Snoek, J., Larochelle, H., Adams, R.P., Practical Bayesian Optimization of Machine Learning Algorithms, Advances in Neural ‎Information Processing Systems, 2, 2012, 2951–2959.‎
‎[34]‎ Putatunda, S., Rama, K., A Comparative Analysis of Hyperopt as Against Other Approaches for Hyper-Parameter ‎Optimization ‎of ‎XGBoost, In Proceedings of the 2018 International Conference on Signal Processing and Machine Learning, 2018, 6-10.‎
‎[35]‎ Hahn, L., Roese-Koerner, L., Friedrichs, K., Kummert, A., Fast and Reliable Architecture Selection for Convolutional ‎Neural ‎Networks, ‎In Proceedings of The 27th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning ‎‎(ESANN), 2019, 179-184.‎
‎[36]‎ Ekici, B.B., Detecting damaged buildings from satellite imagery, Journal of Applied Remote Sensing, 15, 2021, 032004.‎
‎[37]‎ Brownjohn, J., Magalhaes, F., Caetano, E., Cunha, A., Ambient vibration re-testing and operational modal analysis of ‎the ‎Humber ‎Bridge, Engineering Structures, 32, 2010, 2003-2018.‎
‎[38]‎ Yan, W.J., Ren, W.X., An Enhanced Power Spectral Density Transmissibility (EPSDT) approach for operational ‎modal ‎analysis: ‎Theoretical and experimental investigation, Engineering Structures, 102, 2015, 108-119.‎
‎[39]‎ Lecun, Y., Bottou, L., Gradient-based learning applied to document recognition, Proceedings of the IEEE, 86, 1998, 2278-2324.‎
‎[40]‎ Maas, A.L., Hannun, A.Y., Ng, A.Y., Rectifier Nonlinearities Improve Neural Network Acoustic Models, In Proceedings of International Conference on Machine Learning, 2013.‎
‎[41]‎ Ioffe, S., Szegedy, C., Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, ‎Journal of Machine Learning Research, 37, 2015, 448–456.‎
‎[42]‎ Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P., On Large-Batch Training for Deep Learning: Generalization ‎Gap ‎and ‎Sharp Minima, In Proceedings of the 5th International Conference on Learning Representations, 2016.‎
‎[43]‎ Margaliot, M., Hespanha, J.P., Root-mean-square gains of switched linear systems: A variational approach, Automatica, 44, ‎‎2008, ‎‎2398-2402.‎
‎[44]‎ Chen, G., Wu, Z., Gong, C., Zhang, J., Sun, X., DIC-Based Operational Modal Analysis of Bridges, Advances in Civil Engineering, ‎‎2021, ‎‎2021, 6694790.‎
‎[45]‎ Chen, G., Liang, Q., Zhong, W., Gao, X., Cui, F., Homography-based measurement of bridge vibration using UAV and ‎DIC ‎method, ‎Measurement, 170, 2020, 108683.‎