留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

面向边缘智能光学感知的航空紧固件旋转检测

符长虹 陈锟辉 鲁昆瀚 郑光泽 赵吉林

符长虹, 陈锟辉, 鲁昆瀚, 郑光泽, 赵吉林. 面向边缘智能光学感知的航空紧固件旋转检测[J]. 应用光学, 2022, 43(3): 472-480. doi: 10.5768/JAO202243.0303002
引用本文: 符长虹, 陈锟辉, 鲁昆瀚, 郑光泽, 赵吉林. 面向边缘智能光学感知的航空紧固件旋转检测[J]. 应用光学, 2022, 43(3): 472-480. doi: 10.5768/JAO202243.0303002
FU Changhong, CHEN Kunhui, LU Kunhan, ZHENG Guangze, ZHAO Jilin. Aviation fastener rotation detection for intelligent optical perception with edge computing[J]. Journal of Applied Optics, 2022, 43(3): 472-480. doi: 10.5768/JAO202243.0303002
Citation: FU Changhong, CHEN Kunhui, LU Kunhan, ZHENG Guangze, ZHAO Jilin. Aviation fastener rotation detection for intelligent optical perception with edge computing[J]. Journal of Applied Optics, 2022, 43(3): 472-480. doi: 10.5768/JAO202243.0303002

面向边缘智能光学感知的航空紧固件旋转检测

doi: 10.5768/JAO202243.0303002
基金项目: 上海市“科技创新行动计划”自然科学基金项目(20ZR1460100)
详细信息
    作者简介:

    符长虹(1986—),男,副教授,工学博士,主要从事智能视觉感知、边缘计算AI视觉系统研制、无人系统自主导航方面的研究。E-mail:changhongfu@tongji.edu.cn

  • 中图分类号: TN925

Aviation fastener rotation detection for intelligent optical perception with edge computing

  • 摘要: 针对航空紧固件分拣过程中现有方法存在效率低、成本高、精度差等问题,提出一种面向边缘智能光学感知的旋转目标检测方法。构建一种基于强化语义和优化空间的特征融合机制,有效提升目标检测模型的性能;设计一种空洞幻影模块,减少特征融合网络的参数量,有利于模型在工业场景下的边缘部署;采用高斯类环形平滑标签方法,在模型检测层预测分支上实现旋转目标检测,显著提升模型检测性能,并更有助于工业机器人自动抓取。在权威公开旋转数据集上,检测准确率达到77.16%。最后,在嵌入式智能设备上进行边缘部署并测试,整体准确率达到99.76%,推理速度超过20 FPS (frames per second),满足工业应用的要求。
  • 图  1  轻量级航空紧固件旋转检测方法

    Fig.  1  Lightweight aviation fastener rotation detection method

    图  2  强化语义与优化空间特征融合机制

    Fig.  2  Enhanced semantics and optimized space feature fusion mechanism

    图  3  空洞幻影模块

    Fig.  3  Dilated ghost module

    图  4  瓶颈层

    Fig.  4  Bottleneck layer

    图  5  旋转框角度定义[19]

    Fig.  5  Angle definition of rotation bounding box[19]

    图  6  典型航空紧固件

    Fig.  6  Typical aviation fasteners

    图  7  OLAFDet检测结果示例

    Fig.  7  Result examples of OLAFDet detection

    表  1  与前沿旋转框检测方法的对比

    Table  1  Comparison with cutting-edge rotation bounding box detection methods

    方法来源参数量/
    MB
    PL/%BD/%BR/%GTF/%SV/%LV/%SH/%TC/%BC/%ST/%SBF/%RA/%HA/%SP/%HC/%mAP/%
    FR-O[20] CVPR2018 242 79.42 77.13 17.70 64.05 35.30 38.02 37.16 89.41 69.64 59.28 50.30 52.91 47.89 47.40 46.30 54.13
    TOSO[21] ICASSP2020 212 80.17 65.59 39.82 39.95 49.71 65.01 53.58 81.45 44.66 78.51 48.85 56.73 64.40 64.24 36.75 57.92
    PIoU
    Loss[22]
    ECCV2020 80.90 69.70 24.10 60.20 38.30 64.40 64.80 90.90 77.20 70.40 46.50 37.10 57.10 61.90 64.00 60.50
    Axis
    Learning[23]
    RS2020 79.53 77.15 38.59 61.15 67.53 70.49 76.30 89.66 79.07 83.53 47.27 61.01 56.28 66.06 36.05 65.98
    MARNet[24] IJRS2021 88.91 77.91 39.88 71.17 62.79 58.96 66.25 90.87 73.73 79.04 57.57 64.33 62.47 61.64 51.80 67.15
    GSDet[25] TIP2021 81.12 76.78 40.78 75.89 64.50 58.37 74.21 89.92 79.40 78.83 64.54 63.67 66.04 58.01 52.13 68.28
    RADet[26] RS2020 79.45 76.99 48.05 65.83 65.46 74.40 68.86 89.70 78.14 74.97 49.92 64.63 66.14 71.58 62.16 69.09
    RoI
    Transformer[27]
    CVPR2019 273 88.64 78.52 43.44 75.92 68.81 73.68 83.59 90.74 77.27 81.46 58.39 53.54 62.83 58.93 47.67 69.56
    BBAVectors[28] WACV2021 276 88.35 79.96 50.69 62.18 78.43 78.98 87.94 90.85 83.58 84.35 54.13 60.24 65.22 64.28 55.70 72.32
    SCRDet[29] ICCV2019 427 89.98 80.65 52.09 68.36 68.36 60.32 72.41 90.85 87.94 86.86 65.02 66.68 66.25 68.24 65.21 72.61
    GLS-Net[30] RS2020 88.65 77.40 51.20 71.03 73.30 72.16 84.68 90.87 80.43 85.38 58.33 62.27 67.58 70.69 60.42 72.96
    R3Det[6] AAAI2021 787 88.76 83.09 50.91 67.27 76.23 80.39 86.72 90.78 84.68 83.24 61.98 61.35 66.91 70.63 53.94 73.79
    FEDet[31] ICCSE2021 89.09 79.87 51.13 70.20 78.42 80.54 87.84 90.86 83.91 85.31 58.33 66.90 67.74 69.74 63.43 74.89
    TricubeNet[32] WACV2022 88.75 82.12 49.24 72.98 77.64 74.53 84.65 90.81 86.02 85.38 58.69 63.59 73.82 69.67 71.08 75.26
    Beyond
    Bounding-Box[33]
    CVPR2021 89.08 83.20 54.37 66.87 81.23 80.96 87.17 90.21 84.32 86.09 52.34 69.94 75.52 80.76 67.96 76.67
    OLAFDet 18 89.28 85.09 48.75 64.65 80.81 84.70 88.09 90.70 86.72 87.41 60.14 67.12 75.02 81.55 67.39 77.16
    下载: 导出CSV

    表  2  消融分析实验结果

    Table  2  Experimental results of ablation analyses

    基准模型OLAFDet
    检测框类型 水平框 旋转框 旋转框 旋转框
    高斯类环形平滑标签
    强化语义和优化空间
    空洞幻影模块
    mAP/% 42.32 76.86 77.17 77.16
    参数量/MB 14.8 15.8 21.3 18.0
    下载: 导出CSV

    表  3  航空紧固件检测的测试结果

    Table  3  Test results of aviation fastener detection %

    类别螺栓垫片螺帽圆柱销铆钉螺钉
    APs100.00100.00100.00100.00100.00100.00
    APm100.00100.0098.9399.9999.12100.00
    AP100.00100.0099.1799.9999.43100.00
    mAP99.76
    下载: 导出CSV
  • [1] 中华人民共和国国民经济和社会发展第十四个五年规划和2035年远景目标纲要[N]. 人民日报, 2021-03-13.

    Outline of the 14th Five-Year Plan (2021-2025) for national economic and social development and vision 2035 of the People's Republic of China [N]. People's Daily, 2021-03-13. DOI: 10.28655/n.cnki.nrmrb.2021.002455.
    [2] 宋瑨, 王世峰. 基于可变形部件模型HOG特征的人形目标检测[J]. 应用光学,2016,37(3):380-384. SONG Jin, WANG Shifeng. Human kind shape object detection using deformable parts model with HOG features[J]. Journal of Applied Optics,2016,37(3):380-384. doi: 10.5768/JAO201637.0302003
    [3] 张珹. 高铁接触网支持装置紧固件识别与定位的深度学习方法[J]. 工程数学学报,2020,37(3):261-268. ZHANG Cheng. Deep learning methods for fastener identification and location of high speed railway catenary support devices[J]. Chinese Journal of Engineering Mathematics,2020,37(3):261-268. doi: 10.3969/j.issn.1005-3085.2020.03.001
    [4] 王一, 马钲东, 董光林. 基于改进Faster RCNN的零件识别方法研究[J]. 应用光学,2022,43(1):67-73. WANG Yi, MA Zhengdong, DONG Guanglin. Parts recognition method based on improved Faster RCNN[J]. Journal of Applied Optics,2022,43(1):67-73. doi: 10.5768/JAO202243.0102003
    [5] ZHANG Zhenli, ZHANG Xiangyu, PENG Chao, et al. Exfuse: enhancing feature fusion for semantic segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV),Munich,Germany:Springer, 2018: 269-284.
    [6] YANG Xue, YAN Junchi, FENG Ziming, et al. R3Det: refined single-stage detector with feature refinement for rotating object[C]//Proceedings of the AAAI Conference on Artificial Intelligence,Palo Alto,California USA:AAAI Press,2021, 35(4): 3163-3171.
    [7] LIN Tsungyi, DOLLAR Piotr, GIRSHICK Ross, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA: IEEE, 2017: 2117-2125.
    [8] LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),Salt Lake City,UT,USA:IEEE, 2018: 8759-8768.
    [9] 柴斌. 基于深度学习的工件检测和定位系统的研究与实现[D]. 沈阳: 中国科学院大学中国科学院沈阳计算技术研究所, 2021.

    CHAI Bin. Design and research of workpiece defect detection system based on deep learning[D]. Shenyang: Shenyang Insitute of Computing Technology, Chinese Academy of Science. 2021.
    [10] MEHTA Sachin, RASTEGARI Mohammad, CASPI Anat, et al. ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV),Munich, Germany:Springer, 2018: 552-568.
    [11] 刘宽, 郎磊. 轻量化SSD目标检测方法研究[J]. 湖北民族大学学报(自然科学版), 2021, 39(4): 418-424.

    LIU Kuan, LANG Lei. Research on lightweight SSD target detection method[J]. Journal Of HuBei Minzu University(Natural Science Edition) , 2021, 39(4): 418-424.
    [12] TAN Mingxing, LE Quoc. Efficientnet: rethinking model scaling for convolutional neural networks[C]//Proceedings of Machine Learning Research(PMLR),Long Beach,California,USA:PMLR, 2019: 6105-6114.
    [13] 曹富强, 王明泉, 张俊生, 等. 基于深度学习的铸件X射线图像分割研究[J]. 应用光学,2021,42(6):1025-1033. CAO Fuqiang, WANG Mingquan, ZHANG Junsheng, et al. Casting X-ray image segmentation based on deep learning[J]. Journal of Applied Optics,2021,42(6):1025-1033. doi: 10.5768/JAO202142.0602003
    [14] 刘怀广, 丁晚成, 黄千稳. 基于轻量化卷积神经网络的光伏电池片缺陷检测方法研究[J]. 应用光学,2022,43(1):87-94. LIU Huaiguang, DING Wancheng, HUANG Qianwen. Defects detection method of photovoltaic cells based on lightweightconvolutional neural network[J]. Journal of Applied Optics,2022,43(1):87-94. doi: 10.5768/JAO202243.0103003
    [15] 卢艳东, 李积英, 王筱婷. 一种基于改进YOLOv3-tiny的轻量级轨道紧固件检测算法[J]. 铁道标准设计, 2021, 14(2): 1-7.

    LU Yandong, LI Jiying, WANG Xiaoting. A lightweight track fastener detection algorithm based on improved YOLOv3-tiny[J]. Railway Standard Design, 2021, 14(2): 1-7.
    [16] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA: IEEE, 2016: 770-778.
    [17] HAN Kai, WANG Yunhe, TIAN Qi, et al. Ghostnet: more features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Seattle,WA,USA:IEEE, 2020: 1580-1589.
    [18] WANG Chienyao, LIAO H M, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of cnn [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Seattle,WA, USA: IEEE, 2020: 390-391.
    [19] YANG Xue, YAN Junchi. Arbitrary-oriented object detection with circular smooth label[C]//Proceedings of the European Conference on Computer Vision (ECCV),Glasgow,United Kingdom:Springer,2020: 677-694.
    [20] XIA Guisong, BAI Xiang, DING Jian, et al. DOTA: a large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Salt Lake City,UT,USA:IEEE, 2018: 3974-3983.
    [21] FENG Pengming, LIN Youtian, GUAN Jian, et al. TOSO: student’s-T distribution aided one-stage orientation target detection in remote sensing images[C]// IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),Virtual,Barcelona:IEEE, 2020: 4057-4061.
    [22] CHEN Zhiming, CHEN Kean, LIN Weiyao, et al. PIoU loss: towards accurate oriented object detection in complex environments[C]//Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, United Kingdom: Springer, 2020: 195-211.
    [23] XIAO Zhifeng, QIAN Linjun, SHAO Weiping, et al. Axis learning for orientated objects detection in aerial images[J]. Remote Sensing, 2020, 12(6): 908-928.
    [24] CAO Lianyu, ZHANG Xiaolu, WANG Zhaoshun, et al. Multi angle rotation object detection for remote sensing image based on modified feature pyramid networks [J]. International Journal of Remote Sensing,2021, 42(14): 5253-5276.
    [25] LI Wei, WEI Wei, ZHANG Lei. GSDet: object detection in aerial images based on scale reasoning[J]. IEEE Transactions on Image Processing, 2021, 30: 4599-4609.
    [26] LI Yangyang, HUANG Qin, PEI Xuan, et al. RADet: refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images [J]. International Journal of Remote Sensing,2020, 12(3): 389-408.
    [27] DING Jian, XUE Nan, LONG Yang, et al. Learning roI transformer for oriented object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Long Beach,CA,USA:IEEE, 2019: 2844-2853.
    [28] YI Jingru, WU Pengxiang, LIU Bo, et al. Oriented object detection in aerial images with box boundary-aware vectors[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA,: IEEE, 2021: 2149-2158.
    [29] YANG Xue, YANG Jirui, YAN Junchi, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV), Seoul, Korea(South): IEEE, 2019: 8231-8240.
    [30] LI Chengyuan, LUO Bin, HONG Hailong, et al. Object detection based on global-local saliency constraint in aerial images[J]. International Journal of Remote Sensing, 2020, 12(9): 1435-1456.
    [31] WANG Mengyuan, ZHANG Xuanyu, YU Chuanbo, et al. Oriented object detection with fine-grained enhancement and angle constraint[C]//Proceedings of the International Conference on Computer Science & Education (ICCSE),Lancaster University,UK:IEEE,2021: 752-757.
    [32] KIM Beomyoung, LEE Janghyeon, LEE Sihaeng, et al. TricubeNet: 2D kernel-based object representation for weakly-occluded oriented object detection[C]//Proceedings of the IEEE Winter Conference onApplications of Computer Vision(WACV), Waikoloa, HI, USA: IEEE, 2022: 167-176.
    [33] GUO Zonghao, LIU Chang, ZHANG Xiaosong, et al. Beyond bounding-box: convex-hull feature adaptation for oriented and densely packed object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Nashville, TN, USA: IEEE, 2021: 8792-8801.
  • 加载中
图(7) / 表(3)
计量
  • 文章访问数:  475
  • HTML全文浏览量:  355
  • PDF下载量:  43
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-03-07
  • 修回日期:  2022-04-22
  • 网络出版日期:  2022-04-24
  • 刊出日期:  2022-05-12

目录

    /

    返回文章
    返回