方向介绍:
神经网络高效计算指在深度学习任务中,通过优化计算过程和资源利用来提高计算效率和性能。由于深度学习模型通常具有大量的参数和复杂的计算需求,高效计算对于加速训练和推理过程、减少计算资源和能源消耗非常重要。研究内容包括:模型压缩与加速、
NPU加速器设计、NPU加速器设计、低功耗嵌入式智能系统等细分研究方向。
(1) 模型压缩与加速:研究神经网络稀疏、量化、低秩分解、知识蒸馏等技术,对预训练模型进行压缩,降低模型参数存储,提升模型训练/推理速度。
(2) NPU加速器设计:研究基于FPGA/ASIC的神经网络高效计算架构,通过算法-架构软硬协同设计,提升神经网络运行计算效率,降低功耗。
(3) 图像分析与计算:研究处理、解释和分析数字图像数据的算法,通过提高图像数据处理的效率和准确性,以便更好地理解和利用这些数据,在医学成像、卫星遥感、安全监控、自动驾驶等多个行业实际场景中应用广泛,是更智能决策的基础。
(4) 低功耗嵌入式系统:针对嵌入式CPU、GPU、DSP等异构嵌入式计算平台,开发高效的神经网络计算加速库,通过轻量化算法和高效加速库适配,实现神经网络在嵌入式平台上高效率、低功耗运行。
代表性论文:
-
Peisong Wang, Weihan Chen, Xiangyu He, Qiang Chen, Qingshan Liu, Jian Cheng. Optimization-based Post-training Quantization with Bit-split and Stitching. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol.45, No.2, pp.2119-2135, 2023.
-
Weihan Chen, Peisong Wang, Jian Cheng. Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization. ICCV 2021.
-
Tianli Zhao, Jiayuan Chen, Cong Leng, Jian Cheng. TinyNeRF: Towards 100x Compression of Voxel Radiance Fields. AAAI 2023.
-
Peisong Wang, Xiangyu He, Gang Li, Tianli Zhao, Jian Cheng. Sparsity-Inducing Binarized Neural Networks. AAAI 2020.
-
Weixiang Xu, Xiangyu He, Ke Cheng, Peisong Wang, Jian Cheng. Towards Fully Sparse Training: Information Restoration with Spatial Similarity. AAAI 2022.
-
Zejian Liu, Gang Li, Jian Cheng. Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing. DATE 2021. (Best paper candidate)
-
Zejian Liu, Gang Li, Jian Cheng. Efficient Accelerator/Network Co-Search with Circular Greedy Reinforcement Learning. IEEE Transactions on Circuits and Systems-II: Express Briefs, Vol.70, No.7, pp.2615-2619, 2023.
-
Zejian Liu, Gang Li, Jian Cheng. TBERT: Dynamic BERT Inference with Top-k Based Predictors. DATE 2023.
-
Jiaxing Wang, Haoli Bai, Jiaxiang Wu, Xupeng Shi, Junzhou Huang, Irwin King, Michael Lyu, Jian Cheng. Revisiting Parameter Sharing for Automatic Neural Channel Number Search. NeurIPS 2020.
-
Jiaxing Wang, Haoli Bai, Jiaxiang Wu, Jian Cheng. Bayesian Automatic Model Compression. IEEE Journal of Selected Topics in Signal Processing, Vol.14, No.4, pp.727-736, 2020.
-
Tianli Zhao, Qinghao Hu, Xiangyu He, Weixiang Xu, Jiaxing Wang, Cong Leng, Jian Cheng. ECBC: Efficient Convolution via Blocked Columnizing. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), Vol.34, No.1, pp.433-445, 2023.