In recent years, Deep Neural Networks (DNNs) have made remarkable breakthroughs across various fields due to their powerful capabilities. However, training a high-quality DNN remains a highly challenging task. While visualization methods have provided strong support for DNN training, most current mainstream visualization techniques adopt post-hoc analysis strategies. These methods expose several issues in practical applications, such as difficulties in storage due to the large volume of data, excessive I/O overhead, and the inability to enable real-time intervention. The DNN training process generates vast amounts of time-series data, yet existing tools struggle to extract detailed information about the training process, which hinders model optimization.

To address the aforementioned issues, our team from the Advanced Interactive Technology and Application Development Department proposed an in-situ visual analysis framework for deep learning model training data, and developed two key algorithms: in-situ feature extraction and neuron learning pattern abstraction. The former reuses memory data during model execution to analyze dynamic data in real time, solving the data storage and I/O bottleneck problems associated with traditional post-hoc analysis. The latter abstracts three learning patterns of neurons based on in-situ feature data, aiding in visual analysis. The framework performs exceptionally well, achieving a compression rate of up to 1% for the time-series training data of deep neural network models with millions of parameters. It supports visual analysis and traceability throughout the entire training process and enables batch-level neuron information visualization, providing strong support for the optimization of deep learning models. This achievement has been accepted by the journal IEEE Transactions on Visualization and Computer Graphics (IEEE TVCG, a CCF-A journal) in the field of visualization. The first author of the paper is Associate Professor Guan Li from our center, and the corresponding author is Professor Guihua Shan from our center.

This work was supported in part by the Strategic Priority Research Program of Chinese Academy of Sciences under Grant XDB0500103, and in part by the National Natural Science Foundation of China under Grant.

The in-situ analysis workflow of deep learning 

The in-situ visual analytics system for deep learning models

Related work

Li Guan, Junpeng Wang, Yang Wang, Guihua Shan*, Ying Zhao. An In-Situ Visual Analytics Framework for Deep Neural Networks, IEEE Transactions on Visualization and Computer Graphics, 6770-6786, Oct. 2024.