Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning

1.   Introduction

Laser powder bed fusion (LPBF) is a widely adopted metal additive manufacturing technology known for its ability to produce highly complex and precise parts [1, 2]. The LPBF technology offers design freedom and weight reduction opportunities, particularly in industries like aerospace, where reduced weight can lead to reduced energy consumption and pollution [3]. LPBF also offers advantages in productivity, on-site and on-demand production, and streamlined supply chains [4]. Over the years, LPBF has evolved from a rapid prototyping tool to a powerful technology for manufacturing end-use parts [5]. In a typical LPBF process, a laser beam selectively scans a thin layer of metal powder, melting and solidifying it to create a part based on computer designs. The rapid heating and cooling cycles involved in the printing process give rise to various complex phenomena [6, 7].

At the site of laser-matter interaction, a vapor depression in the melt pool, known as the keyhole, is formed due to the recoil pressure from the evaporation of the metal when the laser fluence is sufficient [8, 9]. Multiple reflections and absorptions of the laser rays take place inside the keyhole, significantly enhancing the overall absorption of the laser energy by the metal. This, in turn, leads to increased energy efficiency and improved build rates. The area near the keyhole (i.e. around the laser beam) is hot, while the region away from the keyhole is cooler. The Marangoni force, driven by the gradient in surface tension, transports the liquid from the hot region to the cooler region (assuming a positive dependence of surface tension on temperature because of, say, thermal desegregation of solute), creating complex liquid flow patterns inside the melt pool [6]. Due to the interplay among recoil pressure, Marangoni force and capillary force, an unstable keyhole frequently collapses, giving rise to gas bubbles [10, 11]. Some of the bubbles can result in keyhole pore defects if captured by the advancing solidification front. Monitoring such defects can be notably challenging in risk-averse applications of LPBF, such as those in aerospace and biomedical fields. The associated technological barriers necessitate the development of an accurate closed-loop control system for effective real-time defect detection and manufacturing quality management [3]. By accurately localizing the occurrence of pore defects, the interrogation region can be significantly narrowed down after the printing process, thereby saving time and effort for product qualification and certification.

Acoustic and optical emission sensing are commonly employed to capture process signatures indicative of various physical phenomena during the LPBF process [12]. Konoenko et al developed an operando crack detection approach for LPBF by sensing structural-borne acoustic emissions [13], allowing them to distinguish crack acoustic emission events from background noise. Simonds et al designed an integrating sphere system to measure time-dependent reflected laser [14], which was then converted into time-resolved absorptance data, providing insights into the laser-matter interaction process. In recent years, there has been growing interest in combining machine learning approaches into defect detection approaches [15-17]. Scime and Beuth, for instance, developed an algorithm for detecting and classifying defects during the powder spreading stage of LPBF process [18]. This algorithm employed an unsupervised machine learning approach and utilized optical images collected from commercial printing systems. Mondal et al proposed a physics-informed machine learning approach to produce crack-free parts, evaluating variables related to cracking physics derived from both mechanical models and experimental measurements [19].

However, most work existing in this field focused on distinguishing between keyhole-prone and non-keyhole-prone conditions, rather than achieving localized detection with high spatiotemporal resolution. For instance, Liu et al developed a physics-informed machine learning model for porosity prediction in LPBF [20]. They calculated physical effects, like energy density distribution, based on machine settings and correlated these effects with porosity measured from x-ray computed tomography scans of printed parts. Energy density was correlated with pore size ranges. Shevchik et al explored the feasibility of using acoustic emission for monitoring during the LPBF process [21]. They captured acoustic emission data using a fiber Bragg grating and employed a spectral convolutional neural network to predict print quality in terms of porosity concentration, based on acoustic emission features. Recently, approaches for localized keyhole pore detection were reported, but with generally millisecond-scale temporal resolutions. Tempelman et al proposed a keyhole pore detection system by using a microphone and machine learning [22]. The scanned printed samples with x-ray imaging to label the pore location in the time-series acoustic signals. Their study found that the high-frequency components of the signals were particularly important for pore detection, with window size ranging from 2.5 ms to 7.5 ms. Pandiyan et al tackled a similar problem using deep learning with signals from multiple sensors, incorporating variable time scales [23]. They used operando x-ray imaging to label time-series signals, enabling the study of smaller window sizes ranging from 0.5 ms to 4 ms. Working on a similar problem, Gorgannejad et al recently developed a data fusion machine learning model by labeling the thermal and acoustic time-series signals with synchrotron x-ray imaging information. They achieved a prediction accuracy on the pore generation events of 94% at a time resolution of 2 ms [24].

Nevertheless, a comprehensive understanding of the characteristic acoustic and optical signals associated with keyhole pore generation in LPBF has not yet been achieved. The industry has been increasingly interested in developing advanced in-situ approaches for defect detection in the printing process [25]. To address these challenges, a machine learning-aided process sensing approach for localized detection of keyhole pore generation in LPBF was developed in this study. By using synchrotron x-ray imaging as the ground truth labelling, the capability of using near-infrared (NIR) imaging, ultrasonic and audible frequency microphones, and a backscattered laser sensitive photodiode data to detect keyhole generation events were evaluated. Also, the machine-learning-aided prediction results with multi-sensor data using different combinations of the four sensors were analyzed. In addition, the inherent connection between the optical and acoustic emissions was established, as well as their correlation with the oscillation behaviors of the keyhole. This work unraveled the physics that support the accurate predictions enabled by machine learning models.

2.   Materials and methods

2.1   Materials

Commercial Ti-6Al-4V (Grade 5, McMaster-Carr, USA) plate samples were sectioned utilizing electrical discharge machining. Subsequently, the samples were mechanically polished with a 1200-grit sandpaper, resulting in a final dimension of 50 mm × 3.21 mm × 0.48 mm. In synchrotron experiments, the x-ray beam penetrated through the sample along the thinnest thickness.

2.2   Laser melting process

The scanning laser melting and process sensing experiment was carried out at the 32-ID-B beamline of the advanced photon source (APS) at Argonne National Laboratory (figure 1). The laser scanning setup utilized an ytterbium fiber laser (IPG YLR-500-AC, USA) in conjunction with a galvo laser scanner (intelliSCAN_de 30, SCANLAB GmbH, Germany) [9, 10, 26]. The Gaussian laser operated at a wavelength of 1070 nm and exhibited a spot size of ∼82 μm (1/e²) on the sample surface plane when defocused by 2 mm. Prior to each laser scan, the samples were positioned inside the build chamber and subsequently purged and refilled with high purity Argon gas to atmospheric pressure. A 4 mm single straight line was scanned on the plate sample using continuous-wave (CW) mode laser. The position of the line was chosen strategically to cover the field-of-view (FOV) of the x-ray detector in the center, which is 2 mm wide. This enabled the capture of a representative melting state devoid of interferential factors such as initial laser power and speed ramping, as well as the abrupt laser-off at the end of the track.

Figure  1.  Schematic and photo of simultaneous high-speed synchrotron x-ray imaging and process sensing experiment on LPBF of Ti-6Al-4V. (a) The laser beam had a Gaussian profile and scanned along a single straight line on a bare Ti-6Al-4V plate sample. The multi-sensor system was composed of a NIR camera, ultrasonic (100 kHz) and audible (20 kHz) microphones, and a photodiode. (b) Photo of the corresponding experiment setup in the APS 32-ID-B beamline. The inset photo is the setup inside the sample chamber.

DownLoad: Full-Size Img PowerPoint

Figure 2(a) summarizes all P-V conditions studied in this work. In the keyhole porosity region, many large pores were generated (figure 2(b)). Near the keyhole porosity region, a few small pores were generated (figure 2(c)). In the stable keyhole region, no pore was generated (figure 2(d)). Two scenarios were designed to investigate the efficacy of keyhole pore detection under various collections of laser power-scanning velocity (P-V) conditions. Scenario Large (labelled as ‘L’), denoted as orange dots in figure 2(a), spanned over a relatively large range of scanning velocity (i.e. 250-2000 mm s^-1). There were 133 experiments in this collection. Scenario Small (labelled as ‘S’), denoted as blue dots in figure 2(a), covers a relatively smaller range of scanning velocity (i.e. 300-800 mm s^-1) near the keyhole porosity boundary. There were 64 experiments in this collection.

Figure  2.  P-V conditions of scanning laser melting of Ti-6Al-4V plate samples studied in this work. (a) Each point represents a single straight line scan experiment. The dashed line marks the keyhole porosity boundary. The green zone covers the unstable keyhole or keyhole porosity conditions, while the red zone covers conditions without pore generation. (b)-(d) Representative x-ray image under different P-V conditions: (b) P = 330 W, V = 250 mm s^-1; (c) P = 350 W, V = 500 mm s^-1; (d) P = 370 W, V = 1000 mm s^-1.

DownLoad: Full-Size Img PowerPoint

2.3   High-speed synchrotron x-ray imaging

The APS 32-ID-B high-speed x-ray imaging beamline consists of two essential components: the x-ray source and the detector [27]. Polychromatic x-rays, with the first harmonic energy at 24.1 keV, were generated using a short-period undulator (18 mm) with a gap of 12 mm. The x-ray detector was an optical high-speed camera with accessory optics. A 100-μm thick LuAG:Ce scintillator and a Photron FastCam SA-Z camera (Photron USA, Inc) were used. Visible light emission from the scintillators were coupled to the high-speed detectors via a 45° reflection mirror, a 10× objective lens (NA = 0.28, Mitutoyo Corp., Japan), and a tube relay lens (Mitutoyo Corp., Japan). The x-ray imaging system was operated at a frame rate of 50 kHz, with an exposure time of 1 μs and a spatial resolution of 2 μm pixel^-1.

2.4   Process sensing system

The process sensing system consisted of two major components: NIR camera and non-spatially-resolved sensors. The NIR imaging system included several optical components: a wide-band reflection mirror (protected silver mirror, Thorlabs Inc., USA), a 1070 nm notch filter (Edmund Optics Inc., USA), a 760 nm long-pass filter (Newport Corp., USA), and a high-speed optical camera [26]. Specifically, the camera used for NIR imaging was the Photron FASTCAM NOVA S9 (Photron Inc., Japan), equipped with a Resolv4K zoom lens (Navitar, Inc., USA). The view-angle was set to ∼50°. The NIR camera operated at a spatial resolution of ∼9 μm pixel^-1, with an exposure time of 0.33 μs and a frame rate of 250 kHz.

The non-spatially-resolved sensing suite was integrated into a data acquisition NI cDAQ-9185 (DAQ) (National Instruments Corp., USA). This DAQ system was equipped with two microphones and one photodiode. The first microphone 378C01 (PCB Piezotronics Inc. USA), referred to as the ultrasonic microphone, had a frequency response up to 100 kHz. The second microphone 130F22 (PCB Piezotronics Inc. USA), referred to as the audible microphone, had a frequency response up to 20 kHz. Both microphones had an integral preamplifier. The distance between each microphone and the melt pool was approximately 20 mm. The reflected laser was collected by a high OH fiber (NA = 0.22, Thorlabs, Inc. USA) positioned close to the sample. The fiber went through a vacuum feedthrough to an externally mounted photodiode. The photodiode was an amplified switchable gain silicone detector PDA100A2 (Thorlabs, Inc. USA) with a 1070 nm bandpass filter (Thorlabs, Inc. USA). The photodiode was positioned at ∼45° angle with the sample surface and the gain was set to 0 in the experiment. The microphones and photodiode were sampled at a rate of 1 MHz for synchronization and data acquisition.

2.5   Data synchronization

The triggering signals for the x-ray camera, NIR camera, photodiode, and two microphones were synchronized in the experiments. The triggering mode of the two Photron cameras was set to ‘Random Reset’, i.e. first frame was collected instantaneously (only 1.25 μs delay) once the trigger signal is received. The collection of photodiode and microphones data were controlled by the DAQ which has a response time less than 1 μs. Therefore, the jittering time (uncertainty in data synchronization) in the experiment is about two orders shorter than the time scale discussed in this work.

The power of the IPG laser took maximum 20 μs to ramp up to the set power, and the laser scanner was set to skywriting mode, i.e. laser was powered ON when the scanner reached the set velocity [28]. In each experiment, the laser scanning path was set to be 4 mm, while the x-ray image window was 2 mm wide set to observe the middle 2 mm of the laser melting. Therefore, the laser power and scanning speed ramping occurred outside the x-ray FOV, which is irrelevant to the analysis.

2.6   Data processing and machine learning

Signal processing: signals collected with the non-spatially-resolved sensors (microphones and photodiode) were recorded as time-series data using the DAQ system. When correlating the acoustic emission data with optical emission, the time delay (∼0.06 ms) induced by the propagation of acoustic wave was considered. For the NIR images, the time-series signal was obtained from NIR images by calculating the average intensity of pixels within each image that were larger than the preset threshold, and this process was repeated for the entire image sequences. The preset threshold was optimized based on the prediction accuracy. If the threshold was too low, melt pool region away from the keyhole was captured, which diminished the characteristic keyhole oscillation. If the threshold was set to too high, insufficient information about keyhole was recorded with even frames showing no intensity. The analysis of the time-series signals was performed using MATLAB (MathWorks, USA). To analyze the frequency components of the time-series signals, the fast Fourier transform (FFT) (MATLAB function, fft) was applied [26]. The FFT converts the signal from the time domain into the frequency domain, providing information about the spectral content of the signal at different frequencies. In addition to the FFT operation, scalograms were generated using the continuous wavelet transform (MATLAB function, cwt) applied to time-series signals [29]. A scalogram displays both time and frequency information, providing a visual representation of the keyhole oscillation within a short time window. This allows for the identification of oscillatory patterns or changes in the keyhole dynamics that may not be captured by the FFT alone, since FFT only provides an amplitude spectrum for the entire scan. Detailed information about selecting the wavelet transform parameters can be found in appendix A.

Data Labelling: the scalograms obtained from the continuous wavelet transform were labeled with simultaneous operando x-ray images as the ground truth [26]. The x-ray image sequence was processed using ImageJ software [30]. A background image captured before the laser entered the FOV was subtracted from each frame of the image sequence. This operation enhances the contrast of the keyhole region in the subsequent analysis. After background subtraction, the keyhole morphologies were extracted from the processed images. The timestamp of each image was used to identify the keyhole pore generation moments, which corresponded to the separation of a bubble from the keyhole that eventually formed a keyhole pore. The scalograms were labeled as either ‘Pore’ or ‘Non-pore’ based on whether a keyhole pore generation moment or moments were observed within the specific time window of the scalogram or not. Only pores with a diameter larger than 25 µm were labelled because it has been proven that small spherical pores have little or no impact on the material properties [31]. Those cases where bubbles were recaptured by subsequent keyhole drilling were not labelled as ‘Pore’. Such labeling provided the ground truth data for training and evaluating the machine learning models. The goal for real time control is to achieve a temporal resolution as high as possible; accordingly, the effect of time window length was investigated in this paper. For simplicity and uniformity, ‘segment’ is used to represent the time window length. For example, the 0.5 ms segment means each piece of raw data spanning 0.5 ms time period was used as a single datum point in machine learning models.

Methods to deal with imbalanced dataset: in reality, non-pore cases are often more prevalent than pore cases which makes the class ratio deviate from 1:1. This problem becomes worse for smaller segments. For example, the 0.1 ms segment data has a ratio of non-pore to pore cases of 20:1. For an imbalanced dataset, a machine learning model generally does not work well for identifying the minority class, which in our work is class ‘Pore’. To solve this problem, the ‘down sampling’ strategy was used which reduced the number of samples in non-pore class to create a balanced dataset. The down sampled non-pore dataset was randomly chosen from the entire non-pore dataset. This data selection procedure was repeated 10 times every time the training was performed. The reason why oversampling (i.e. data replication) was not chosen was that if the model made one mistake in classifying one pore case, the mis-prediction would be repeated for all the identical pore cases and thereby decrease the accuracy.

Deep learning with single-sensor data: choosing neural network architecture is one of the key topics in deep learning. Only by adapting the right neural network, can validity and fidelity of the model be promised. AlexNet, SqueezeNet, GoogLeNet, ResNet-50, etc. are commonly used neural networks for various computer vision applications including image classification. In this work, we chose SqueezeNet developed by Forrest N. Iandola et al for our binary image classification problem (‘Non-pore’ and ‘Pore’) [32]. Detailed information about the neural network selection can be found in appendix B. SqueezeNet is a convolutional neural network with 68 layers, 75 connections and 1.2 M total learnables. The original architecture is designed for the 1000 categorical image classification problem. For our binary classification problem, the ‘conv10′ and ‘ClassificationLayer_predictions’ layers were replaced accordingly.

For the SqueezeNet training, the labeled 2D scalograms were resized to fit the input of SqueezeNet and then randomly divided into training and testing data with a ratio of 8:2. Every accuracy was an average of 10 repeated runs to ensure robustness and reliability. In order to speed up the training procedure, transfer learning was adapted by loading a version of the network that was pre-trained on more than a million images from the ImageNet database. Stochastic gradient descent with momentum (SGDM) algorithm was used to improve the neural network training. Some hyperparameters like MiniBatchSize, MaxEpochs, InitialLearnRate were tuned to get the best accuracy. The overall workflow for the deep learning workflow for detecting keyhole pores using single-sensor data is shown in figure 3.

Figure  3.  Chart describing the deep learning workflow for detecting keyhole pores in the LPBF process using single-sensor data. (a) The time-series signals were obtained from the sensing data, which were then converted into scalograms. These scalograms were labeled as either ‘Pore’ or ‘Non-pore’ based on operando synchrotron x-ray images. The labeled scalograms were randomly divided into training and testing data sets with a ratio of 8:2. The trained model derived from the training process was assessed using standalone testing data, which were never involved in the training process. (b) The confusion matrix of our deep learning approach.

DownLoad: Full-Size Img PowerPoint

Machine learning with multi-sensor data: the overall workflow for the machine learning approach using multi-sensor data is shown in figure 4. For the feature extraction, SqueezeNet neural networks were trained separately for different types of signals. The features were extracted from the ‘pool10′ layer from the trained SqueezeNet neural network. Two features could be extracted from each sensor. The extracted features were stacked together to generate a single data matrix, which was used as the input for the classification. Several machine learning algorithms including support vector machine (SVM), K-Nearest Neighbors (KNN), ensemble, Naïve Bayes, decision tree and preset neural networks were applied to the classification problem. Bayesian optimization was used to select their hyperparameter values. To rule out the effect of randomness, every accuracy was an average of ten repeated runs to ensure robustness and reliability. Due to the flexibility of the feature fusion framework, the input data could be combinations of different sensor signals. During the training for neural networks and classification machine learning algorithms, only the training dataset was used to make sure that none of the testing dataset was included in the training process.

Figure  4.  Schematic diagram of the machine learning workflow for detecting keyhole pores in the LPBF process using multi-sensor data. The features extracted from different sensors were stacked together to form one matrix. The matrix was used to as the input for different machine learning classification algorithms (SVM, KNN, ensemble, Naïve Bayes, decision tree and preset neural networks). The trained model was assessed using a separate set of testing data. By fusing features extracted from different sensors, the prediction accuracy for different sensor combinations could be evaluated.

DownLoad: Full-Size Img PowerPoint

Evaluation metrics: instead of using the conventional parameters (i.e. accuracy, precision, recall, F-1 score) to evaluate model performance, here we used the overall accuracy, true positive rate which was ‘Non-pore’ prediction accuracy and true negative rate which was ‘Pore’ prediction accuracy as the evaluation metrics because the non-pore and pore prediction accuracy are the most important information needed for the community. The confusion matrix is present in figure 3(b). The accuracy is calculated as $\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}$, which represents the overall correctness of the predictions. The ‘Non-pore’ prediction accuracy (true positive rate) is calculated as $\frac{\text{TP}}{\text{TP}+\text{FN}}$. The ‘Pore’ prediction accuracy (true negative rate) is calculated as $\frac{\text{TN}}{\text{TN}+\text{FP}}$.

All the analyses were performed using MATLAB R2023b with Wavelet Toolbox version 23.2, Deep Learning Toolbox version 23.2, and Statistics and Machine Learning Toolbox version 23.2. They were run on a computer with an AMD Ryzen Threadripper PRO 5995wx 64-CORES, 2701 MHz, 64 Cores, 128 Logical Processors processor using 512 GB of RAM and NVIDIA RTX 6000 Ada Generation GPU.

3.   Results and discussion

3.1   Process sensing of keyhole oscillation

Our previous work established a strong correlation between the keyhole oscillation behavior and pore generation [26]. Specifically, the keyhole pores are associated with the perturbative oscillation of an unstable keyhole, which is caused by the stochastic collapsing of the protrusions in front and rear keyhole walls. Therefore, the detection of a pore generation is essentially the probing of keyhole oscillation frequency and modes.

Optical and acoustic signals serve as valuable indicators of the laser melting process. The keyhole oscillation is characterized by a distinct fluctuation observed in the NIR images. On the extracted NIR intensity curve, the call-out of NIR images corresponded to the three consecutive crests and the two troughs between them (figure 5(a)). Assuming a uniform emissivity near the keyhole region, the optical emission captured by the NIR imaging provides an estimation of the temperature distribution around the keyhole [26]. Brighter NIR intensity indicates a hotter temperature, while dimmer NIR intensity corresponds to a colder temperature. Previous studies [33, 34] have demonstrated that the vapor plume velocity is directly related to the temperature. A higher temperature is associated with a faster vapor plume velocity, and vice versa. The variation in the plume displaces the surrounding gas, leading to the generation of sound [35]. The acoustic emission during the laser melting process could be detected by the microphones and used to characterize the process itself (figure 5(a)). Notably, the acoustic signal was in phase with the NIR intensity. This is because a high-temperature keyhole induced stronger plume variations, resulting in louder sound generation, which was confirmed by the presence of plume structures in the three call-out NIR images corresponding to the crests of the acoustic intensity curve. In contrast, a low-temperature keyhole generated weaker plume variations, leading to quieter sound. This was supported by the absence of plume structures in the two call-out NIR images corresponding to the troughs of the acoustic intensity curve. The result shown here is consistent with a previous work discussing the interplay between the plume structure and keyhole dynamics [36].

Figure  5.  Comparison of time-series signals in different modalities. (a) Alignment of time-series signals between NIR and ultrasonic microphone with the LPBF chamber door open. Data at certain time steps were called out with corresponding NIR images. (b) Alignment of time-series signals in all modalities with the chamber door closed. The laser spot size was ∼82 μm, the laser power was 200 W, and the scan speed was 600 mm s^-1.

DownLoad: Full-Size Img PowerPoint

It is important to note that the laser scan in figure 5(a) was conducted with the chamber door open (i.e. laser melting in air). Another laser scan (figure 5(b)) with the same P-V condition was performed with the chamber door closed (i.e. laser melting in Ar). The consistent results reveal that the gas environment, as well as the ambient light and sound conditions at the beamline, had no major effects on the sensor data. In figure 5(b), the signals in all four modalities are presented in a column. It is evident that the signal from the ultrasonic microphone well aligns with the NIR signal, exhibiting matching peaks. By contrast, the audible signal exhibits fewer peaks within the examined time window. This can be attributed to the higher maximum frequency response of the ultrasonic microphone (100 kHz), compared to the dominant frequency of the keyhole oscillation (∼34 kHz in the case shown in figure 5). The maximum frequency response of the audible microphone was only 20 kHz, which was lower than the keyhole oscillation frequency. As a result, the ultrasonic microphone was more responsive to the keyhole oscillation than the audible microphone. The photodiode signal appears dissimilar to the signals from the other modalities, which will be discussed in the next section. However, the generally good alignment between most of the modalities confirms the inherent connection between optical and acoustic signals.

An FFT was performed on the time-series signals (figures 6(a)-(j)) to gain more insights in the frequency domain (figures 6(b)-(k)). In the FFT spectra of the NIR signal (figure 6(a)), a single dominant frequency of 34 kHz is visible, indicating the presence of a well-defined keyhole oscillation. This frequency corresponds to the periodic oscillation observed in the NIR intensity curve in figure 5, and the period of 0.029 ms matches well with the estimated period obtained by subtracting the timestamps of consecutive crests in figure 5. This frequency is also consistent with previous literature [26, 37]. This is the intrinsic keyhole oscillation defined in our previous work [26], which is primarily caused by the varying balance between Marangoni convection, surface tension, and recoil pressure.

Figure  6.  Time-series signals (left column), FFT spectrum (middle column), and wavelet analyzes (right column) of single-track scanning laser melting of Ti-6Al-4V plate samples. (a)-(c) NIR. (d)-(f) ultrasonic microphone. (g)-(i) audible microphone. (j)-(l) photodiode. The laser spot size was ∼82 μm, the laser power was 200 W, and the scan speed was 600 mm s^-1.

DownLoad: Full-Size Img PowerPoint

The dominant frequency of the ultrasonic microphone signal (figure 6(e)) is the same as that of the NIR signal. This observation is in line with the good alignment between the ultrasonic and NIR signals in figure 5. Although the FFT of the audible microphone signal reveals multiple influential frequency components, the 34 kHz frequency representing the intrinsic oscillation is still noticeable (figure 6(h)). This is due to the high data collection rate of 1 MHz, which far exceeds the required sampling rate according to the Nyquist-Shannon theorem. However, because the frequency response of the audible microphone is limited to 20 kHz, it is less sensitive to the high-frequency components of the keyhole oscillation.

A recent paper proposed that the microphone signal collected during the LPBF process were associated with both the vapor plume variation and the keyhole geometry [38]. That study considered the keyhole as a whistle, which could generate sound in certain frequency when a vapor flow was ejected out. It was found that the characteristic frequency of the acoustic signal was strongly dependent on the keyhole morphology. This might be able to explain the differences in frequencies between acoustic signal and optical signal observed in our study to some extent. Indeed, a comprehensive understanding of the acoustic emission from the LPBF process demands more research efforts. It is particularly important to develop an approach to distinguish the characteristic acoustic frequency (airborne pressure wave) and the event frequency (keyhole oscillation).

The time-series signal from the photodiode appears different from the other modalities at first glance. However, its FFT spectrum (figure 6(k)) reveals that the photodiode also captured the 34 kHz intrinsic oscillation to a certain extent. The photodiode in our experiment primarily measured the back reflection of the processing laser, which might be qualitatively correlated with the keyhole morphology and thereby could serve as an indicator of keyhole oscillation. The dissimilarity between the photodiode signal and the other modalities could be attributed to two factors. Firstly, the photodiode detected the reflected laser from only a specific solid angle rather than covering the entire angle range, so it was less sensitive to the morphological change of the keyhole and laser absorptivity than using a device like integrating sphere [14, 37]. Secondly, the photodiode also detected optical emissions from the hot melt pool within the narrow spectrum of the bandpass filter, which was associated with the keyhole oscillation but might not be in phase with the laser back reflection.

To capture both temporal details and frequency information, wavelet analysis was performed on the time-series signals, resulting in 2D scalograms (figures 6(c)-(l)). Unlike FFT, which calculates frequency based on the entire length of signal without specifying the corresponding time intervals, the scalogram reveals the frequency components and their time evolution. The scalogram of the NIR intensity (figure 6(c)) shows that the signal oscillates at a relatively constant frequency of ∼34 kHz as the time progresses. Such single-frequency characteristic suggests a stable keyhole condition with defined intrinsic oscillation.

The scalogram of the ultrasonic microphone (figure 6(f)) is very similar to that of the NIR signal, indicating a strong correlation between these two modalities. In comparison, the scalogram of the audible signal (figure 6(i)) exhibits different frequency components due to its distinct frequency response, as discussed previously. However, the scalogram provides clear temporal resolution of frequency components, which is missing in FFT (figure 6(h)). Such temporal resolution is crucial for correlating frequency information with the onset of pore generation. Although the intrinsic oscillation frequency is less pronounced in the scalogram of the photodiode signal (figure 6(l)) than in figure 6(c) or (f), it is still discernible. This suggests that the photodiode is able to probe some aspects of the keyhole oscillation, albeit with ambiguity. The scalograms provide valuable insight into the time and frequency characteristics of the signals, enabling the analysis of keyhole pore generation and its correlation with keyhole oscillation.

When shifting from a ‘Non-pore’ case (figure 6) to a ‘Pore’ (figure 7) case, the frequency of intrinsic oscillation decreases and it is not a well-defined single frequency anymore. This is because the occasional collapsing of protrusions on keyhole walls disturbs the intrinsic oscillation under unstable keyhole condition. The comparison between different sensor signals in the unstable keyhole case is similar to that in the stable keyhole case. The NIR (figures 7(a)-(c)) and ultrasonic (figures 7(d)-(f)) signals exhibit substantial similarity, indicating the reliability of these two sensors for detecting keyhole oscillations. The audible signal (figures 7(g)-(i)) is more sensitive to the low-frequency components of keyhole oscillation, mainly due to its low-frequency response up to 20 kHz. The photodiode data (figures 7(j)-(l)) displays a seemingly dissimilar frequency compared to the others, which is also observed in the ‘Non-pore’ case. This explains the lower prediction accuracy when using photodiode data.

Figure  7.  Time-series signals (left column), FFT spectrum (middle column), and wavelet analyzes (right column) of single-track scanning laser melting of Ti-6Al-4V plate samples. (a)-(c) NIR. (d)-(f) ultrasonic microphone. (g)-(i) audible microphone. (j)-(l) photodiode. The laser spot size was ∼82 μm, the laser power was 200 W, and the scan speed was 300 mm s^-1.

DownLoad: Full-Size Img PowerPoint

The frequency and mode of keyhole oscillations, ranging from stable to unstable, can be identified based on the scalograms (figure 8). The NIR, ultrasonic microphone, and photodiode exhibit similar scalogram features across different scan speeds at the same laser power (figures 8(c), (f) and (l)). In the stable keyhole mode with a scan speed of 600 mm s^-1, the intrinsic oscillation is characterized by a dominating single-mode frequency (∼34 kHz in this case) throughout the time period. When the melting mode transitions (figures 8(b), (e) and (k)) to the unstable keyhole state (figures 8(a), (d) and (j)), the oscillation shifts from the single mode to multi-mode, meaning multiple frequencies are superimposed within the same time period. The lower-frequency components are associated with the intrinsic oscillation, while the higher-frequency components are associated with the perturbative oscillation. The intrinsic oscillation frequency decreases as the scan speed decreases. This is because a lower scan speed results in increased energy density, leading to a larger melt pool volume and a more significant damping effect [14, 26, 39].

Figure  8.  Scalograms of single-track laser melting of Ti-6Al-4V plate based on signals in different sensing modality. (a)-(c) Scalograms of processed NIR data, (d)-(f) scalograms of ultrasonic microphone data, (g)-(i) scalograms of audible microphone data, and (j)-(l) scalograms of photodiode data under various laser scan speeds. The laser spot size was ∼82 μm, and the laser power was 200 W.

DownLoad: Full-Size Img PowerPoint

The scalograms of the audible microphone exhibit noticeable differences compared to those in other sensing modalities. This is mainly because this microphone is more sensitive to the audible range of frequency (<20 kHz). In the stable keyhole mode, the frequency features associated with the intrinsic oscillation are less distinctive (figure 8(i)), because the intrinsic oscillation frequency (∼34 kHz) is higher than the maximum soundwave frequency that the audible microphone is sensitive to. When the keyhole enters the transition or unstable mode, the intrinsic oscillation frequency decreases and crosses the ultrasonic edge, entering the audible range (figures 8(g) and (h)). Consequently, the frequency features associated with the intrinsic oscillation becomes more observable in the scalogram.

3.2   Machine-learning-aided localized keyhole pore detection using single-sensor data

The number of cases in the training set and testing set are summarized in table 1. For larger segment length, the dataset contains smaller number of cases with the same number of experiments. The dataset for Scenario ‘L’ is larger than that of Scenario ‘S’ because more pores were generated and additionally more experiments were conducted. Down sampling was applied to all the dataset to ensure that the non-pore and pore class ratio was 1:1.

Table  1.  Training set/testing set for different segments under different scenarios.

Scenario	Segment length	Training set/ testing set	Class ratio (Non-pore: Pore)
L(arge)	0.1 ms	980/246	1:1
	0.2 ms	804/202	1:1
	0.5 ms	518/130	1:1
S(mall)	0.1 ms	208/52	1:1
	0.2 ms	188/48	1:1
	0.5 ms	134/34	1:1

 | Show Table

DownLoad: CSV

The prediction accuracy results based on single sensor data are presented in figure 9. The accuracy using NIR signal is the highest, which indicates that the keyhole oscillation behaviors are well captured by 2D NIR images. For Scenario ‘S’, the prediction accuracies of the acoustic sensors are higher than that of the single photodiode sensor, with the 100 kHz microphone working better than the 20 kHz microphone (figure 9(a)). This is consistent with our observations about the time-series signals from different sensors discussed above. However, for Scenario ‘L’, the prediction accuracies of acoustic sensors are lower than that of photodiode sensor (figure 9(b)). The 20 kHz microphone yields only ∼50% prediction accuracy, equivalent to a complete failure for a binary classification.

Figure  9.  Deep learning prediction results using single-sensor data for different scenarios. Overall prediction accuracy, Non-pore prediction accuracy and Pore prediction accuracy for (a) Scenario ‘S’ and (b) Scenario ‘L’. P-V conditions covered in both scenarios are present in figure 2(a).

DownLoad: Full-Size Img PowerPoint

While using NIR and photodiode signals as the input, the prediction accuracies for Scenario ‘L’ is higher than that of Scenario ‘S’ which is reasonable because the data variability for Scenario ‘L’ is larger. This finding makes the low prediction accuracy for Scenario ‘L’ using 100 kHz and 20 kHz acoustic signals unusual. Figure 10 depicts the raw acoustic signals and frequency analysis results for a similar P-V condition from Scenario ‘S’ and ‘L’ and the corresponding fft, cwt results. Data shown in figures 10(g) and (j), collected at a later time in the experiment than those in figures 10(a) and (d), appear to exhibit a much lower signal-to-noise ratio. The decay of data quality is also evident in the frequency analyzes. The FFT spectra shown in figures 10(h) and (k) are almost featureless.

Figure  10.  Performance decay of microphones caused by x-ray beam damage. Raw time-series acoustic signals (left column), FFT spectra (middle column), and wavelet analyzes (right column) of two similar P-V conditions. Data in (a) and (d) were collected at an earlier time in the synchrotron experiment than (g) and (j).

DownLoad: Full-Size Img PowerPoint

We attribute the decay of microphone data to the sensor damage caused by x-ray radiation. At the beamline, the drop of signal-to-noise ratio in microphone data occurred after less than a day of experiment. Our synchrotron experiment involved using an intense white beam for x-ray imaging with the high-energy flux unfiltered. Therefore, even though the microphones were not positioned in the x-ray path, the scattered x-ray still caused severe damage to the microphones. Based on our observation, the 20 kHz microphone lost its performance more quickly than the 100 kHz microphone.

Figure 9 also reveals a general trend of accuracy decrease as the segment length gets shorter. This is mainly because less features are captured by a short segment of data both in time domain and frequency domain. It is straightforward that shorter segment would include less information in time domain. As for the frequency domain, with the decrease of segment time, the lowest resolvable frequency increases because at least half of the wave needs to be included in the segment. Additionally, the lower limit of the calculated frequency for continuous wavelet transform increases with the decrease of signal length. With the same sampling rate, the signal length decreases with the decrease of segment length since the signal length equals to sampling rate multiplied by segment length. This makes choosing proper wavelet parameters critical. Detailed information about how to choose them is discussed in appendix A. The corresponding signal length and lower limit of frequency for different segment length are listed in table 2. The lowest resolvable frequency increases with the decrease of segment length.

Table  2.  Lowest resolved frequency for different sensors under different sampling rate and segment length.

Sensor	Sampling rate (kHz)	Segment length (ms)	Signal length	Lowest resolved frequency (kHz)	Lower frequency limit for cwt (kHz)
NIR	250	0.1	25	5	4.13
		0.2	50	2.5	3.18
		0.5	125	1	0.83
20 kHz/100 kHz microphone & photodiode	1000	0.1	100	5	4.13
		0.2	200	2.5	3.18
		0.5	500	1	0.83

 | Show Table

DownLoad: CSV

Another reason for the lower accuracy at a shorter segment length is related to data labelling. Based on the simultaneous x-ray images, we were able to identify the bubble pinch-off moment and label the pore generation events at a temporal resolution up to 0.02 ms (calculated from the x-ray imaging with the frame rate of 50 kHz). Only bubbles that eventually became pores in the sample were labelled as ‘Pore’. Many bubbles were recaptured by the subsequent keyhole and vanished. Sometimes, this recapturing event may take as long as 0.2 ms. As shown in figure 11, a bubble that pinched off at 0 ms was eventually recaptured by the keyhole at 0.2 ms. This type of bubble was labeled as ‘Non-pore’ in our dataset. However, 0-0.2 ms might be labelled as ‘Pore’ if the signals beyond 0.2 ms were not included in this time segment. This means, for time segment length smaller than 0.2 ms, certain mislabeling rate exists. As the time segment length continues to decrease, the mislabeling rate increases, contributing partially to the decrease of prediction accuracy.

Figure  11.  X-ray images of a bubble recapturing event in laser melting of a Ti-6Al-4V plate. The specific bubble was highlighted in the red dashed circle. The bubble survived in the melt pool for 0.2 ms before vanishing. This causes the mislabeling of data in the machine learning model when using a segment smaller than 0.2 ms. The laser spot size was ∼82 μm, the laser power was 300 W, and the scan speed was 250 mm s^-1.

DownLoad: Full-Size Img PowerPoint

The inherent frequency limit and mislabeling rate suggest that there may be a fundamental limit in the temporal resolution when using the frequency information to detect keyhole pore generation in LPBF. Regardless the number of sensors and datasets, it could be challenging to achieve high prediction accuracy at <100 µs time resolution which is constrained by the physics underlying keyhole dynamics.

In general, the keyhole pore prediction accuracy for Scenario ‘L’ (figure 9(b)) is much higher than that of Scenario ‘S’ (figure 9(a)). The results for acoustic sensor are not accounted because the data quality was much worse for Scenario ‘L’ due to sensor damage. For 0.5, 0.2, and 0.1 ms segments, the highest prediction accuracy for ‘L’ is 12%, 9% and 1% higher than those of ‘S’, respectively. This difference can be contributed to data variability and/or size of the datasets. In order to further investigate the effect of data constitution, one more sample filtering criteria, denoted as ‘Separation’, was adopted: ‘Non-pore’ cases were only selected from those P-V conditions which did not generate pores at all. This differs from previous sample selection in which ‘Non-pore’ cases were also extracted from those laser melting processes where pores were generated occasionally. In essence, this analysis is meant to distinguish keyhole behaviors under different P-V conditions. This may better resemble the practical printing processes, in which the unstable keyhole mode melting is triggered by the sudden drift of laser conditions.

Figure 12 shows the prediction results using single sensor data adopting the ‘Separation’ data selection criteria. The prediction accuracies are largely increased for all sensors under both scenarios, except for the damaged 20 kHz microphone under Scenario ‘L’, which remains low. Using the NIR imaging data as the input, prediction accuracies of 97%, 96%, and 91% can be achieved with 0.5, 0.2, and 0.1 ms temporal resolution, respectively. Notably, the prediction accuracies for ‘Pore’ cases are even higher. Under Scenario ‘S’, the prediction accuracy drops but remains beyond 80% even for the 0.1 ms segment. The lower accuracy of the 0.5 ms segment here is mainly caused by the smaller training dataset.

Figure  12.  Deep learning prediction results using single-sensor data for different scenarios with ‘Separation’ data selection. Overall prediction accuracy, Non-pore prediction accuracy and Pore prediction accuracy for (a) Scenario ‘S’ and (b) Scenario ‘L’.

DownLoad: Full-Size Img PowerPoint

Since our machine learning model relies on frequency information to predict keyhole pores, the sampling rate of raw sensor signals needs to be sufficiently high. The effect of sampling rate on the prediction accuracy was studied by intentionally downgrading the original rate. Figure 13 shows the overall prediction accuracy, Non-pore prediction accuracy and Pore prediction accuracy at different frequency bandwidth (a)-(c) and sampling rates (d)-(f) under Scenario ‘L’ with ‘separation’ criteria applied. In figures 13(a)-(c), the x-axis represents the upper limit of the frequency bandwidth in scalograms that were used in the machine learning model, while the lower limit was set to be 1 kHz. For both NIR and photodiode signals, the prediction accuracies increase rapidly with increasing frequency bandwidth in scalograms. However, the prediction accuracy with the NIR signal reaches the plateau more quickly than that of the photodiode signal and it is consistently higher. This once again demonstrates the high fidelity of the NIR imaging for capturing keyhole oscillation and pore generation. Figures 13(d)-(f) are plots of prediction accuracies as a function of signal sampling rate, which show the similar trend. Based on Nyquist-Shannon sampling theorem, in order to capture f Hz frequency information, the sampling rate needs to be at least 2f Hz. Based on our analys, in order to achieve a reasonably high prediction accuracy, the data collection rate should be higher than 100 kHz for the NIR imaging and higher than 600 kHz for the photodiode.

Figure  13.  Prediction accuracies at different frequency bandwidth and sampling rate under Scenario ‘L’ with ‘separation’ criteria applied. (a)-(c) Accuracy results using photodiode and NIR signals under different selections of frequency bandwidth in scalograms. (d)-(f) Accuracy results using photodiode and NIR signals at different sampling rates. The full frequency bandwidth corresponding to each sampling rate was used. The original sampling rate was 1 MHz for photodiode and 250 kHz for NIR. The segment time was 0.5 ms.

DownLoad: Full-Size Img PowerPoint

3.3   Machine-learning-aided localized keyhole pore detection using multi-sensor data

The prediction accuracy results for different combinations of sensors are shown in figure 14. For Scenario ‘L’, since the 20 kHz microphone was damaged by x-ray radiation when the data were collected, its signal was not included in the feature fusion. By comparing the data fusion prediction accuracies with those relying on single-sensor data (figures 9 and 12), it appears that the feature fusion does not improve the prediction accuracy. Instead, in some cases, the prediction accuracy using multi-sensor data is lower than using any single sensor. We speculate that the low prediction accuracy may be caused by the lack of complementary characteristics in these sensor signals. For example, the process information detected using the 20 kHz microphone may already be covered in the 100 kHz microphone data. Also, the 100 kHz microphone data exhibit similarity with the NIR data, as shown in figure 5.

Figure  14.  Machine learning prediction results using multi-sensor data from different combinations of four sensors under different schemes. In each sub-figure, the left, middle, and right panels show the overall prediction accuracy, Non-pore prediction accuracy, and Pore prediction accuracy, respectively. (a-b) Results for Scenario ‘S’ without and with adopting ‘separation’ data selection criteria. (c-d) Results for Scenario ‘L’ without and with adopting ‘separation’ data selection criteria. In the legend, ‘100′ stands for 100 kHz microphone, ‘20′ stands for 20 kHz microphone, ‘PD’ stands for photodiode and ‘NIR’ stands for near infrared camera.

DownLoad: Full-Size Img PowerPoint

4.   Conclusions

In this study, machine learning models using single sensor (i.e. NIR, 100 kHz and 20 kHz microphones, and photodiode) and multiple sensors to predict keyhole pore generation in LPBF of Ti-6Al-4V were developed. Simultaneous synchrotron x-ray imaging provided the high-fidelity ground truth for benchmarking different sensors. For keyhole pore detection with single-sensor data, scalograms generated by continuous wavelet transform were used as input and deep learning using SqueezeNet architecture was applied. For prediction with multi-sensor data, a data fusion scheme with features extracted using trained SqueezeNet neural network architecture and classification using different machine learning algorithms was developed to evaluate the efficacy of different sensor combinations.

Consistent with our previous study, it was found that the keyhole pore generation is directly associated with the keyhole oscillation behavior, which could be captured by the light and sound sensors. Discovered in the present study was that the acoustic signal and optical signal were well aligned with each other. Based on NIR images and x-ray images, it was found that the airborne acoustic waves were caused by the rapid ejection of vapor plume from the keyhole. The repeated morphological change of the keyhole was responsible for the periodical pressure variations in the ambient gas and hence sound waves.

When predicting keyhole pores with single-sensor data, the NIR camera signal was found to offer the highest prediction accuracy, followed by 100 kHz and 20 kHz microphone, and the photodiode sensitive to the processing laser wavelength had the lowest prediction accuracy. Data collected under more than 70 P-V conditions were grouped into two scenarios: one covering a larger range of scanning velocity (‘L’) and one covering smaller (‘S’). For Scenario ‘L’, over 90% prediction accuracy was achieved with a temporal resolution as short as 0.1 ms. In addition to the data variability, the effects of raw data sampling rate and frequency bandwidth in scalograms on the prediction accuracy were also quantitatively investigated. The results revealed that high prediction accuracies could be achieved when the NIR data sampling rate was higher than 100 kHz and photodiode data sampling rate higher than 600 kHz.

It was interesting to find that the photodiode was quite robust in synchrotron experiment, while the microphones could be damaged by the high-energy x-ray radiation gradually, even though they were not placed in the x-ray beam path. The mechanism of such performance degradation is still unclear though.

Feature fusion using different combinations of the light and sound sensors did not increase the prediction accuracy. By no means does the study presented here discourage the application of multi-modality sensors and feature fusion approaches. It only suggests that one should consider whether additional sensors may provide richer information on the LPBF process. After all, the sensors in this study were positioned in an off-axis geometry and all probe the keyhole oscillation behavior.

5.   Future perspectives

Even though the NIR data in our study offers the highest prediction accuracy for keyhole pores, it is often impractical to set up a NIR camera in a commercial LPBF system and use its 2D optical imaging data for part-level process monitoring and control. In contrast, photodiodes and microphones are more accessible, and the 1D data they generate are much easier to store and process. As demonstrated in this work, reasonably high prediction accuracy can be achieved even with a single photodiode or microphone. Adding more sensors at different locations in the chamber, measuring light and sound waves across various wavelengths, should further enhance prediction accuracy. Meanwhile, like all machine learning approaches, classification accuracy can also be improved by increasing the data used to train the algorithm.

Regarding temporal resolution, our study suggests that a fundamental limit may exist for keyhole pore prediction based on frequency information. This is governed collectively by keyhole oscillation frequency and the dynamic interaction between gas bubbles and the advancing keyhole. While prediction accuracy can undoubtedly be improved, achieving a temporal resolution better than 100 µs may be challenging. Assuming a laser scanning speed of 1 m s^-1, this translates into a 0.1 mm spatial resolution. In other words, this approach can detect keyhole pore generation within a 100 µm distance. For practical applications, we believe this is sufficiently high, given that keyhole pores are typically tens of micrometers in size. Therefore, additional efforts on further increasing the temporal resolution may not be needed.

Future work in this research area may focus on quantifying two phenomena associated with keyhole pore detection: (1) pore motions inside the melt pool and (2) pore removal upon repeated melting. In this approach, we labeled the moments when bubbles separate from the keyhole as ‘Pore’ cases. However, after a bubble pinches off, it will move inside the melt pool, primarily driven by melt flow. Therefore, the final pore locations in a part differ from their generation locations. In our prior work, we quantified the separation of bubble generation location and final pore locations for Ti-6Al-4V [26]. For other materials and different laser beam sizes, the spatial or temporal separation between these two locations need to be measured again, which can be achieved through multi-physics simulations. Another factor that could reduce prediction accuracy in practical LPBF processes is pore removal induced by laser remelting. A keyhole pore might be removed by direct interaction with the scanning keyhole or dragged to the sample surface by melt flow. These are stochastic events that may be difficult to simulate, so innovations are needed to account for such effects.

Appendix A.   Parameter selection for continuous wavelet transform (CWT)

The time-bandwidth product $P^2$ and symmetry $\gamma$ are two parameters affect the shape of a Morse wavelet and control the frequency limits which are correlated with signal length and sampling rate. Very small time-bandwidth and large symmetry values produce undesired time-domain sidelobes and frequency-domain asymmetry [41]. Based on these two factors, two wave parameter sets were adopted in our work: $P^2$ = 60, $\gamma$ = 3 for longer time segment and $P^2$ = 5, $\gamma$ = 5 for shorter time segment. Another parameter is VoicesPerOctave which means the number of voices per octave that the cwt scales are discretized and 48 was chosen to get high resolution. The signal extension pattern at the boundary was set as symmetrically.

Appendix B.   Neural network architecture selection and hyperparameter optimization

Choosing neural network architecture is one of the key topics in deep learning. Only by adapting the right NN, can validity and fidelity of the model be promised. If the NN is too complex, the model is prone to overfitting. If the NN is too simple, it will not capture the complexity behind the problem we studied. In MATLAB, there are 20 pretrained image networks as listed in table A1. We tested these 20 networks accordingly and chose the one with highest validation accuracy for our keyhole porosity detection problem. For networks giving similar accuracy results, the one with faster training speed was chosen. Based on these two criteria, SqueezeNet was chosen in this study as shown in figure A1. Tuning hyperparameters is also crucial in training a neural network. In our study, we used SGDM solver for training and specified the ‘Minibatchsize’ as 16. The ‘InitialLearnRate’ was specified as 510^-5 in order to get better stability and validation accuracy. The epoch number was chosen among 6 ∼ 50 to avoid underfitting and overfitting.

  Figure A1.  The SqueezeNet architecture, which has 68 layers, 75 connections and 1.24 M learnables (this figure is obtained with MATLAB plot deep learning graph function).

DownLoad: Full-Size Img PowerPoint

  Table A1.  Pretrained image networks in MATLAB [42].

Image networks	Total learnable	Depth	Number of layers	Number of connections
SqueezeNet	1.24 M	18	68	75
GoogleNet	7 M	22	144	170
ResNet-50	25.6 M	50	177	192
EfficientNet-b0	5.31 M	82	290	363
DarkNet-53	41.6 M	53	184	206
DarkNet-19	20.8 M	19	64	63
ShuffleNet	1.4 M	50	172	187
NasNet-mobile	5.3 M	N/A	913	1072
NASNet-Large	88.9 M	N/A	1243	1462
Xception	22.9 M	71	170	181
Places365-GoogleNet	7 M	22	144	170
MobileNet-v2	3.5 M	53	154	163
DenseNet-201	20 M	201	708	805
ResNet-18	11.7 M	18	71	78
Inception-ResNet-v2	55.9 M	164	824	921
Inception-v3	23.9 M	48	315	349
ResNet-101	44.6 M	101	347	379
VGG-19	144 M	19	47	46
VGG-16	138 M	16	41	40
AlexNet	61 M	8	25	24

 | Show Table

DownLoad: CSV

As shown in figure A1, the SqueezeNet is a 68-layer deep neural network with 1.24 million learnable parameters which requires a larger dataset and longer time for training. Therefore, creating more data and pre-training the SqueezeNet algorithm will be beneficial. In addition, some models designed for small dataset with higher prediction speed but little sacrifice in prediction accuracy can be a future focus of study [43].

Appendix C.   Classification algorithm selection

MATLAB offers ‘fitcauto’ function to automatically select classification model with optimized hyperparameters. We chose SVM, KNN, ensemble, Naïve Bayes, decision tree and preset neural networks to select. Bayesian optimization was used to optimize the hyperparameters. The corresponding hyperparameters to be optimized for different algorithms are listed in table A2. One representative optimization procedure is shown in figure A2.

  Figure A2.  Representative optimization process for machine learning algorithms using fused features from multiple sensors to detect keyhole pores in LBPF. During it, the classification algorithm was selected among SVM, KNN, ensemble, Naïve Bayes, decision tree and preset neural networks. The corresponding hyperparameters for different classification models were optimized using Bayesian optimization which were columns like Hyperparameter: Value in the figure.

DownLoad: Full-Size Img PowerPoint

  Table A2.  Hyperparameters to optimize for selected classification learners.

Classification learner	Hyperparameters
SVM	BoxConstraint, KernalScale, Standardize
KNN	Distance, NumNeighbors, Standardize
Ensemble	Method, NumLearningCycles, LearnRate, MinLeafSize
Naïve Bayes	DistributionNames, Standardize, Width
Decision tree	MinLeafSize
Neural Network	Activations, Lambda, LayerSizes, Standardize

 | Show Table

DownLoad: CSV