TY - JOUR
T1 - Object classification and visualization with edge artificial intelligence for a customized camera trap platform
AU - Nazir, Sajid
AU - Kaleem, Mohammad
PY - 2024/3
Y1 - 2024/3
N2 - The camera traps have revolutionized the image and video capture in ecology and are often used to monitor and record animal presence. With miniaturization of low power electronic devices, better battery technologies, and software advancements, it has become possible to use the edge devices, such as Raspberry Pi as camera traps that can not only capture images and videos, but can also enable sophisticated image processing, and off-site communications. These developments can help to provide near real-time insights and reduce the manual processing of images. The on-board image classification and visualization is facilitated by the advancements in the Deep Neural Networks (DNN), transfer learning approaches, and software libraries. This paper provides an investigation of image classification with transfer learning approaches using pre-trained DNN models, and visualizations with Explainable Artificial Intelligence (XAI) techniques on Raspberry Pi Zero (RPi-Z) edge device. The MobileNetV2 model was used for image classification on the Florida-Part1 datasetobtaining the results for precision, recall, and F1-score as 0.95, 0.96, and 0.95respectively. We also compared the model performance of MobileNetV2,EfficientNetV2B0, and MobileViT models for classification on the Extinction datasetwith the best results for precision, recall, and F1-score as 0.97, 0.96, and 0.96respectively, obtained with the EfficientNetV2B0 model. Two XAI techniques, Gradientweighted-Class Activation Mapping (Grad-CAM) and Occlusion Sensitivity were usedfor visualization through heatmaps, to highlight the relative importance of the imageareas contributing to the DNN model’s prediction, that can also help to understand themodel’s performance and bias. The results provide practical use case scenarios forutilizing the transfer learning approaches, model optimization and deployment to edgedevices, and model visualizations in ecological research.
AB - The camera traps have revolutionized the image and video capture in ecology and are often used to monitor and record animal presence. With miniaturization of low power electronic devices, better battery technologies, and software advancements, it has become possible to use the edge devices, such as Raspberry Pi as camera traps that can not only capture images and videos, but can also enable sophisticated image processing, and off-site communications. These developments can help to provide near real-time insights and reduce the manual processing of images. The on-board image classification and visualization is facilitated by the advancements in the Deep Neural Networks (DNN), transfer learning approaches, and software libraries. This paper provides an investigation of image classification with transfer learning approaches using pre-trained DNN models, and visualizations with Explainable Artificial Intelligence (XAI) techniques on Raspberry Pi Zero (RPi-Z) edge device. The MobileNetV2 model was used for image classification on the Florida-Part1 datasetobtaining the results for precision, recall, and F1-score as 0.95, 0.96, and 0.95respectively. We also compared the model performance of MobileNetV2,EfficientNetV2B0, and MobileViT models for classification on the Extinction datasetwith the best results for precision, recall, and F1-score as 0.97, 0.96, and 0.96respectively, obtained with the EfficientNetV2B0 model. Two XAI techniques, Gradientweighted-Class Activation Mapping (Grad-CAM) and Occlusion Sensitivity were usedfor visualization through heatmaps, to highlight the relative importance of the imageareas contributing to the DNN model’s prediction, that can also help to understand themodel’s performance and bias. The results provide practical use case scenarios forutilizing the transfer learning approaches, model optimization and deployment to edgedevices, and model visualizations in ecological research.
KW - Data science
KW - Computer vision
KW - Deep learning
KW - Model generalization
KW - Fine tuning
KW - Explainable AI
KW - Hyperparameter tuning
KW - Vision transformers
U2 - 10.1016/j.ecoinf.2023.102453
DO - 10.1016/j.ecoinf.2023.102453
M3 - Article
SN - 1574-9541
VL - 79
JO - Ecological Informatics
JF - Ecological Informatics
M1 - 102453
ER -