Object classification and visualization with edge artificial intelligence for a customized camera trap platform

Sajid Nazir*, Mohammad Kaleem

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

The camera traps have revolutionized the image and video capture in ecology and are often used to monitor and record animal presence. With miniaturization of low power electronic devices, better battery technologies, and software advancements, it has become possible to use the edge devices, such as Raspberry Pi as camera traps that can not only capture images and videos, but can also enable sophisticated image processing, and off-site communications. These developments can help to provide near real-time insights and reduce the manual processing of images. The on-board image classification and visualization is facilitated by the advancements in the Deep Neural Networks (DNN), transfer learning approaches, and software libraries. This paper provides an investigation of image classification with transfer learning approaches using pre-trained DNN models, and visualizations with Explainable Artificial Intelligence (XAI) techniques on Raspberry Pi Zero (RPi-Z) edge device. The MobileNetV2 model was used for image classification on the Florida-Part1 dataset
obtaining the results for precision, recall, and F1-score as 0.95, 0.96, and 0.95
respectively. We also compared the model performance of MobileNetV2,
EfficientNetV2B0, and MobileViT models for classification on the Extinction dataset
with the best results for precision, recall, and F1-score as 0.97, 0.96, and 0.96
respectively, obtained with the EfficientNetV2B0 model. Two XAI techniques, Gradientweighted-
Class Activation Mapping (Grad-CAM) and Occlusion Sensitivity were used
for visualization through heatmaps, to highlight the relative importance of the image
areas contributing to the DNN model’s prediction, that can also help to understand the
model’s performance and bias. The results provide practical use case scenarios for
utilizing the transfer learning approaches, model optimization and deployment to edge
devices, and model visualizations in ecological research.
Original languageEnglish
Article number102453
JournalEcological Informatics
Volume79
Early online date2 Jan 2024
DOIs
Publication statusPublished - Mar 2024

Keywords

  • Data science
  • Computer vision
  • Deep learning
  • Model generalization
  • Fine tuning
  • Explainable AI
  • Hyperparameter tuning
  • Vision transformers

ASJC Scopus subject areas

  • Ecological Modelling
  • Applied Mathematics
  • Ecology, Evolution, Behavior and Systematics
  • Computer Science Applications
  • Ecology
  • Computational Theory and Mathematics
  • Modelling and Simulation

Fingerprint

Dive into the research topics of 'Object classification and visualization with edge artificial intelligence for a customized camera trap platform'. Together they form a unique fingerprint.

Cite this