👾Integrating artificial intelligence

Cloud vs. fog vs. edge: where to run inference?

Prerequisites

Types of wireless camera traps are discussed below; it's worth reviewing and understanding them before diving into this page.

It's also worth reviewing out Intro to AI for processing camera trap data.

Having access to real-time data is nice, but individual camera traps can easily produce tens-of-thousands of images in a short amount of time (many of which are empty or don't contain animals of interest), so in order to leverage the real-time nature of the data you need automated image processing - and that likely will involve using machine learning (ML) models to predict what's in the images.

Before proceeding it's also worth noting that using machine learning for camera trap image classification is a surprisingly difficult data science challenge, and at the moment, while ML can greatly increase labeling efficiency and help surface much of what you're interested in detecting in an automated fashion, it doesn't replace the need to review images entirely. For most use-cases, you will still need a "human-in-the-loop" to review a lot if not all of the images.

Cloud vs. Fog vs. Edge detection

There are essentially three places where your machine learning models can live and be executed (requesting an ML model prediction is called "inference"), each with different strengths and weaknesses:

• Edge detection is when the inference is performed on each individual camera itself before being transmitted. This is most applicable to satellite-connected cameras (and to a lesser degree cellular cameras) in which bandwidth is highly limited and it's important to only send the smallest and most relevant amount of information to the cloud / end-user. The advantage is that it makes satellite-cameras more cost effective to scale; the disadvantage is that you need to either trust your ML models or have a use-case with some tolerance for false-positive/negatives because the end users likely have very limited ability to verify that the predictions are correct (see the section on satellite cameras in the "Types of wireless camera traps" page for more on that). Machine learning is also a power-intensive process and edge detection may require the installation of larger batteries and solar panels at each device.

• Detection in the Fog is similar to edge detection in that the machine learning is run locally, on-the-ground, before transmitting images to the cloud, but the difference is that instead of performing inference on each individual camera, it's done at a centralized base-station computer after the local cameras have transmitted their images to it. This is most applicable to locally-networked, radio-based camera traps, and may be useful in situations in which you're semi-bandwidth constrained (perhaps you're using an existing low-bandwidth satellite internet connection as your backhaul and you want to filter out all empty images before uploading them to the cloud) or perhaps you don't need the image data to be uploaded and available on the cloud at all but still want to use real-time data and ML processing.

• Detection in the Cloud is, just like it sounds, when the machine learning processing happens in a cloud environment after the images have been uploaded. It's useful if you're not bandwidth constrained and it's not cost-prohibitive to transmit all of your images to the cloud and perform automated labeling and filtering there. For this reason, it's most applicable to cellular cameras and locally-networked, radio-based camera traps with high-throughput backhauls.

Thinking about your end-user first

In addition to considering budget and and existing connectivity options, another thing to think about is who will be using the data, where will they be physically located, and what they will be doing with the information they get from it.

To take an example from the biosecurity (invasive species management) world, you can split the biosecurity monitoring use-cases into two broad buckets: incursion detection and eradication monitoring. For an incursion detection deployment, in which long-term monitoring cameras are set up on remote islands to try to detect new unwanted invasive species incursions, inference in the cloud might make sense because the people who are going to have to react to an incursion and make decisions about them are likely to be in an office somewhere on the mainland rather than on the ground.

If you're using cameras for an on-going eradication effort, however, it's the opposite: the team using the detection information most immediately are already on the the island itself - so it might not make sense for the camera data to get uploaded to the cloud at all. They may also be trying to use the labeled data to do population modeling on the fly to assess whether their efforts are having impact and/or where they should direct their trapping resources, so having a system for wirelessly collecting the data and processing using machine learning might be very helpful. In this case, performing detection in the fog might make the most sense.