Check out all the on-demand sessions from the Intelligent Security Summit here.
As the number of machine learning (ML) use cases grow and evolve, an increasing number of MLops organizations are using more ML at the edge — that is, they are investing in running ML models on devices at the periphery of a network, including smart cameras, IoT computing devices, mobile devices or embedded systems.
ABI Research, a global technology intelligence firm, recently forecasted that the edge ML enablement market will exceed $5 billion by 2027. While the market is still in a “nascent stage,” according to Lian Jye Su, research director at ABI Research, companies looking to ease the challenges of edge ML applications are turning to a variety of platforms, tools and solutions to boost an end-to-end MLops workflow.
“We are absolutely seeing MLops organizations increase the use of EdgeML,” said Lou Flynn, senior product manager for AI and analytics at SAS. “Enterprises big and small are running to the cloud for various reasons, but the cloud doesn’t lend itself to every use case — so organizations from nearly every industry, including aerospace, manufacturing, energy and automotive leverage Edge AI to gain competitive advantage.”
Here are 5 reasons MLops teams are giving edge ML a thumbs-up:
Intelligent Security Summit On-Demand
Learn the critical role of AI & ML in cybersecurity and industry specific case studies. Watch on-demand sessions today.
1. Edge devices have become faster and more powerful.
“We have seen multiple companies focus on end-to-end processes around Edge ML,” said Frederik Hvilshøj, lead ML engineer at data-centric computer vision company Encord. The two major reasons, he explained, are that edge devices have become increasingly powerful while model compression has becomes more effective, which allows for running more powerful models at a higher speed. Edge devices also typically live much closer to the data source which removes the necessity to move big volumes of data.
“The combination of the two means that high performance models can be run on edge devices at a close-to real time speed,” he said. “Previously, GPUs living on central servers were necessary to get the high model throughput — but at the cost of having to transfer data back and forth which made the use case less practical.”
2. Edge ML offers greater efficiency.
Today’s distributed data landscape is ripe with opportunity to analyze content to gain efficiencies, said Lou Flynn, senior product manager for AI and Analytics at SAS.
“Many data sources originate from remote locations, such as a warehouse, a stand-alone sensor at a large agricultural site, or even a CubeSat [a square-shaped miniature satellite] as part of a constellation of electro-optical imaging sensors,” he explained. “Each of these scenarios depicts use cases that could gain efficiencies by running Edge ML vs. waiting for data to reconcile in cloud storage.”
3. Bandwidth and cost savings are key.
“You need to run ML models on the edge because of physics (bandwidth limitations, latency) and cost,” said Kjell Carlsson, head of data science strategy at Domino Data Lab, who explained that IoT is not feasible if data from every sensor needs to be streamed to the cloud to be analyzed.
“The network in a supermarket would not support the high definition streaming from a couple dozen cameras, let alone the hundreds of cameras and other sensors you would want in a smart store,” he said. By running ML on the edge, you also avoid the cost of data transfer, he added.
“For example, a Fortune 500 manufacturer is using edge ML to continuously monitor equipment to predict equipment failure and alert staff to potential issues,” he said. “Using Domino’s MLOps platform, they are monitoring 5000+ signals with 150+ deep learning models.”
4. EdgeML helps scale the right data.
The real value of edge ML, said Hvilshøj, is that with distributed devices, you can scale your model inference without having to buy larger servers.
“With scaling inference out of the way, the next issue is collecting the right data for the next training iteration,” he said. In many cases, collecting raw data is not hard, but choosing data to label next becomes hard for large volumes of data. The compute resources on the edge devices can help identify what might be more relevant to label.
“For example, if the edge device is a phone and the user of the phone dismisses a prediction, this can be a good indicator that the model was wrong,” he said. “In turn, the particular piece of data would be good for retraining the model with proper labels.”
5. MLops organizations want more flexibility.
According to Flynn, MLops organizations should leverage their models to not only make better decisions, but also to optimize these models for different hardware profiles — for example, using technology like the Apache TVM (Tensor Virtual Machine) to compile models to run more efficiently on different cloud providers and across devices with varying hardware (CPU, GPU and or FPGAs). One SAS customer — Georgia-Pacific, an American pulp and paper company — uses edge computing at many of their remote manufacturing facilities where high-speed connectivity often isn’t reliable or cost-effective.
“This flexibility gives MLops teams agility to support a wide variety of use cases, enabling them to bring processing to their data on a growing pool of devices,” he said. “While the range of devices are vast, they often come with resource limitations that could constrain model deployment. This is where model compression comes into play. Model compression reduces the footprint of the model and enables it to run on more compact devices (like an edge device) while improving the model’s computational performance.”
VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.