5 min Analytics

Down the embedded AI model pipeline

Down the embedded AI model pipeline

Data-centric AI development platform company Dataloop has dialled into Qualcomm with the aim of fuelling AI model development for specifically for mobile, automotive, IoT and computing devices powered by Snapdragon system-on-a-chip semiconductor products. With the need to create a smaller footprint, more lightweight and potentially air-gapped (or at least occasionally disconnected) computing models in these deployment scenarios (that still operate securely), the two companies say that the result of their collaboration is an automated ‘AI pipeline’ that runs on the Dataloop platform with application and model development tools, reference architectures and standardisation templates.

Software application developers and AI model data scientists can use the Dataloop AI platform with the Qualcomm AI hub to deploy models across devices powered by Snapdragon platforms. Because processing cycles can be more limited, restricted and/or less manageable in embedded environments, 

What’s inside an AI pipeline?

Tel-Aviv-born (and headquartered) Dataloop is designed to allow AI engineers and developers to streamline the AI lifecycle through an automated pipeline. What’s inside that pipeline? Nir Buschi, co-founder at Dataloop AI says that this includes data curation, labelling, model fine-tuning. Due to the partnership and integration with Qualcomm AI Hub, this same set of pipeline technologies can be optimised, compiled and profiled as a ready-to-deploy model.

According to Qualcomm product man Siddhika Nevrekar, “Qualcomm Technologies is collaborating with Dataloop to streamline on-device AI deployment. With Dataloop’s automated pipelines and robust data management, developers can effortlessly create powerful AI systems and seamlessly deploy them on-device using our Qualcomm AI Hub.” 

Dataloop supports AI teams throughout the entire AI application deployment process, enabling them to consistently build and deploy applications swiftly and accurately. With its data-agnostic approach and support for diverse types of unstructured data, it addresses the needs of industries dealing with complex data workflows and challenges in data quality and AI model training. 

Why is embedded AI tough?

“It will go without saying to experienced embedded engineers, that the decision to commit any code module into an embedded system product must be careful and deliberate. Firmware pushed to products must be solid, and reliable.   Usually products are expected to have a long lifetime between updates, if updates are even feasible at all.  And they are expected to work out of the box,” explains Jeff Sieracki, co-founder and CTO of Reality Analytics (now acquired by Renesas Electronics), a company known for its AI tools that detect real-world events in sensor and signal data.

Testing and validation are widely argued to be better done in sandbox environment. Sieracki says that his firm’s tools, for example provide the user the ability to experiment, retrain and test with as much data as they choose before the classifier ever leaves the safety of the cloud.

“We even support ‘live” testing through cloud APIs, so that the customer can have every confidence they have tested and characterised a classifier or detector module before ever committing the resources to push it to firmware and customer devices,” he added.

Samsara: situational solutions

Overall here, we can remind ourselves that new edge AI capabilities enable real-time processing and data capture at the source, improving decision-making and operational efficiency in some cases, when deployed intelligently. 

“With industries outside of tech like construction or long-haul trucking, companies are often operating in remote environments with low connectivity, where it can take minutes for data to reach the cloud,” commented Evan Welbourne, Samsara’s VP of machine learning engineering. “In situations where every second counts, edge devices can be used to alert workers so that they avoid events like road collisions or avoid falling asleep at the wheel. Leveraging the edge to process data locally, run models and limit what’s actually sent to the cloud not only improves response times and minimises bandwidth constraints, but it can actually save lives.”

“However says Welbourne, edge AI still faces complex challenges — companies want models that are cost-effective without compromising reliability. These devices need to pack enough computational power to handle advanced AI systems, while managing the heat they generate and they must have the energy capacity to avoid constant recharging or replacement. 

“But this also highlights one of the biggest challenges when running AI models on the edge: energy consumption. As AI continues to grow more complex, it’s getting harder for our grid to keep up with the immense power needed to run elaborate AI systems,” said Welbourne. “The technology has huge potential to revolutionise how AI systems operate, especially for enterprise companies. However, it still needs further development before it can really stand out as an alternative to cloud computing.”

Is there an AI lifecycle yet?

It feels like we’re crystallising something of an AI lifecycle in that we’re now talking more directly about so-called “data teams” and the work needed to take models from prototype to full-scale production with robust data management for visualising and searching through unstructured data and also working with an orchestration layer for customizing production AI pipelines, large language models and perhaps even MLOps tooling.

Spoiler alert, that’s exactly how our first company mentioned here Dataloop describes itself. How this market now standardises, template-izes and automates will be one of the key telling factors.

Free image use: Wikipedia Commons