SiMa.ai’s Palette software addresses ML developer’s steep learning curve by avoiding the arcane practice of embedded programming. Palette software is a unified suite of tools, functioning much like an ML developer’s familiar cloud equivalent environments, with pushbutton software commands to create, build and deploy on multiple devices. Palette can manage all dependencies and configuration issues within a container environment while securely communicating to the edge devices. This approach still enables embedded programmers flexibility to perform low level optimization of the code. The reaction from our ML developers was “This is exactly what I am looking for!”

Palette is the delivery mechanism for Any, 10x, and Pushbutton.

Any:
Our ML compiler front end leverages open source Tensor Virtual Machine (TVM) framework, and thus supports the industry’s widest range of ML models and ML frameworks for computer vision.

10x:
Our compiler and patented static scheduling approach eliminates stalls, minimizes data movement, caching and improves utilization of our ultra dense machine learning tiled architecture. This combination produces not only a high TOPS/watt rating, our FPS/watt efficiency is 10x times better than competing solutions that often resort to hand coded solutions.

Pushbutton:
We designed our innovative software front-end to automatically partition and schedule your entire application across all of the MLSoC™ compute subsystems. We created a suite of specialized and generalized optimization and scheduling algorithms for our back-end compiler. These algorithms automatically convert your ML network into highly optimized assembly code that runs on the Machine Learning Accelerator (MLA). No manual intervention needed for improved performance.

Palette Software Functional Description

Palette Software Platform provides an integrated development environment for full stack ML application development on a host that can be easily cross-compiled to the MLSoC target silicon, eliminating the need for ML developers to do their algorithm porting on the embedded platform. This cross compilation frees the developer to utilize the desktop as a convenient development platform, contained in a Docker hosted image that contains all of the tools in a single package for full stack ML development. A Push-button build enables the cross compilation to create binary images for the heterogeneous target processors contained in the MLSoC silicon. Push-button using the device manager CLI deploys these images to the device, where they are unpacked, verified and initiated to execute the resulting builds. Device manager commands also manage and control the debugging and logging of events on the MLSoC for real-time monitoring by the host development platform. The deployment capabilities can support a large number of devices simultaneously, creating a deployment capability that can extend a developers MLOps environment to deploy, execute and gather statistics back from the edge device(s). To understand the flow for creating ML applications using Palette, we have a simplified flow that describes the major components we utilize to create, build and deploy an ML application on the MLSoC silicon platform.

Develop an ML Model

ML Model Developer incorporates a parser, quantizer and multi-mode compiler to generate executable code for the Machine Learning Architecture (MLA) . The parser, based on open source TVM, can receive neural networks defined in a wide variety of NN frameworks, providing the capability to support Any ML network. The ML Model development tool performs graph transformations to produce a network graph used for quantization and auto-partitioning. The auto-partitioning identifies through quantization those layers that will be targeted on for computation and those layers that are targeted for execution on a CPU core. The resulting graph network with quantization defined is then compiled with an advanced SiMa.ai proprietary compiler for memory allocation, code generation and scheduling, producing an executable for use with the MLSoC. Those layers auto-partitioned to CPU are compiled using TVM’s ARM compiler. A Json file is generated specifying the sequence of the MLA and ARM code execution to compute the network.

Develop an ML Enabled Computer Vision Pipeline

The second major component is the computer vision pipeline creation tools, to incorporate the user’s compiled ML model(s) from the ML Model Developer, but leveraging the SiMa.ai provide example pipelines, library plug-ins that define the pre and post processing functions that configure the data as well as SiMa.ai example ML models. Using a simple Json file with a sequence of commands or editing a SiMa.ai example Json file, the use defines the input data streams from PCIe, Ethernet or other peripheral, the computer vision pre-processing functions, ML model, the post processing and analytic application software, to create a gstreamer pipeline. Each pipeline element can be built with functional parameters for each plug-in defined. The developer can choose to utilize an existing pipeline from SiMa’s library to modify that pipeline and/or it’s parameters to deploy and test on the MLSoC platform. The Palette software then builds executable images using auto code generation tools for each of the embedded video and application processors contained in the MLSoC for deployment to the silicon for evaluation and test. This process can be quickly iterated to modify the pipeline, it’s components or to tune the pipeline and it’s parameters to achieve the desired system requirements.

Deploy and run an ML Model on the MLSoC device

The third major component is the deployment and device management tool. Palette provides a deployment command line capability to connect to the development board environment, configure and update the development board and download the ML application pipeline executables to the board. Utilizing a secure link from the host development to the targeted MLSoC device(s), users can issue commands and scripts to the device manager that can download, unpackage and install the application pipelines, then execute, stop and update the execution pipeline parameters. Additional command scripts enable the user to debug the software execution on the device and stream to the host platform logs of the MLSoC code execution. The secure connection is utilized to monitor the execution, extract metrics and can provide connectivity to a production host MLOps server/cloud solution that can manage the edge MLSoC device(s).

Palette™ Software Product Brief

To receive more information on Palette Software please fill out the form and our SiMa.ai team will get back to you

FIRST NAME *
Fill out this field
LAST NAME *
Fill out this field
EMAIL ADDRESS *
Please enter a valid email address.
COMPANY *
Fill out this field
APPLICATION/INDUSTRY
Select an option
MESSAGE
Fill out this field