1.1K Star 6K Fork 4.8K

OpenHarmony / docs

 / 详情

[Feature Request] Add Light-weight AI framework to OHOS

已完成
需求
创建于  
2020-10-11 14:43

[Feature Request]

Feature: Light weight AI Sub-System
We would like Harmony OS to support the Light weight AI Sub-System: A robust framework to support AI application in OHOS. The framework which can cater to different application / use case requirements. The Light weight AI fits in the OHOS as an optional module which can be used on High capacity devices or light weight devices.

Light Weight AI subsystem positioning

Work in progress
Need PMC approval: Light weight AI run-time interfaces kit
Need PMC approval: Light weight AI run-time framework
Need PMC approval: Third party dlpack used in Light weight AI run-time framework
PR sample app: Sample Applications using Light weight AI run-time

Motivation

  • Low Energy Consumption
    -- Micro-controllers consume very little energy and are efficient
    -- Micro-controllers are usually not wired and typically rely on batteries or energy harvesting.
  • Flexibility
    -- We can add AI ability to various devices without network connectivity, which are normally constrained by bandwidth, power and / or latency.
    -- Micro-controllers are very common and the possibilities are endless when we bring AI to micro-controllers.
  • Cost
    -- Hardware limitation and design specific to program limitation
    -- Little external third-party configuration
  • Privacy
    -- Users do not have to worry about privacy as no data is required to leave the device and the computation will happen on the device itself.

Background
Efficient RAM Reduction :
Our Solution is Customizable, based on the memory requirement can choose between:

  • Shared params solutions
  • On demand memory allocation

Efficient ROM reduction :
Depending on the need to the application, users can choose the optimization level which meets their requirements.

Proposed interface
Run-time
/**
* @yutong zhang Constructor, initializes the runtime and allocate necessary memory.
*
* @chancelai graph_byte Pointer to the graph data of the model.
* @chancelai param_byte Pointer to the param data of the model.
* @chancelai kernel_funcs The list of kernel functions used for this model.
*
/
Runtime(const char
graph_byte, const char* params_byte, std::vector* kernel_funcs);

Set-Input
/**
* @yutong zhang Function to set the input tensor to the model.
*
* @chancelai name The name of the input tensor for which input need to set.
* @chancelai tensor The input tensor value to be set for execution of model.
*
/
void SetInput(const char
name, DLTensor* tensor);

Run
/**
* @yutong zhang Function to execute the model.
*
*/
void Run();

Get-Result
/**
* @yutong zhang Return tensor for given output index after execution.
*
* @chancelai index The output index.
* @chancelai tensor The output pointer where the data is copied.
*
/
void GetOutput(int32_t index, DLTensor
tensor);

Sample Application
Sample application has been implemented to use the above service. The Sample application is maintained in sample camera apps.
The Sample applications generated in the bin can be invoked to see the results

Sample Results
Sample Application list and results

Future Enhancements
Currently the Light-weight AI framework is already work in progress as per the above mentioned PR’s and is waiting for approval to create repo.
Apart from the current optimized framework capabilities, the following enhancements are being planned to be supported.

  • Feature Enhancements
    -- Memory planner (30-Oct)
    -- CMSIS-NN (30-Oct)
    -- Camera integrated demo (30-Oct)
    -- TVM Model Conversion (30-Nov)
    -- Enhanced Tiny kernels (30-Nov)
    -- 4-bit quantized kernels (30-Dec)
    -- AMP (30-Dec)

  • Model Zoo
    -- Person detection
    -- Image recognition
    -- Speech detection
    -- MNIST
    -- Magic wand

  • Hardware support
    -- Currently the Hi3516 and Hi3518 boards are being supported
    -- Plan to support additional hardware in future (Q1-2021)
    -- NXP IMXULL
    -- Audrino
    -- Sparkfun

Roadmap summary

输入图片说明

重现步骤

报错信息

评论 (1)

VinodSKumar 创建了需求
VinodSKumar 关联仓库设置为OpenHarmony/docs
VinodSKumar 修改了描述
NEEN 负责人设置为jinguang
NEEN 添加了
 
feature
标签
展开全部操作日志

Close the issue temporarily. it will be open the issue again or create a new issue after the feature having some progress

jinguang 任务状态待办的 修改为已完成

登录 后才可以发表评论

状态
负责人
项目
里程碑
Pull Requests
关联的 Pull Requests 被合并后可能会关闭此 issue
分支
开始日期   -   截止日期
-
置顶选项
优先级
预计工期 (小时)
参与者(2)
5126245 dongjinguang 1625667980
其他
1
https://gitee.com/openharmony/docs.git
git@gitee.com:openharmony/docs.git
openharmony
docs
docs

搜索帮助

14c37bed 8189591 565d56ea 8189591