7 Star 44 Fork 13

PaddlePaddle / PaddleSlim

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

PaddleSlim

PaddleSlim是一个专注于深度学习模型压缩的工具库,提供低比特量化、知识蒸馏、稀疏化和模型结构搜索等模型压缩策略,帮助开发者快速实现模型的小型化。

产品动态

  • 🔥 2022.01.18: 发布YOLOv8自动化压缩示例,量化预测加速2.5倍。

  • 【直播分享】2022-12-13 20:30 《自动化压缩技术详解及ViT模型实战》,微信扫码报名

2022.08.16:自动化压缩功能升级
模型 Base mAPval
0.5:0.95
ACT量化mAPval
0.5:0.95
模型体积压缩比 预测时延FP32
预测时延INT8
预测加速比
PPYOLOE-s 43.1 42.6 3.9倍 6.51ms 2.12ms 3.1倍
YOLOv5s 37.4 36.9 3.8倍 5.95ms 1.87ms 3.2倍
YOLOv6s 42.4 41.3 3.9倍 9.06ms 1.83ms 5.0倍
YOLOv7 51.1 50.9 3.9倍 26.84ms 4.55ms 5.9倍
YOLOv7-Tiny 37.3 37.0 3.9倍 5.06ms 1.68ms 3.0倍
历史更新
  • 2022.07.01: 发布v2.3.0版本

    • 发布自动化压缩功能
      • 支持代码无感知压缩:开发者只需提供推理模型文件和数据,既可进行离线量化(PTQ)、量化训练(QAT)、稀疏训练等压缩任务。
      • 支持自动策略选择,根据任务特点和部署环境特性:自动搜索合适的离线量化方法,自动搜索最佳的压缩策略组合方式。
      • 发布自然语言处理图像语义分割图像目标检测三个方向的自动化压缩示例。
      • 发布X2Paddle模型自动化压缩方案:YOLOv5YOLOv6YOLOv7HuggingFaceMobileNet
    • 升级量化功能
      • 统一量化模型格式;离线量化支持while op;修复BERT大模型量化训练过慢的问题。
      • 新增7种离线量化方法, 包括HIST, AVG, EMD, Bias Correction, AdaRound等。
    • 支持半结构化稀疏训练
    • 新增延时预估工具
      • 支持对稀疏化模型、低比特量化模型的性能预估;支持预估指定模型在特定部署环境下 (ARM CPU + Paddle Lite) 的推理性能;提供 SD625、SD710、RK3288 芯片 + Paddle Lite 的预估接口。
      • 提供部署环境自动扩展工具,可以自动增加在更多 ARM CPU 设备上的预估工具。
  • 2021.11.15: 发布v2.2.0版本

    • 支持动态图离线量化功能.
  • 2021.5.20: 发布V2.1.0版本

    • 扩展离线量化方法
    • 新增非结构化稀疏
    • 增强剪枝功能
    • 修复OFA功能若干bug

更多信息请参考:release note

基础压缩功能概览

PaddleSlim支持以下功能,也支持自定义量化、裁剪等功能。

Quantization Pruning NAS Distilling

注:

  • *表示仅支持静态图,**表示仅支持动态图
  • 敏感度裁剪指的是通过各个层的敏感度分析来确定各个卷积层的剪裁率,需要和其他裁剪方法配合使用。

PaddleSlim在典型视觉和自然语言处理任务上做了模型压缩,并且测试了Nvidia GPU、ARM等设备上的加速情况,这里展示部分模型的压缩效果,详细方案可以参考下面CV和NLP模型压缩方案:


表1: 部分场景模型压缩加速情况

注意事项
  • YOLOv3: 在移动端SD855上加速3.55倍。
  • PP-OCR: 体积由8.9M减少到2.9M, 在SD855上加速1.27倍。
  • BERT: 模型参数由110M减少到80M,精度提升的情况下,Tesla T4 GPU FP16计算加速1.47倍。

不同压缩方法效果

自动压缩效果

image
表3: 自动压缩效果

离线量化效果对比

image
表2: 多种离线量化方法效果对比

安装

安装发布版本:

pip install paddleslim

安装develop版本:

git clone https://github.com/PaddlePaddle/PaddleSlim.git & cd PaddleSlim
python setup.py install
  • 验证安装:安装完成后您可以使用 python 或 python3 进入 python 解释器,输入import paddleslim, 没有报错则说明安装成功。
  • 版本对齐:
PaddleSlim PaddlePaddle PaddleLite
2.0.0 2.0 2.8
2.1.0 2.1.0 2.8
2.1.1 2.1.1 >=2.8
2.3.0 2.3.0 >=2.11
2.4.0 2.4.0 >=2.11
develop develop >=2.11

文档教程

模型压缩技术

快速开始

更多教程

进阶教程详细介绍了每一步的流程,帮助您把相应方法迁移到您自己的模型上。

推理部署

CV模型压缩

多场景效果展示

本系列教程均基于Paddle官方的模型套件中模型进行压缩,若您不是模型套件用户,更推荐使用快速教程和进阶教程。

NLP模型压缩

API文档

FAQ

1. 量化训练或者离线量化后的模型体积为什么没有变小?

答:这是因为量化后保存的参数是虽然是int8范围,但是类型是float。这是因为Paddle训练前向默认的Kernel不支持INT8 Kernel实现,只有Paddle Inference TensorRT的推理才支持量化推理加速。为了方便量化后验证量化精度,使用Paddle训练前向能加载此模型,默认保存的Float32类型权重,体积没有发生变换。

2. macOS + Python3.9环境或者Windows环境下, 安装出错, "command 'swig' failed"

答: 请参考https://github.com/PaddlePaddle/PaddleSlim/issues/1258

许可证书

本项目的发布受Apache 2.0 license许可认证。

贡献代码

我们非常欢迎你可以为PaddleSlim提供代码,也十分感谢你的反馈。

技术交流

  • 如果你发现任何PaddleSlim存在的问题或者是建议, 欢迎通过GitHub Issues给我们提issues。

  • 欢迎加入PaddleSlim 微信技术交流群

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

PaddleSlim是一个模型压缩工具库,包含模型剪裁、定点量化、知识蒸馏、超参搜索和模型结构搜索等一系列模型压缩策略 展开 收起
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Python
1
https://gitee.com/paddlepaddle/PaddleSlim.git
git@gitee.com:paddlepaddle/PaddleSlim.git
paddlepaddle
PaddleSlim
PaddleSlim
develop

搜索帮助