1 Star 0 Fork 102

json / iFlytekSpark-13B

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

iFlytekSpark-13B

目录

模型介绍

讯飞星火开源-13B(iFlytekSpark-13B)拥有130亿参数,新一代认知大模型,一经发布,众多科研院所和高校便期待科大讯飞能够开源。 为了让大家使用的更加方便,科大讯飞增加了更多的数据,并针对工具链进行了优化。此次正式开源拥有130亿参数的iFlytekSpark-13B模型(讯飞星火开源-13B),也是首个基于全国产化算力平台“飞星一号”的大模型,正式开源!

iFlytekSpark-13B不仅具备通用任务处理能力如聊天、问答、文本提取和分类等,还具备数据分析和代码生成等生产力功能。我们特别在学习辅助、数学、推理等领域进行了深度优化,大幅提升模型的实用性和易用性。详细的评测结果见下面评测部分。

本次开源,既包含基础模型iFlytekSpark-13B-base、精调模型iFlytekSpark-13B-chat,也开源了微调工具iFlytekSpark-13B-Lora、人设定制工具iFlytekSpark-13B-Charater,让企业和学术研究可以基于这些全栈自主创新的星火优化套件方便地训练自己的专用大模型。

星火开源-13B在多项知名公开评测任务中名列前茅,在文本生成、语言理解、文本改写、行业问答、机器翻译等企业典型场景中,通过对学习辅助、语言理解等领域的深入研究和优化,大幅提升了其实用性,在处理复杂的自然语言任务时更加得心应手,确保了其在面对多样化和专业化的应用场景时能够保持高效和准确,效果显著优于其他同等尺寸的开源模型。

这对于追求高性能而对成本敏感的企业来说,无疑是一个巨大的吸引力,也为各行各业的企业提供了一种性价比高的解决方案。

模型结构

在iFlytekSpark-13B中,我们使用Rotary Embedding作为位置编码方法,GELU作为激活函数,其中layer_num为40,head_num为40,hidden_size为5120,ffn_hidden_size为28672。

效果评测

我们在八个具有挑战性的中英文测试集上对模型进行性能评估。其中chat模型采用0-shot进行测试,base模型在C-EVAL,MMLU,CMMLU,FinanceIQ测试集上采用5-shot进行测试,其余测试集采用0-shot进行测试。

数据集介绍

  • C-EVAL:C-Eval 是一个全面的中文基础模型评估套件,涵盖了52个不同的学科和四个难度级别,验证集包括1346个选择题,测试集包含12342个选择题。本项目采用C-Eval验证集进行测试。
  • MMLU:MMLU 是一个庞大的多任务数据集,由各种学科的多项选择题组成。其中包括57个任务,涵盖了人文学科、社会科学、自然科学和其他对某些人学习很重要的领域。
  • CMMLU:CMMLU 是一个综合性的中文评估基准,涵盖了从基础学科到高级专业水平的67个主题。涵盖了自然科学、人文科学和社会科学等领域。
  • AGIEVAL:AGIEval 是一个专门为评估基础模型在以人类为中心的标准化考试(如大学入学考试、法学院入学考试、数学竞赛和律师资格考试)的语境中而设计的基准测试。
  • ARC:包含了ARC-E和ARC-C,它们分别是ARC数据集中的简单集和挑战集,分别有5197 和2590 个问题。这些问题是仅文本的英语语言考试问题,跨越了多个年级水平。
  • GaoKao:GaoKao收集了从 2010 年到 2022 年的高考试题,包括 1781 道客观题和 1030 道主观题。本项目报告结果为GaoKao中客观题结果。
  • FinanceIQ:FinanceIQ 是一个专注于金融领域的中文评估数据集,涵盖了10个金融大类及36个金融小类,总计7173个单项选择题。

测评结果

C_EVAL MMLU CMMLU AGIEVAL ARC_E ARC_C GaoKao FinanceIQ 平均
iFlytekSpark-13B-base 70.88 58.76 70.01 50.44 84.78 71.16 56.42 60.21 65.33
iFlytekSpark-13B-chat 82.54 63.02 75.69 56.96 89.47 77.34 67.49 65.48 72.25

使用教程

NPU

华为昇腾NPU上使用教程

GPU

GPU上使用教程

工具链

自我认知

自我认知工具链使用教程

声明、联系我们

协议

请您知悉,无论您是否已实际阅读星火开源-13B大模型许可协议,当您通过部署及使用该模型服务即表示确认同意本协议或实际使用、复制、分发、修改本协议中的讯飞星火认知大模型-13B模型时,均表示您与科大讯飞股份有限公司(以下称“许可方”)已就本协议达成一致,本协议具有合同效力。如果您不同意本协议的任一内容,或者无法准确理解许可方对本协议条款的解释,请停止使用本服务。否则,即表示您已接受本协议所述的所有条款及其适用条件,同意受本协议约束。

联系我们

如果你想给我们的研发团队和产品团队留言,可以通过邮件(iflytekspark@iflymail.com.cn)联系我们。
同时,也欢迎你加入我们的开源社群。
星火开源社群

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

讯飞星火开源-13B(iFlytekSpark-13B)拥有130亿参数,新一代认知大模型,一经发布,众多科研院所和高校便期待科大讯飞能够开源。 为了让大家使用的更加方便,科大讯飞增加了更多的数据,并针对工具链进行了优化。此次正式开源拥有130亿参数的iFlytekSpark-13B模型(讯飞星火开源-13B),也是首个基于全国产化算力平台“飞星一号”的大模型,正式开源! 展开 收起
Python
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Python
1
https://gitee.com/json_gitee/iFlytekSpark-13B.git
git@gitee.com:json_gitee/iFlytekSpark-13B.git
json_gitee
iFlytekSpark-13B
iFlytekSpark-13B
master

搜索帮助