Bug Fixes:

1. Fix compatibility issues with trust_remote_code across multiple scenarios.

Full Changelog: v1.37.0...v1.37.1

中文版

特性

1. 为数据集新增基于 split 的过滤功能，支持用户按需加载指定的数据 split，并新增相应的参数校验逻辑。（#1704）
2. 增强数据集下载日志，新增调试级别日志以记录请求详情与响应指标；同时修复了数据集预览时的参数传递错误问题。（#/1700）
3. 将 Ascend NPU 的 Dockerfile 更新为基于 CANN 基础镜像构建，不再依赖 vllm-ascend 镜像，便于后续独立管理 CANN 版本与 vllm-ascend 版本。（#1687）
4. 新增适用于 ms-swift 4.1.x 的 Metax Dockerfile。（#1689）

修复

1. 修复了补丁对 kernels <= 0.12 版本的兼容性问题。（#1708）
2. 为 pipeline 和 build_trainer 函数引入 trust_remote_code 参数，要求用户显式授权后方可执行远程代码或插件，提升使用安全性。（#1703）

English Version

Features

1. Added split-based filtering for datasets, enabling users to load specific data splits on demand, along with corresponding parameter validation logic. (#1704)
2. Enhanced dataset download logging by adding debug-level logs to capture request details and response metrics; also fixed an incorrect parameter passing issue during dataset preview. (#1700)
3. Updated the Ascend NPU Dockerfile to build from the CANN base image instead of the vllm-ascend image, allowing independent version management of CANN and vllm-ascend going forward. (#1687)
4. Added a MetaX Dockerfile compatible with ms-swift 4.1.x. (#1689)

Bug Fixes

1. Fixed a compatibility issue with patches for kernels <= 0.12. (#1708)
2. Introduced the trust_remote_code parameter for the pipeline and build_trainer functions, requiring users to explicitly grant authorization before executing remote code or plugins, thereby improving usage security. (#1703)

What's Changed

• update ascend dockerfile by @addsubmuldiv in #1687
• [Fix] Fix msdatasets split issue by @wangxingjun778 in #1704
• fix patch does not support kernels<=0.12 by @tastelikefeet in #1708
• add metax dockerfile and its requirements for ms-swift 4.1.x by @WendaDeng in #1689
• [Fix] Fix dataset preview args, private streaming auth, and download retries by @wangxingjun778 in #1700
• [Fix] fix upload cache ignore by @wangxingjun778 in #1709
• Fix plugin rce by @tastelikefeet in #1703
• fix plugin by @tastelikefeet in #1712
• fix trainer pass trust_remote_code to model by @tastelikefeet in #1713
• [Fix] fix invalid commit by @wangxingjun778 in #1715
• [Fix] Fix split detection by @wangxingjun778 in #1714
• update npu dockerfile by @xvxuopop in #1716
• Fix pipeline parameter by @tastelikefeet in #1717

New Contributors

• @WendaDeng made their first contribution in #1689
• @xvxuopop made their first contribution in #1716

Full Changelog: v1.36.3...v1.37.0

Release Notes

中文版

修复 (Fix)

• CLI 上传报告：修复上传计数逻辑，优化报告生成流程 (#1694)
• CLI 帮助文档：统一 --endpoint 参数描述，移除冗余站点标签 (#1696)
• 创建模型仓库：为 create 命令添加 endpoint 支持，新增 --exist_ok 标志；移除冗余的文件大小检查逻辑 (#1699)

特性 (Feature)

• CLI endpoint 重构：统一 login/create/upload/download 命令的 endpoint 解析逻辑，新增 resolve_endpoint 工具函数，支持 URL 自动补全与环境变量回退 (#1695)
• Kernels 下载支持：通过 monkey-patching 机制，支持从 ModelScope 下载 kernels 库，兼容 HuggingFace 接口 (#1697)

English Version

Fixes

• Upload Report: Fixed upload count logic and optimized report generation flow (#1694)
• CLI Help Docs: Unified --endpoint parameter description across commands; removed redundant site labels (#1696)
• Model Repo Creation: Added endpoint support and --exist_ok flag to create command; removed redundant non-LFS file size checks (#1699)

Features

• CLI Endpoint Refactor: Standardized endpoint resolution for login/create/upload/download commands with new resolve_endpoint utility; supports URL auto-completion and env var fallbacks (#1695)
• Kernels Download Support: Enabled downloading kernels via ModelScope using monkey-patching for HuggingFace API compatibility (#1697)

What's Changed

• [Fix] Upload report by @wangxingjun778 in #1694
• [Feat & Fix] Refactor endpoint arg for CLI by @wangxingjun778 in #1695
• [Fix] Update helper desc by @wangxingjun778 in #1696
• Support kernels downloading by @tastelikefeet in #1697
• [Fix] Add endpoint for creating model repo by @wangxingjun778 in #1699

Full Changelog: v1.36.2...v1.36.3

Release Notes (English)

Authentication & API

• [Fix] OpenAPI Auth Support (#1686)
  • Centralized bearer token handling via _build_bearer_headers in HubApi
  • Unified response parsing with _parse_openapi_response for maintainability
  • Enhanced robustness: null-safe URL parsing, JSON decoding error handling, and response type validation

Upload Module

• [Refactor&Fix] Upload Folder Module (#1690)
  • Introduced UploadTracker for resumable uploads and progress tracking
  • Added adaptive batching and "ReAct" progressive retry strategy
  • Application-level retries for blob uploads with improved error classification
  • File size integrity checks and UTF-8 encoding for cross-platform compatibility
  • Fixed retry backoff calculation and upload hang issues

版本更新说明（中文）

认证与API

• [修复] OpenAPI 认证支持 (#1686)
  • 在 HubApi 基类中统一实现 bearer token 认证逻辑
  • 统一响应解析方法，提升代码可维护性
  • 增强健壮性：空值安全的URL解析、JSON解码异常兼容处理、响应类型校验

上传模块

• [重构&修复] 上传文件夹模块优化 (#1690)
  • 新增 UploadTracker 管理器，支持断点续传与进度追踪
  • 支持自适应批处理与"ReAct"渐进式重试兜底策略
  • 为blob上传增加应用层重试机制，优化错误分类
  • 新增文件大小校验，强制UTF-8编码确保跨平台兼容
  • 修复重试退避时间计算逻辑及上传卡死问题

What's Changed

• [Feature] Add openapi auth support by @wangxingjun778 in #1686
• [Refactor&Fix] Upload folder module refactor by @wangxingjun778 in #1690

Full Changelog: v1.36.1...v1.36.2

📦 Release Notes (English)

ModelScope Download Module Refactor

• Architecture Upgrade: Rebuilt download module with producer-consumer pipeline for significantly improved throughput and responsiveness.
• Memory Optimization: Added server-side prefix filtering + paginated listing to reduce memory footprint for large-scale snapshots.
• Reliability Enhancements:
  • Retry mechanism for hash validation failures
  • Larger buffer size for hash computation
  • Fault-tolerant reporting with strict exception handling for pagination/partial-download errors
• Error Handling: Introduced FileDownloadError for granular single-file download failure tracking.
• Config Updates:
  • Default dataset page size set to 200
  • Updated build dependencies (pip, setuptools)

📦 发布说明（中文）

ModelScope 下载模块重构

• 架构升级：采用生产者 - 消费者流水线重构下载模块，显著提升下载吞吐与响应速度。
• 内存优化：支持服务端 prefix 过滤 + 分页列表拉取，大幅降低大规模快照下载时的内存占用。
• 可靠性增强：
  • 哈希校验失败自动重试
  • 增大哈希计算缓冲区
  • 容错上报机制，确保分页错误/部分下载失败时严格抛出异常，避免快照状态不一致
• 错误处理：新增 FileDownloadError 异常类，支持单文件下载失败的精细化追踪。
• 配置更新：
  • 默认数据集分页大小调整为 200
  • 更新构建依赖（pip, setuptools）

💡 Migration Tip: This is a backward-compatible refactor. No user code changes required, but users will notice faster and more stable downloads, especially for large-scale datasets.

What's Changed

• [Refactor] Refactor the modelscope download module by @wangxingjun778 in #1683

Full Changelog: v1.36.0...v1.36.1

中文版

✨ 新特性

• 文件夹上传优化：新增生产者-消费者管道机制，并发上传与进度追踪，大幅提升超大型数据集上传效率（#1671）

问题修复

• CSV 预览加载：修复流式加载 CSV 时的兼容性问题，支持 engine=python 并修复字段长度溢出错误（#1665）
• Swift 4.0 兼容：适配 ms-swift 4.0 包结构调整，更新模块导入路径，保持向后兼容（#1680）

部署

• GPG 密钥修复：更新 Dockerfile 中 GPG KEY 配置，支持多密钥验证（#1678）
• Ascend 镜像优化：切换至预构建 vllm-ascend 基础镜像，精简构建步骤，加快镜像生成速度（#1679）

🇬🇧 English Version

✨ Features

• Folder Upload Enhancement: Introduced producer-consumer pipeline for folder uploads with concurrent management, and progress tracking, significantly improving large-scale upload efficiency (#1671)

Bug Fixes

• CSV Preview Loading: Fixed streaming CSV loading issues, added engine=python support and resolved field size OverflowError (#1665)
• Swift 4.0 Compatibility: Updated import paths to align with ms-swift 4.0 package restructuring while maintaining backward compatibility (#1680)

Deployment

• GPG KEY Fix: Updated Dockerfile GPG KEY configuration to support multi-key verification (#1678)
• Ascend Dockerfile Optimization: Migrated to prebuilt vllm-ascend base image and streamlined build process for faster image generation (#1679)

What's Changed

• [Feature] Add producer-consumer pipeline for uploading folder by @wangxingjun778 in #1671
• [Fix] Fix preview csv stream loading by @wangxingjun778 in #1665
• Fix GPG KEY by @wangxingjun778 in #1678
• [bugfix] compat swift 4.0 by @Jintao-Huang in #1680
• update ascend dockerfile by @addsubmuldiv in #1679

New Contributors

• @addsubmuldiv made their first contribution in #1679

Full Changelog: v1.35.4...v1.36.0

📝 Release Notes / 发布说明

🇨🇳 Chinese (中文)

新功能 (Features)

• 支持 Collections 功能：引入了对模型/数据集集合（Collections）的支持，方便用户更好地组织和管理资源。 (#1656)

修复与优化 (Fixes & Improvements)

• 提升下载 CLI 健壮性：增强了 ModelScope 命令行下载工具的稳定性，改善用户体验。 (#1662)
• 修复下载 Token 认证问题：解决了在下载过程中可能出现的 Token 认证失败问题，确保权限验证正常。 (#1674)
• 修复 Skill 目录拼接错误：修正了 Skill 模块中目录路径拼接的逻辑错误。 (#1657)
• 优化预览加载与 HF 数据集工具：修复了预览加载问题，并对 Hugging Face 数据集相关工具类进行了重构，提升代码质量和可维护性。 (#1654)

🇺🇸 English

Features

• Collections Support: Added support for Collections, allowing users to better organize and manage models and datasets. (#1656)

Fixes & Improvements

• Enhanced Download CLI Robustness: Improved the stability and robustness of the ModelScope command-line download tool. (#1662)
• Fixed Token Authentication for Downloads: Resolved issues related to token authentication during the download process to ensure secure and successful access. (#1674)
• Fixed Skill Directory Concatenation: Corrected a bug in the directory path concatenation logic within the Skill module. (#1657)
• Preview Loading Fix & HF Utils Refactor: Fixed issues with preview loading and refactored Hugging Face dataset utilities for better code structure and maintainability. (#1654)

What's Changed

• merge 1.35.1 to master by @tastelikefeet in #1653
• Feat/collections by @wangxingjun778 in #1656
• [Fix] Skill DIR concat by @wangxingjun778 in #1657
• [Fix | Refactor] Fix preview loading and refactor hf datasets utils by @wangxingjun778 in #1654
• [Fix] Enhance the robustness of ModelScope download cli by @wangxingjun778 in #1662
• [Fix] fix token auth for downloading by @wangxingjun778 in #1674

Full Changelog: v1.35.3...v1.35.4

中文版本

新特性

1. 提升与 datasets 库的兼容性，适配了 4.0 及以上版本的变更，修复了因文件大小获取错误导致的数据预览加载问题，同时重构了代码结构并优化了 API 缓存机制以提升性能。

English Version

New Feature

1. Improves compatibility with datasets library 4.0+, fixes dataset preview loading issues caused by incorrect file size reporting, and refactors the codebase with optimized API caching for better performance.

Full Changelog: v1.35.2...v1.35.3

中文版本

新特性

1. 支持skills的下载

# Skills will be default placed to `~/.agents/skills`
# Download a collection of skills
modelscope download --collection MiniMax/MiniMax-Office-skills [--token xxx] [--local_dir ./temp]
# Download a batch skills of a single skill
modelscope skills add @MiniMax-AI/minimax-pdf @xxx/xxx [--token xxx]

English Version

New Feature

1. Support the downloading of skills

# Skills will be default placed to `~/.agents/skills`
# Download a collection of skills
modelscope download --collection MiniMax/MiniMax-Office-skills [--token xxx] [--local_dir ./temp]
# Download a batch skills of a single skill
modelscope skills add @MiniMax-AI/minimax-pdf @xxx/xxx [--token xxx]

Full Changelog: v1.35.1...v1.35.2

中文版本

Bug修复

1. 修复了oss2被全局依赖的问题
2. 修复了packaging没有被加入依赖的问题

English Version

BugFix

1. Fix an issue that oss2 was required by global import.
2. Fix an issue that packaging was not included in the dependencies.

What's Changed

• [Fix] lazy import oss2 by @wangxingjun778 in #1649
• fix packaging dependency by @tastelikefeet in #1651

Full Changelog: v1.35.0...v1.35.1