Skip to content

评测MInimax2.5在数据集terminal_bench_v2上的精度,结果得分只有1.16,官网得分51.7 #1387

Description

@wenruihua

评测命令:
evalscope eval
--model /models/MiniMax-M2.5
--api-url http://0.0.0.0:8000/v1/chat/completions
--api-key EMPTY
--eval-type openai_api
--eval-batch-size 32
--datasets terminal_bench_v2
--work-dir ./outputs/MiniMax-M2.5
--dataset-dir ./terminal-bench-2
--ignore-errors
--generation-config '{
"timeout": 36000,
"stream": true
}' 2>&1 | tee H20-terminal_bench_v2-MiniMax-M2.5.log

Image

Mnimax2.5官网得分:
https://www.modelscope.cn/models/MiniMax/MiniMax-M2.5/summary

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions