CompoSET — Model reports · AI Benchmark Radar

AI Benchmark Radar / Equality Machine · 核对于 2026-11-02

对应2026年10月30日论文v1及文中数据冻结标识662230d。完整检索成绩、编辑类型分项与生成式模型小样本分别保留。

评测隔离了什么

每道测试包含两张近似图像及各自描述。编辑只改变一个明确因素，场景保持一致；模型对四种图文组合排序，用于检验语义绑定，而非仅识别不同场景。

CompoSET paper v1 — task, tables and limitations · CompoSET author evaluation repository

数据与构建

公开集含80个合成场景、1,776组原图/编辑图，覆盖16类编辑与三种描述长度。编辑涉及身份与外观、数量、空间关系、活动与状态。Claude Sonnet 4.6生成场景与描述规格，Gemini 2.5 Flash Image生成图像，再由作者人工审核；不提供训练集。

CompoSET paper v1 — task, tables and limitations · CompoSET official dataset

如何理解Group准确率

图到文和文到图两个方向均要求正确配对胜过错误配对。Group只有在两个方向同时通过时才计成功，不给部分分。本页保留论文公布的Group百分数；可交换随机分数下的机会水平是16.67%，不能将两个方向当独立事件直接相乘。

CompoSET paper v1 — task, tables and limitations

区分实验配置

表8给出全部22个双编码器的确定性余弦评分，表9按编辑类型细分14个模型。表11–12使用分层抽取的200例生成式模型小样本：开源权重模型采用描述似然，API模型采用温度0的成对选择并交换选项顺序以抵消位置偏差。它们的样本范围与评分方式不同。

CompoSET paper v1 — task, tables and limitations · CompoSET author evaluation repository

结果的适用边界

这是英文描述、静态图像、单一编辑的检索测试。合成图像、单人审核和仅80个基础场景限制了外推；角色交换仅45例。场景内相关性会扩大不确定区间。某一编辑子集的高分不等于整体成绩，生成式小样本也不能与双编码器总体直接排名。

CompoSET paper v1 — task, tables and limitations

查看原始发布

论文直接链接作者评测仓库与Hugging Face数据集。仓库可导出逐样本I2T、T2I和Group值，并说明权重配置。这里展示作者报告的成绩，本站未重新运行模型。

CompoSET paper v1 — task, tables and limitations · CompoSET author evaluation repository · CompoSET official dataset

完整评测：22个双编码器

表8报告整体严格Group百分数；只有这些整体成绩进入本页的标准化主成绩序列。

完整评测：22个双编码器
模型	整体Group（%）	论文95%置信区间
CLIP B/32	4.4	[3.9,5.0]
CLIP B/16 (OpenAI)	4.8	[4.2,5.4]
CLIP B/16 (LAION-400M)	8.4	[7.6,9.1]
CLIP B/16 (LAION-2B)	7.8	[7.1,8.5]
CLIP L/14 (OpenAI)	5.0	[4.5,5.7]
CLIP L/14 (LAION-2B)	8.4	[7.7,9.2]
CLIP H/14 (LAION-2B)	8.4	[7.7,9.1]
SigLIP B/16	10.2	[9.4,11.0]
SigLIP L/16-384	12.7	[11.9,13.7]
SigLIP SO400M/14	13.5	[12.6,14.4]
SigLIP-2 B/16	11.3	[10.5,12.2]
SigLIP-2 L/16-384	13.6	[12.7,14.6]
SigLIP-2 SO400M/14	13.9	[13.0,14.9]
NegCLIP B/32	7.0	[6.4,7.7]
CLoVe B/32	5.5	[5.0,6.2]
DreamLIP B/16 (30M)	13.7	[12.8,14.6]
DFN B/16	8.4	[7.7,9.2]
DFN H/14	10.8	[10.0,11.7]
EVA-02 B/16	7.8	[7.1,8.5]
EVA-02 L/14	8.6	[7.9,9.4]
MetaCLIP-2 H/14	8.9	[8.2,9.7]
Jina-v4	8.5	[7.8,9.3]

完整数据集、确定性余弦评分。这些表内区间按样本行计算；附录C的场景聚类区间更宽。

CompoSET paper v1 — task, tables and limitations

编辑子集：逐模型、逐编辑类型Group

表9包含224个子集测量。每行只针对一种编辑类型，均不代表完整评测成绩。

编辑子集：逐模型、逐编辑类型Group
模型	编辑类型	合并样本数n	Group百分数［95%区间］
CLIP B/32	modify_color	618	20.4 [17.2,23.6]
CLIP B/16 LAION-2B	modify_color	618	29.9 [26.3,33.5]
CLIP H/14 LAION-2B	modify_color	618	28.8 [25.2,32.4]
SigLIP B/16	modify_color	618	37.2 [33.4,41.0]
SigLIP SO400M	modify_color	618	39.5 [35.6,43.3]
SigLIP-2 B/16	modify_color	618	39.2 [35.3,43.0]
SigLIP-2 L/16	modify_color	618	38.5 [34.7,42.3]
SigLIP-2 SO400M	modify_color	618	36.2 [32.5,40.0]
NegCLIP B/32	modify_color	618	30.6 [26.9,34.2]
CLoVe B/32	modify_color	618	27.7 [24.1,31.2]
DreamLIP B/16	modify_color	618	52.4 [48.5,56.4]
DFN H/14	modify_color	618	35.1 [31.3,38.9]
EVA-02 L/14	modify_color	618	26.5 [23.1,30.0]
MetaCLIP-2 H/14	modify_color	618	27.2 [23.7,30.7]
CLIP B/32	modify_pattern	303	6.3 [3.5,9.0]
CLIP B/16 LAION-2B	modify_pattern	303	14.5 [10.6,18.5]
CLIP H/14 LAION-2B	modify_pattern	303	9.6 [6.3,12.9]
SigLIP B/16	modify_pattern	303	20.1 [15.6,24.6]
SigLIP SO400M	modify_pattern	303	21.5 [16.8,26.1]
SigLIP-2 B/16	modify_pattern	303	17.8 [13.5,22.1]
SigLIP-2 L/16	modify_pattern	303	16.8 [12.6,21.0]
SigLIP-2 SO400M	modify_pattern	303	23.8 [19.0,28.6]
NegCLIP B/32	modify_pattern	303	8.6 [5.4,11.7]
CLoVe B/32	modify_pattern	303	8.9 [5.7,12.1]
DreamLIP B/16	modify_pattern	303	29.4 [24.2,34.5]
DFN H/14	modify_pattern	303	9.9 [6.5,13.3]
EVA-02 L/14	modify_pattern	303	14.2 [10.3,18.1]
MetaCLIP-2 H/14	modify_pattern	303	11.9 [8.2,15.5]
CLIP B/32	modify_material	483	1.7 [0.5,2.8]
CLIP B/16 LAION-2B	modify_material	483	6.8 [4.6,9.1]
CLIP H/14 LAION-2B	modify_material	483	8.3 [5.8,10.7]
SigLIP B/16	modify_material	483	11.2 [8.4,14.0]
SigLIP SO400M	modify_material	483	16.4 [13.1,19.7]
SigLIP-2 B/16	modify_material	483	10.8 [8.0,13.5]
SigLIP-2 L/16	modify_material	483	14.7 [11.5,17.9]
SigLIP-2 SO400M	modify_material	483	13.7 [10.6,16.7]
NegCLIP B/32	modify_material	483	4.3 [2.5,6.2]
CLoVe B/32	modify_material	483	3.3 [1.7,4.9]
DreamLIP B/16	modify_material	483	12.2 [9.3,15.1]
DFN H/14	modify_material	483	12.6 [9.7,15.6]
EVA-02 L/14	modify_material	483	8.9 [6.4,11.4]
MetaCLIP-2 H/14	modify_material	483	8.1 [5.6,10.5]
CLIP B/32	modify_shape	309	2.6 [0.8,4.4]
CLIP B/16 LAION-2B	modify_shape	309	8.7 [5.6,11.9]
CLIP H/14 LAION-2B	modify_shape	309	10.7 [7.2,14.1]
SigLIP B/16	modify_shape	309	10.0 [6.7,13.4]
SigLIP SO400M	modify_shape	309	12.3 [8.6,16.0]
SigLIP-2 B/16	modify_shape	309	11.0 [7.5,14.5]
SigLIP-2 L/16	modify_shape	309	14.6 [10.6,18.5]
SigLIP-2 SO400M	modify_shape	309	12.6 [8.9,16.3]
NegCLIP B/32	modify_shape	309	6.8 [4.0,9.6]
CLoVe B/32	modify_shape	309	4.5 [2.2,6.8]
DreamLIP B/16	modify_shape	309	14.6 [10.6,18.5]
DFN H/14	modify_shape	309	9.4 [6.1,12.6]
EVA-02 L/14	modify_shape	309	7.4 [4.5,10.4]
MetaCLIP-2 H/14	modify_shape	309	12.0 [8.4,15.6]
CLIP B/32	modify_transparency	300	0.7 [0.0,1.6]
CLIP B/16 LAION-2B	modify_transparency	300	1.7 [0.2,3.1]
CLIP H/14 LAION-2B	modify_transparency	300	0.7 [0.0,1.6]
SigLIP B/16	modify_transparency	300	1.3 [0.0,2.6]
SigLIP SO400M	modify_transparency	300	1.7 [0.2,3.1]
SigLIP-2 B/16	modify_transparency	300	2.3 [0.6,4.0]
SigLIP-2 L/16	modify_transparency	300	2.3 [0.6,4.0]
SigLIP-2 SO400M	modify_transparency	300	1.0 [0.0,2.1]
NegCLIP B/32	modify_transparency	300	1.0 [0.0,2.1]
CLoVe B/32	modify_transparency	300	0.7 [0.0,1.6]
DreamLIP B/16	modify_transparency	300	1.7 [0.2,3.1]
DFN H/14	modify_transparency	300	1.3 [0.0,2.6]
EVA-02 L/14	modify_transparency	300	0.3 [0.0,1.0]
MetaCLIP-2 H/14	modify_transparency	300	2.3 [0.6,4.0]
CLIP B/32	modify_object	303	5.9 [3.3,8.6]
CLIP B/16 LAION-2B	modify_object	303	9.9 [6.5,13.3]
CLIP H/14 LAION-2B	modify_object	303	14.2 [10.3,18.1]
SigLIP B/16	modify_object	303	8.9 [5.7,12.1]
SigLIP SO400M	modify_object	303	23.8 [19.0,28.6]
SigLIP-2 B/16	modify_object	303	18.8 [14.4,23.2]
SigLIP-2 L/16	modify_object	303	28.4 [23.3,33.5]
SigLIP-2 SO400M	modify_object	303	29.7 [24.6,34.8]
NegCLIP B/32	modify_object	303	7.3 [4.3,10.2]
CLoVe B/32	modify_object	303	5.0 [2.5,7.4]
DreamLIP B/16	modify_object	303	8.3 [5.2,11.3]
DFN H/14	modify_object	303	16.5 [12.3,20.7]
EVA-02 L/14	modify_object	303	19.5 [15.0,23.9]
MetaCLIP-2 H/14	modify_object	303	18.2 [13.8,22.5]
CLIP B/32	swap_color	312	2.2 [0.6,3.9]
CLIP B/16 LAION-2B	swap_color	312	1.9 [0.4,3.4]
CLIP H/14 LAION-2B	swap_color	312	1.6 [0.2,3.0]
SigLIP B/16	swap_color	312	3.2 [1.3,5.2]
SigLIP SO400M	swap_color	312	3.2 [1.3,5.2]
SigLIP-2 B/16	swap_color	312	5.1 [2.7,7.6]
SigLIP-2 L/16	swap_color	312	2.6 [0.8,4.3]
SigLIP-2 SO400M	swap_color	312	3.5 [1.5,5.6]
NegCLIP B/32	swap_color	312	3.2 [1.3,5.2]
CLoVe B/32	swap_color	312	2.3 [0.6,3.9]
DreamLIP B/16	swap_color	312	18.6 [14.3,22.9]
DFN H/14	swap_color	312	3.5 [1.5,5.6]
EVA-02 L/14	swap_color	312	1.6 [0.2,3.0]
MetaCLIP-2 H/14	swap_color	312	0.3 [0.0,0.9]
CLIP B/32	swap_material	303	0.0 [0.0,0.0]
CLIP B/16 LAION-2B	swap_material	303	1.7 [0.2,3.1]
CLIP H/14 LAION-2B	swap_material	303	2.0 [0.4,3.5]
SigLIP B/16	swap_material	303	3.0 [1.1,4.9]
SigLIP SO400M	swap_material	303	2.0 [0.4,3.5]
SigLIP-2 B/16	swap_material	303	4.3 [2.0,6.6]
SigLIP-2 L/16	swap_material	303	1.3 [0.0,2.6]
SigLIP-2 SO400M	swap_material	303	2.6 [0.8,4.4]
NegCLIP B/32	swap_material	303	0.3 [0.0,1.0]
CLoVe B/32	swap_material	303	0.0 [0.0,0.0]
DreamLIP B/16	swap_material	303	3.0 [1.1,4.9]
DFN H/14	swap_material	303	2.0 [0.4,3.5]
EVA-02 L/14	swap_material	303	1.0 [0.0,2.1]
MetaCLIP-2 H/14	swap_material	303	1.0 [0.0,2.1]
CLIP B/32	modify_count	300	1.7 [0.2,3.1]
CLIP B/16 LAION-2B	modify_count	300	0.7 [0.0,1.6]
CLIP H/14 LAION-2B	modify_count	300	2.3 [0.6,4.0]
SigLIP B/16	modify_count	300	2.0 [0.4,3.6]
SigLIP SO400M	modify_count	300	4.0 [1.8,6.2]
SigLIP-2 B/16	modify_count	300	3.7 [1.5,5.8]
SigLIP-2 L/16	modify_count	300	5.0 [2.5,7.5]
SigLIP-2 SO400M	modify_count	300	5.3 [2.8,7.9]
NegCLIP B/32	modify_count	300	1.7 [0.2,3.1]
CLoVe B/32	modify_count	300	0.7 [0.0,1.6]
DreamLIP B/16	modify_count	300	3.3 [1.3,5.4]
DFN H/14	modify_count	300	4.7 [2.3,7.1]
EVA-02 L/14	modify_count	300	4.0 [1.8,6.2]
MetaCLIP-2 H/14	modify_count	300	3.3 [1.3,5.4]
CLIP B/32	modify_presence	312	0.6 [0.0,1.5]
CLIP B/16 LAION-2B	modify_presence	312	1.0 [0.0,2.0]
CLIP H/14 LAION-2B	modify_presence	312	1.6 [0.2,3.0]
SigLIP B/16	modify_presence	312	1.0 [0.0,2.0]
SigLIP SO400M	modify_presence	312	1.0 [0.0,2.0]
SigLIP-2 B/16	modify_presence	312	1.9 [0.4,3.4]
SigLIP-2 L/16	modify_presence	312	2.9 [1.0,4.7]
SigLIP-2 SO400M	modify_presence	312	1.0 [0.0,2.0]
NegCLIP B/32	modify_presence	312	1.3 [0.0,2.5]
CLoVe B/32	modify_presence	312	0.0 [0.0,0.0]
DreamLIP B/16	modify_presence	312	0.6 [0.0,1.5]
DFN H/14	modify_presence	312	1.0 [0.0,2.0]
EVA-02 L/14	modify_presence	312	0.6 [0.0,1.5]
MetaCLIP-2 H/14	modify_presence	312	0.6 [0.0,1.5]
CLIP B/32	modify_spatial	297	0.3 [0.0,1.0]
CLIP B/16 LAION-2B	modify_spatial	297	0.7 [0.0,1.6]
CLIP H/14 LAION-2B	modify_spatial	297	0.0 [0.0,0.0]
SigLIP B/16	modify_spatial	297	0.3 [0.0,1.0]
SigLIP SO400M	modify_spatial	297	0.0 [0.0,0.0]
SigLIP-2 B/16	modify_spatial	297	0.0 [0.0,0.0]
SigLIP-2 L/16	modify_spatial	297	0.3 [0.0,1.0]
SigLIP-2 SO400M	modify_spatial	297	0.3 [0.0,1.0]
NegCLIP B/32	modify_spatial	297	0.3 [0.0,1.0]
CLoVe B/32	modify_spatial	297	0.7 [0.0,1.6]
DreamLIP B/16	modify_spatial	297	0.0 [0.0,0.0]
DFN H/14	modify_spatial	297	0.0 [0.0,0.0]
EVA-02 L/14	modify_spatial	297	0.7 [0.0,1.6]
MetaCLIP-2 H/14	modify_spatial	297	0.0 [0.0,0.0]
CLIP B/32	swap_spatial	300	0.0 [0.0,0.0]
CLIP B/16 LAION-2B	swap_spatial	300	0.0 [0.0,0.0]
CLIP H/14 LAION-2B	swap_spatial	300	0.0 [0.0,0.0]
SigLIP B/16	swap_spatial	300	0.0 [0.0,0.0]
SigLIP SO400M	swap_spatial	300	0.7 [0.0,1.6]
SigLIP-2 B/16	swap_spatial	300	0.7 [0.0,1.6]
SigLIP-2 L/16	swap_spatial	300	0.7 [0.0,1.6]
SigLIP-2 SO400M	swap_spatial	300	1.0 [0.0,2.1]
NegCLIP B/32	swap_spatial	300	0.3 [0.0,1.0]
CLoVe B/32	swap_spatial	300	0.0 [0.0,0.0]
DreamLIP B/16	swap_spatial	300	0.3 [0.0,1.0]
DFN H/14	swap_spatial	300	0.0 [0.0,0.0]
EVA-02 L/14	swap_spatial	300	1.0 [0.0,2.1]
MetaCLIP-2 H/14	swap_spatial	300	0.3 [0.0,1.0]
CLIP B/32	modify_action	303	7.9 [4.9,11.0]
CLIP B/16 LAION-2B	modify_action	303	17.2 [12.9,21.4]
CLIP H/14 LAION-2B	modify_action	303	26.1 [21.1,31.0]
SigLIP B/16	modify_action	303	22.1 [17.4,26.8]
SigLIP SO400M	modify_action	303	46.5 [40.9,52.2]
SigLIP-2 B/16	modify_action	303	21.1 [16.5,25.7]
SigLIP-2 L/16	modify_action	303	46.5 [40.9,52.2]
SigLIP-2 SO400M	modify_action	303	48.8 [43.2,54.5]
NegCLIP B/32	modify_action	303	13.2 [9.4,17.0]
CLoVe B/32	modify_action	303	6.9 [4.1,9.8]
DreamLIP B/16	modify_action	303	17.5 [13.2,21.8]
DFN H/14	modify_action	303	33.3 [28.0,38.6]
EVA-02 L/14	modify_action	303	25.1 [20.2,30.0]
MetaCLIP-2 H/14	modify_action	303	31.4 [26.1,36.6]
CLIP B/32	modify_pose	312	3.8 [1.7,6.0]
CLIP B/16 LAION-2B	modify_pose	312	5.4 [2.9,8.0]
CLIP H/14 LAION-2B	modify_pose	312	4.5 [2.2,6.8]
SigLIP B/16	modify_pose	312	10.6 [7.2,14.0]
SigLIP SO400M	modify_pose	312	9.6 [6.3,12.9]
SigLIP-2 B/16	modify_pose	312	10.3 [6.9,13.6]
SigLIP-2 L/16	modify_pose	312	10.3 [6.9,13.6]
SigLIP-2 SO400M	modify_pose	312	13.5 [9.7,17.2]
NegCLIP B/32	modify_pose	312	7.1 [4.2,9.9]
CLoVe B/32	modify_pose	312	5.1 [2.7,7.6]
DreamLIP B/16	modify_pose	312	14.1 [10.2,18.0]
DFN H/14	modify_pose	312	12.8 [9.1,16.5]
EVA-02 L/14	modify_pose	312	5.1 [2.7,7.6]
MetaCLIP-2 H/14	modify_pose	312	4.5 [2.2,6.8]
CLIP B/32	modify_state	438	0.5 [0.0,1.1]
CLIP B/16 LAION-2B	modify_state	438	0.7 [0.0,1.5]
CLIP H/14 LAION-2B	modify_state	438	1.1 [0.1,2.1]
SigLIP B/16	modify_state	438	0.9 [0.0,1.8]
SigLIP SO400M	modify_state	438	2.1 [0.7,3.4]
SigLIP-2 B/16	modify_state	438	2.3 [0.9,3.7]
SigLIP-2 L/16	modify_state	438	3.2 [1.5,4.8]
SigLIP-2 SO400M	modify_state	438	2.7 [1.2,4.3]
NegCLIP B/32	modify_state	438	1.1 [0.1,2.1]
CLoVe B/32	modify_state	438	0.2 [0.0,0.7]
DreamLIP B/16	modify_state	438	0.5 [0.0,1.1]
DFN H/14	modify_state	438	1.8 [0.6,3.1]
EVA-02 L/14	modify_state	438	0.7 [0.0,1.5]
MetaCLIP-2 H/14	modify_state	438	0.7 [0.0,1.5]
CLIP B/32	swap_role	135	0.0 [0.0,0.0]
CLIP B/16 LAION-2B	swap_role	135	0.0 [0.0,0.0]
CLIP H/14 LAION-2B	swap_role	135	0.0 [0.0,0.0]
SigLIP B/16	swap_role	135	1.5 [0.0,3.5]
SigLIP SO400M	swap_role	135	1.5 [0.0,3.5]
SigLIP-2 B/16	swap_role	135	0.7 [0.0,2.2]
SigLIP-2 L/16	swap_role	135	1.5 [0.0,3.5]
SigLIP-2 SO400M	swap_role	135	3.0 [0.1,5.8]
NegCLIP B/32	swap_role	135	2.2 [0.0,4.7]
CLoVe B/32	swap_role	135	0.7 [0.0,2.2]
DreamLIP B/16	swap_role	135	2.2 [0.0,4.7]
DFN H/14	swap_role	135	1.5 [0.0,3.5]
EVA-02 L/14	swap_role	135	3.0 [0.1,5.8]
MetaCLIP-2 H/14	swap_role	135	3.0 [0.1,5.8]

14个双编码器，合并短、中、长描述。保留论文点估计与区间；角色交换仅45组图像，即135个图像对×描述观测。

CompoSET paper v1 — task, tables and limitations

14个模型的编辑类型与描述长度汇总

表10对14个模型的Group取平均。这些汇总值不是任何单个模型的成绩。

14个模型的编辑类型与描述长度汇总
编辑类型	每档n	短描述Group %	中描述Group %	长描述Group %
modify_color	206	35.3	34.7	30.5
modify_pattern	101	16.1	15.2	14.4
modify_material	161	10.2	10.0	8.3
modify_shape	103	11.5	10.6	7.3
modify_transparency	100	1.6	1.3	1.1
modify_object	101	15.3	15.7	14.9
swap_color	104	3.1	3.4	4.8
swap_material	101	1.2	1.8	2.1
modify_count	100	3.4	3.1	2.5
modify_presence	104	0.3	0.5	2.5
modify_spatial	99	0.4	0.1	0.3
swap_spatial	100	0.4	0.4	0.3
modify_action	101	27.2	27.3	23.4
modify_pose	104	10.4	9.3	5.3
modify_state	146	1.4	1.6	1.0
swap_role	45	1.4	0.8	2.2

表10的模型均值子集诊断，不进入主成绩序列。

CompoSET paper v1 — task, tables and limitations

200例生成式模型小样本

表11采用与双编码器不同的评分方式：开源权重模型计算描述似然，API模型成对选择并交换选项顺序。

200例生成式模型小样本
模型	评分方式	图到文%	文到图%	Group %
BLIP-2 (OPT-2.7B)	Open-weight (caption log-likelihood)	5.0	11.0	2.5
LLaVA-1.6 (Mistral-7B)	Open-weight (caption log-likelihood)	30.5	3.0	1.5
Qwen2-VL (7B Instruct)	Open-weight (caption log-likelihood)	36.5	0.5	0.5
Gemini 2.5 Flash	Frontier API (prompted pairwise, counterbalanced)	64.5	16.0	12.0
Claude Haiku 4.5	Frontier API (prompted pairwise, counterbalanced)	46.5	17.0	13.0
Claude Sonnet 4.6	Frontier API (prompted pairwise, counterbalanced)	71.5	47.0	42.5

分层200例子集。API温度0，似然按长度归一化；论文把I2T作为似然组的优先诊断指标。Group按原文展示，但不混入整体成绩。

CompoSET paper v1 — task, tables and limitations

API模型按描述长度划分

表12给出API模型小样本中各描述长度的Group。

API模型按描述长度划分
模型	短描述Group %	中描述Group %	长描述Group %
Gemini 2.5 Flash	15.9	19.7	13.6
Claude Haiku 4.5	11.1	19.7	7.6
Claude Sonnet 4.6	44.4	46.5	36.4

每档约67例，温度0且交换选项顺序；这些是特定子集成绩，不代表完整评测。

CompoSET paper v1 — task, tables and limitations

原始发布

2026年11月2日核对的论文v1及作者直接链接的发布材料，成绩继续归属于评测发布者。

核对于 2026-11-02

已收录的评测资料

这条来源记录收录了 22 条数值成绩。来源区分了 22 个模型身份或配置。同一模型可能有多条记录；比较前请核对具体版本、工具和运行预算。查看成绩及逐条证据 →

报告指标: Overall strict Group accuracy
原始单位: percent
指标方向: 数值越高越好

从这些资料开始

论文 · 任务设计与实验 · arxiv.org

引用文档及日期（1）

CompoSET paper v1 · overall strict Group accuracy (Table 8) · 2026-10-30

评测信息与证据

来源: Model reports
发布者: 未收录
评测发布日期: 2026-10-30
采集日期: 未收录
模态: 未收录

本站保留来源报告的结果，不代表独立复测。请先核对任务版本、评测协议、分数单位和报告日期；不同运行条件不能直接合并为统一排名。

模型	数值	单位	协议	报告日期与口径	来源
CLIP B/16 (LAION-2B)	7.8	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP B/16 (LAION-400M)	8.4	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP B/16 (OpenAI)	4.8	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP B/32	4.4	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP H/14 (LAION-2B)	8.4	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP L/14 (LAION-2B)	8.4	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLIP L/14 (OpenAI)	5.0	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
CLoVe B/32	5.5	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
DFN B/16	8.4	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
DFN H/14	10.8	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
DreamLIP B/16 (30M)	13.7	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
EVA-02 B/16	7.8	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
EVA-02 L/14	8.6	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
Jina-v4	8.5	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
MetaCLIP-2 H/14	8.9	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
NegCLIP B/32	7.0	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP B/16	10.2	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP L/16-384	12.7	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP SO400M/14	13.5	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP-2 B/16	11.3	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP-2 L/16-384	13.6	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)
SigLIP-2 SO400M/14	13.9	percent	Paper v1 Table 8; full CompoSET release, all caption verbosity tiers; deterministic dual-encoder cosine similarity; strict bidirectional Group pass rate; paper-reported checkpoint identities in Table 4.	2026-10-30 · document_publication	CompoSET paper v1 · overall strict Group accuracy (Table 8)

评测隔离了什么

数据与构建

如何理解Group准确率

区分实验配置

结果的适用边界

查看原始发布

完整评测：22个双编码器

编辑子集：逐模型、逐编辑类型Group

14个模型的编辑类型与描述长度汇总

200例生成式模型小样本

API模型按描述长度划分

原始发布

已收录的评测资料

从这些资料开始

评测信息与证据

报告成绩 22

相关评测探索

继续阅读

六个新评测详情：上下文、Agent 与可执行任务

Navers Lab / Einsia：六个评测分别测什么

AI 评测与排行榜，分别告诉你什么

怎样比较 AI 评测成绩，避免混用评测口径

四个新 Agent 评测：适应变化、安全执行与视频交付

继续探索