Natural Language Track

Success Rate (SR) ↑ metrics — Precision (SR-P), Recall (SR-R), and F1 Score (SR-F1) — evaluated across non-occluded (No-Occ), Easy, Medium, and Hard levels (higher is better). The Average SR-F1 determines overall ranking.

Scroll table horizontally
Method Name Team No-Occ Easy Medium Hard Average
SR-P SR-P SR-R SR-F1 SR-P SR-R SR-F1 SR-P SR-R SR-F1 SR-F1
dbxvla Dobot 0.8612 0.9134 0.8938 0.8962 0.8507 0.7668 0.7870 0.7004 0.6574 0.6572 0.8004
sxaidp sxai 0.8506 0.9046 0.8850 0.8875 0.8409 0.7563 0.7768 0.7004 0.6574 0.6572 0.7931
internVL THU 0.8501 0.9024 0.8829 0.8853 0.8404 0.7563 0.7768 0.6939 0.6491 0.6500 0.7906
nltk 0.8411 0.9030 0.8833 0.8859 0.8433 0.7586 0.7794 0.6917 0.6435 0.6470 0.7884
wpl wpl_lab 0.8419 0.8936 0.8744 0.8768 0.8402 0.7572 0.7772 0.7000 0.6546 0.6563 0.7880
flytek-vla ustc 0.8444 0.8999 0.8805 0.8828 0.8363 0.7537 0.7736 0.6828 0.6407 0.6408 0.7854
qwenvl qwen_vla 0.8331 0.8929 0.8731 0.8759 0.8326 0.7472 0.7683 0.7004 0.6574 0.6572 0.7836
zib_1 eureka 0.8331 0.8867 0.8679 0.8703 0.8314 0.7475 0.7682 0.6976 0.6519 0.6522 0.7810
noahvl 0.8271 0.8910 0.8715 0.8741 0.8355 0.7510 0.7718 0.6865 0.6463 0.6461 0.7798
NV NV 0.8351 0.8903 0.8707 0.8733 0.8315 0.7474 0.7678 0.6870 0.6389 0.6426 0.7797
superbird TongJi University 0.8261 0.8888 0.8697 0.8722 0.8298 0.7465 0.7670 0.6809 0.6407 0.6406 0.7765
ds_ddss scu 0.8230 0.8857 0.8672 0.8694 0.8274 0.7439 0.7646 0.6865 0.6435 0.6434 0.7751
buding 0.8140 0.8855 0.8662 0.8687 0.8301 0.7473 0.7673 0.6809 0.6380 0.6387 0.7722
bunny_is_best Bunny_fdu 0.8246 0.8894 0.8707 0.8729 0.8219 0.7394 0.7593 0.6707 0.6333 0.6313 0.7720
UNOGrasp FBK-TeV 0.9212 0.7543 0.7277 0.7350 0.6919 0.5456 0.5903 0.4500 0.3648 0.3872 0.6584
Gemini Robotics ER 2 FBK-TeV 0.8194 0.6670 0.6632 0.6582 0.5135 0.4414 0.4562 0.2306 0.1759 0.1911 0.5312
Gemini Robotics ER 1.5 FBK-TeV 0.4528 0.6187 0.6063 0.6067 0.4636 0.3661 0.3932 0.1833 0.1398 0.1537 0.4016