Bofei Zhang (张博飞)

Email: zhangbofei5675[at]outlook[dot]com

Beijing Institute for General Artificial Intelligence (BIGAI)

Experience

Career

2023/3-Current; Research Engineer @ Beijing Institute for General Artificial Intelligence (BIGAI)
- Currently working on Vision Language Model & Agentic Task Post-Training.
2020/6-2023/3; Software Engineer @ ByteDance
- Tiktok & Lark

Education

2018/9-2020/6; Master in Data Science @ New York University
2013/8-2018/5; Bachelor in Biomedical Engineering @ The Ohio State University

News

Apr 19, 2025	Release dataset, model and training & evaluation code for TongUI: Building Generalized GUI Agents by Learning from Multimodal Web Tutorials
Feb 07, 2025	Multi-modal Agent Tuning (MAT): A framework for auto-generating multimodal tool-usage trajectories (20K MM-Traj), boosting MiniCPM & Qwen-VL tool use by 20%. This work is accepted by ICLR 2025!
Aug 02, 2024	Introducing 🔥FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models. Checkout Here for more details! 🔥FIRE are accepted by NeurIPS 2024!

Latest Posts

Feb 07, 2025	Tutorial of training Multi-modal Agent Tuning projects with LLaMA-Factory

Selected Publications

Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage

Zhi Gao* , Bofei Zhang* , Pengxiang Li*, and 7 more authors

International Conference on Learning Representations (ICLR), 2025

Bib Website

@article{mat,
  title = {Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage},
  author = {Gao, Zhi and Zhang, Bofei and Li, Pengxiang and Ma, Xiaojian and Yuan, Tao and Fan, Yue and Wu, Yuwei and Jia, Yunde and Zhu, Song-Chun and Li, Qing},
  equalauthor = {Gao, Zhi and Zhang, Bofei and Li, Pengxiang},
  correspondence = {Li, Qing},
  year = {2025},
  journal = {International Conference on Learning Representations (ICLR)},
}

FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models

Pengxiang Li* , Zhi Gao* , Bofei Zhang*, and 6 more authors

Neural Information Processing Systems: Datasets and Benchmarks (NeurIPS D&B), 2024

arXiv Bib Website

@article{2024fire,
  title = {FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models},
  author = {Li, Pengxiang and Gao, Zhi and Zhang, Bofei and Yuan, Tao and Wu, Yuwei and Harandi, Mehrtash and Jia, Yunde and Zhu, Song-Chun and Li, Qing},
  journal = {Neural Information Processing Systems: Datasets and Benchmarks (NeurIPS D&B)},
  year = {2024},
  equalauthor = {Li, Pengxiang and Gao, Zhi and Zhang, Bofei},
  correspondence = {Wu, Yuwei and Li, Qing},
}

Deep Reinforcement Learning for Option Replication and Hedging

Journal of Financial Data Science, 2020
Semi-supervised learning for predicting total knee replacement with unsupervised data augmentation Oral

Society of Photo-Optical Instrumentation Engineers (SPIE), 2020

Oral
Attention-Based CNN for KL Grade Classification: Data from the Osteoarthritis Initiative

International Symposium on Biomedical Imaging, 2020
Prediction of Total Knee Replacement and Diagnosis of Osteoarthritis using Deep Learning on Knee Radiographs: Data from the Osteoarthritis Initiative

Radiology, 2020

IntLIM: integration using linear models of metabolomics and gene expression data

BMC bioinformatics, 2018

Bib

@article{Siddiqui2018IntLIM,
  title = {IntLIM: integration using linear models of metabolomics and gene expression data},
  author = {},
  journal = {BMC bioinformatics},
  volume = {19},
  pages = {81},
  year = {2018},
}

RaMP: a comprehensive relational database of metabolomics pathways for pathway enrichment analysis of genes and metabolites

Metabolites, 2018

Bib

@article{zhang2018RaMP,
  title = {RaMP: a comprehensive relational database of metabolomics pathways for pathway enrichment analysis of genes and metabolites},
  author = {},
  journal = {Metabolites},
  volume = {8},
  pages = {16},
  year = {2018},
  url = {https://rampdb.nih.gov/}
}