🥕HumanAesExpert : Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessment

Zhichao Liao1 â™±, Xiaokun Liu2, Wenyu Qin2, Qingyu Li2, Qiulin Wang2,
Pengfei Wan2, Di Zhang2, Long Zeng1 ✉, Pingfa Feng1,
1Tsinghua University,  2Kuaishou Technology, 
â™±Internship at KwaiVGI, Kuaishou Technology   ✉Corresponding author 
Paper arXiv Code (Coming Soon)
HumanAesExpert-1B HumanAesExpert-1B HumanAesExpert-8B HumanAesExpert-8B HumanBeauty_Dataset HumanBeauty_Dataset

Abstract

Image Aesthetic Assessment (IAA) is a long-standing and challenging research task. However, its subset, Human Image Aesthetic Assessment (HIAA), has been scarcely explored, even though HIAA is widely used in social media, AI workflows, and related domains. To bridge this research gap, our work pioneers a holistic implementation framework tailored for HIAA. Specifically, we introduce HumanBeauty, the first dataset purpose-built for HIAA, which comprises 108k high-quality human images with manual annotations. To achieve comprehensive and fine-grained HIAA, 50K human images are manually collected through a rigorous curation process and annotated leveraging our trailblazing 12-dimensional aesthetic standard, while the remaining 58K with overall aesthetic labels are systematically filtered from public datasets. Based on the HumanBeauty database, we propose HumanAesExpert, a powerful Vision Language Model for aesthetic evaluation of human images. We innovatively design an Expert head to incorporate human knowledge of aesthetic sub-dimensions while jointly utilizing the Language Modeling (LM) and Regression head. This approach empowers our model to achieve superior proficiency in both overall and fine-grained HIAA. Furthermore, we introduce a MetaVoter, which aggregates scores from all three heads, to effectively balance the capabilities of each head, thereby realizing improved assessment precision. Extensive experiments demonstrate that our HumanAesExpert models deliver significantly better performance in HIAA than other state-of-the-art models. Our datasets, models, and codes are publicly released to advance the HIAA community.

HumanBeauty Dataset Construction Pipeline

First, we select six diverse open-source datasets as data sources and perform data filtering to build our HumanBeauty-58k. Additionally, we manually collect and annotate 50k human images across multiple dimensions to create our HumanBeauty-50k. Finally, we map all the scores into text of rating level to form QA pairs for training.

HumanAesExpert Overview

Our approach uses both LM and Regression heads, along with an Expert head and MetaVoter.

Visualization Results

The Visualization Results of Our Model, where ‘’( )'' indicate the Ground Truth scores. From A to L, they respectively represent facial brightness, facial feature clarity, facial skin tone, facial structure, facial contour clarity, facial aesthetic, outfit, body shape, looks, general appearance aesthetic, environment and overall aesthetic scores.

BibTeX

If you find this project useful in your research, please consider cite: 

@misc{liao2025humanaesexpertadvancingmultimodalityfoundation,
      title={HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessment}, 
      author={Zhichao Liao and Xiaokun Liu and Wenyu Qin and Qingyu Li and Qiulin Wang and Pengfei Wan and Di Zhang and Long Zeng and Pingfa Feng},
      year={2025},
      eprint={2503.23907},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.23907}, 
}