Skip to content
@Q-Future

Visual Evaluation with Foundation Models

We are working towards a future that one foundation model can be a multi-purpose expert for low-level visual perception and visual evaluation.

👁️‍🗨️ Low-level Visual Perception in the Foundation Model Era

🔖Aiming at next-era cornerstone research

Low-level Visual Perception | Multi-Modality Large Language Models | Visual Quality Assessment

📖Main Projects

  • Co-Instruct: Homepage, Repo, Demo. Open-ended visual quality comparer (up to 4 images), low-level visual assistant, an improved version of ②Q-Instruct [CVPR 2024].

  • Q-Align [ICML 2024]: Homepage, Repo, Demo. A unified visual scorer for images and videos, via text-instructed alignment on multi-modality foundation models; can efficiently fine-tune to more datasets with stable good performance. State-of-the-art on IQA, VQA, and IAA.

  • Q-Instruct [CVPR 2024]: Homepage, Repo, 200K Dataset, Technical Report A large-scale instruction tuning dataset to improve low-level perceptual abilities of foundation models.

  • Q-Bench+ [ICLR2024, Spotlight]: Homepage, Repo, Data-Single, Data-Pair, Preprint The first low-level benchmark for foundation models on low-level vision.

🖋️Extension Projects

  • Q-Boost: Homepage A discussion on boosting the IQA performance for non-specially-IQA-aligned MLLMs.

  • [Pending]Chinese-Q-Bench/质衡: Homepage, Repo The first attempt to test multi-lingual abilities on low-level vision.

Maintained by Teo Wu@Singapore and Zicheng Zhang@Shanghai.

Pinned Loading

  1. A-Bench A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    117 3

  2. Co-Instruct Co-Instruct Public

    ④[ECCV 2024, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    59 4

  3. Q-Align Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Python 235 16

  4. Q-Instruct Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Python 189 8

  5. Q-Bench Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Jupyter Notebook 228 12

Repositories

Showing 10 of 12 repositories
  • Co-Instruct Public

    ④[ECCV 2024, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    Q-Future/Co-Instruct’s past year of commit activity
    59 4 2 0 Updated Aug 12, 2024
  • Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Q-Future/Q-Align’s past year of commit activity
    Python 235 16 4 0 Updated Aug 12, 2024
  • Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Q-Future/Q-Instruct’s past year of commit activity
    Python 189 8 11 0 Updated Aug 12, 2024
  • Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Q-Future/Q-Bench’s past year of commit activity
    Jupyter Notebook 228 12 1 0 Updated Aug 12, 2024
  • A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    Q-Future/A-Bench’s past year of commit activity
    117 3 0 0 Updated Aug 11, 2024
  • .github Public

    We are an open-source collaborative project to bring new possibilities to IQA!

    Q-Future/.github’s past year of commit activity
    2 0 0 0 Updated Aug 2, 2024
  • Q-Refine Public

    [MM 2024 Oral] Refiner for AIGC

    Q-Future/Q-Refine’s past year of commit activity
    Jupyter Notebook 23 Apache-2.0 1 1 0 Updated Jul 29, 2024
  • Q-Ground Public

    Official codes for "Q-Ground: Image Quality Grounding with Large Multi-modality Models", ACM MM2024 (Oral)

    Q-Future/Q-Ground’s past year of commit activity
    21 0 0 0 Updated Jul 28, 2024
  • LMM-PCQA Public

    Official repo for `LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM', ACM MM2024 Oral

    Q-Future/LMM-PCQA’s past year of commit activity
    Python 9 0 0 0 Updated Jul 23, 2024
  • Q-Future/Compare2Score’s past year of commit activity
    Python 11 MIT 1 1 0 Updated Jul 8, 2024

Top languages

Loading…

Most used topics

Loading…