박경문 교수님 연구실(일반인공지능연구실) ICCV 2023 총 2편 게재 승인
일반인공지능연구실(지도교수: 박경문)의 논문 2편이 컴퓨터비전 분야의 Top-tier 학술대회인 International Conference on Computer Vision 2023 (ICCV23)에 게재 승인되었습니다.
논문 제목: “Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning”
“Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning” 논문은 기존의 online continual learning scenario가 실제 현실의 복잡한 데이터를 잘 반영하지 못함을 지적합니다. 불규칙하게 분포하는 현실의 데이터 특성을 효과적으로 모사하기 위해 Stochastic incremental Blurry Scenario (Si-Blurry)를 제시합니다. 이러한 현실적인 데이터에서 발생할 수 있는 Forgetting 문제와 Class Imbalance 문제를 해결하기 위해 Instance-wise Logit Masking, Contrastive Visual Prompt Tuning, Adaptive Feature Scaling, Gradient Similarity-based Focal Loss의 네가지 모듈을 포함하는 Mask and Visual Prompt tuning (MVP)를 제시합니다. 이를 통해 복잡한 replay memory management를 사용하지 않고도 기존의 방법들보다 낮은 forgetting과 높은 accuracy를 달성하였습니다.
[논문정보]
Online Class Incremental Learning on Stochastic Blurry Task Boundary via Mask and Visual Prompt Tuning
Jun Yeong Moon, Keon Hee Park, Jung Uk Kim, and Gyeong-Moon Park
International Conference on Computer Vision (ICCV), 2023
Abstract:
Continual learning aims to learn a model from a continuous stream of data, but it mainly assumes a fixed number of data and tasks with clear task boundaries. However, in real-world scenarios, the number of input data and tasks is constantly changing in a statistical way, not a static way. Although recently introduced incremental learning scenarios having blurry task boundaries somewhat address the above issues, they still do not fully reflect the statistical properties of real-world situations because of the fixed ratio of disjoint and blurry samples. In this paper, we propose a new Stochastic incremental Blurry task boundary scenario, called Si-Blurry, which reflects the stochastic properties of the real-world. We find that there are two major challenges in the Si-Blurry scenario: (1) inter- and intra-task forgettings and (2) class imbalance problem. To alleviate them, we introduce Mask and Visual Prompt tuning (MVP). In MVP, to address the inter- and intra-task forgetting issues, we propose a novel instance-wise logit masking and contrastive visual prompt tuning loss. Both of them help our model discern the classes to be learned in the current batch. It results in consolidating the previous knowledge. In addition, to alleviate the class imbalance problem, we introduce a new gradient similarity-based focal loss and adaptive feature scaling to ease overfitting to the major classes and underfitting to the minor classes. Extensive experiments show that our proposed MVP significantly outperforms the existing state-of-the-art methods in our challenging Si-Blurry scenario.
논문 제목: “LFS-GAN: Lifelong Few-Shot Image Generation”
“LFS-GAN: Lifelong Few-Shot Image Generation” 논문은 이미지 생성 분야에서 소수샷 평생 학습을 처음으로 다룬 연구입니다. 본 논문에서는 lifelong few-shot image generation 태스크를 정의하고 기존의 평생 이미지 생성 연구나 소수샷 이미지 생성 연구의 한계를 지정합니다. 적은 수의 이미지로 구성된 여러 태스크를 학습하면서 발생하는 파괴적 망각 문제와 과잉 적합 문제를 해결하기 위해 본 논문에서는 LFS-GAN이라는 프레임워크를 제안합니다. LFS-GAN의 학습은 Learnable Factorized Tensor (LeFT)와 cluster-wise mode seeking loss를 통해 이루어집니다. LeFT는 효율적이고 효과적인 가중치 분해 및 복원 방법을 통해 기학습된 가중치를 새로운 태스크에 맞게 변형시킵니다. 뿐만 아니라, 보다 다양한 이미지를 생성하기 위해 본 연구에서는 적은 수의 학습 이미지로도 효과적인 cluster-wise mode seeking loss를 제안합니다. 그 결과 LFS-GAN은 본 연구에서 새롭게 제안한 lifelong few-shot image generation task에서 기존의 평생 이미지 생성 연구나 소수샷 이미지 생성 연구와 비교하여 보다 고품질이고 다양한 이미지를 여러 태스크에 걸쳐서 파괴적 망각 없이 생성할 수 있게 됩니다.
[논문 정보]
LFS-GAN: Lifelong Few-Shot Image Generation
Juwon Seo, Ji-su Kang, and Gyeong-Moon Park
International Conference on Computer Vision (ICCV), 2023
Abstract:
We address a challenging lifelong few-shot image generation task for the first time. In this situation, a generative model learns a sequence of tasks using only a few samples per task. Consequently, the learned model encounters both catastrophic forgetting and overfitting problems at a time. Existing studies on lifelong GANs have proposed modulation-based methods to prevent catastrophic forgetting. However, they require considerable additional parameters and cannot generate high-fidelity and diverse images from limited data. On the other hand, the existing few-shot GANs suffer from severe catastrophic forgetting when learning multiple tasks. To alleviate these issues, we propose a framework called Lifelong Few-Shot GAN (LFS-GAN) that can generate high-quality and diverse images in lifelong few-shot image generation task. Our proposed framework learns each task using an efficient task-specific modulator - Learnable Factorized Tensor (LeFT). LeFT is rank-constrained and has a rich representation ability due to its unique reconstruction technique. Furthermore, we propose a novel mode seeking loss to improve the diversity of our model in low-data circumstances. Extensive experiments demonstrate that the proposed LFS-GAN can generate high-fidelity and diverse images without any forgetting and mode collapse in various domains, achieving state-of-the-art in lifelong few-shot image generation task. Surprisingly, we find that our LFS-GAN even outperforms the existing few-shot GANs in the few-shot image generation task.
2023.07.26
