<Multi-modal> [BEiT] Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

<Multi-modal> [BEiT] Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

과거(2022.08)에 나온 논문을 읽어보고 간단히 정리했습니다. 혹시 부족하거나 잘못된 내용이 있다면 댓글 부탁드립니다 ️ usechatgpt init success [Microsoft Corporation] vision 그리고 vision-language task를 고루 잘 수행하는 multimodal foundation model, BEiT-3 여러 pre-training 기법 중에서 오직 masked "language" modeling 기법만을 사용한 것이 특징 배경 Transformers의 아키텍쳐가 엄청나게 좋은 성능을 보이면서 다양한 분야로 퍼져 나갔고, 현재는 multi-modal 분야에도 이것이 활발하게 사용되고 있습니다.(Multiway Transformer) 물론 아직까지 이것이..

원문링크 : <Multi-modal> [BEiT] Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

 등록된 다른 글

<LK Lab, Retrieval> [Np Decoding] Nonparametric Decoding for Generative Retrieval (2023.05)

<LK Lab, Retrieval> [Np Decoding] Nonparametric Decoding for Generative Retrieval (2023.05)

Optimization Algorithms(4) - Learning Rate Decay

Optimization Algorithms(4) - Learning Rate Decay

네이버 부스트캠프 AI Tech 4기 최종 합격 후기!!(비전공자)

네이버 부스트캠프 AI Tech 4기 최종 합격 후기!!(비전공자)

[Short Paper Reveiw] AutoML-GPT: Automatic Machine Learning with GPT

[Short Paper Reveiw] AutoML-GPT: Automatic Machine Learning with GPT

<Benchmark> [MMHAL-BENCH] Aligning Large Multimodal Models with Factually Augmented RLHF

<Benchmark> [MMHAL-BENCH] Aligning Large Multimodal Models with Factually Augmented RLHF

<KD, Fusion> Knowledge Fusion of Large Language Models (2024.01)

<KD, Fusion> Knowledge Fusion of Large Language Models (2024.01)

<Distillation, Decoding> [Proxy-tuning] Tuning Language Models by Proxy (2024.01)

<Distillation, Decoding> [Proxy-tuning] Tuning Language Models by Proxy (2024.01)

Vector

Vector