[논문 리뷰] Big Bird: Transformers for Longer Sequences

이번 게시물에서는 Big Bird를 제시한 논문인 Big Bird: Transformers for Longer Sequences에 대해 다뤄보도록 하겠다. 해당 논문은 2020년도 NeurIPS에 소개되었다. 원문 링크는 다음과 같다. Big Bird: Transformers for Longer Sequences Transformers-based models, such as BERT, have been one of the most successful deep learning models for NLP. Unfortunately, one of their core limitations is the quadratic dependency (mainly in terms of memory) on the sequen..

원문링크 : [논문 리뷰] Big Bird: Transformers for Longer Sequences