10월 3주차 논문 요약: GQA, LLM, LLMLingua, LLeMA, ToRA

10월 3주차 논문 요약: GQA, LLM, LLMLingua, LLeMA, ToRA

최근(2023.10)에 나온 논문들을 읽어보고 간단히 정리했습니다. 혹시 부족하거나 잘못된 내용이 있다면 댓글 부탁드립니다 ️ GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints (2023.05) [Google Research] Multi-head Attention(MHA)만큼의 품질이 보장되고, Multi-query Attention(MQA)만큼의 속도를 낼 수 있는 Group-query Attention(GQA)를 제안 기존 Transformer 아키텍쳐에서 사용되는 Multi-head Attention의 경우 메모리 사용량이 지나치게 많이 요구되어 이를 적용하기가 점점 더 어려워지는 추세였음 이..

원문링크 : 10월 3주차 논문 요약: GQA, LLM, LLMLingua, LLeMA, ToRA

 등록된 다른 글

Deep Neural Network(2)

Deep Neural Network(2)

<Embedding> LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2024.04)

<Embedding> LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (2024.04)

PCA algorithm

PCA algorithm

<LLM, Zero-shot> [T0] Multitask Prompted Training Enables Zero-Shot Task Generalization (2022.03)

<LLM, Zero-shot> [T0] Multitask Prompted Training Enables Zero-Shot Task Generalization (2022.03)

[Short Papaer Review] Learning to Reason and Memorize with Self-Notes

[Short Papaer Review] Learning to Reason and Memorize with Self-Notes

Variances and covariances

Variances and covariances

<CoT> [Self-Consistency] Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022.03 -> 2023.03)

<CoT> [Self-Consistency] Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022.03 -> 2023.03)

[BOJ] 1244 : 스위치 켜고 끄기 [구현](Python)

[BOJ] 1244 : 스위치 켜고 끄기 [구현](Python)