<Distillation, Decoding> [Proxy-tuning] Tuning Language Models by Proxy (2024.01)

<Distillation, Decoding> [Proxy-tuning] Tuning Language Models by Proxy (2024.01)

관심있는 NLP 논문을 읽어보고 간단히 정리했습니다. 혹시 부족하거나 잘못된 내용이 있다면 댓글 부탁드립니다 ️ usechatgpt init success [Allen Institue for AI] - a lightweight decoding-time algorithm, proxy-tuning을 제안 - output vocabulary에 대한 prediction만을 활용하는 테크닉 - 사이즈가 작은 두 모델의 확률 분포차를 큰 베이스 모델에 반영하는 방식으로, 본 논문에서는 Llama 패밀리 모델들을 사용 1. Introduction LLM을 직접 학습시키는 것은 너무나도 많은 비용을 필요로 하기도 하고, 사실 요즘엔 애초에 접근 자체가 불가능한 경우가 많습니다. 회사 이름값을 못하는 OpenAI의..

원문링크 : <Distillation, Decoding> [Proxy-tuning] Tuning Language Models by Proxy (2024.01)

 등록된 다른 글

Building AI Projects(1) - Workflow of a machine learning project

Building AI Projects(1) - Workflow of a machine learning project

<CoT, Prompting> [Google DeepMind] Chain-of-Thought Reasoning Without Prompting (2024.02)

<CoT, Prompting> [Google DeepMind] Chain-of-Thought Reasoning Without Prompting (2024.02)

<Attention> LongNet: Scaling Transformers to 1,000,000,000 Tokens

<Attention> LongNet: Scaling Transformers to 1,000,000,000 Tokens

Optimization Algorithms(2) - Exponentially Weighted Averages

Optimization Algorithms(2) - Exponentially Weighted Averages

[BOJ] 2212 : 센서 [그리디](Python)

[BOJ] 2212 : 센서 [그리디](Python)

[BOJ] 9251 : LCS [다이나믹 프로그래밍](Python)

[BOJ] 9251 : LCS [다이나믹 프로그래밍](Python)

<Tool> ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

<Tool> ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

<Decompose> Self-Discover: Large Language Models Self-Compose Reasoning Structures (2024.02)

<Decompose> Self-Discover: Large Language Models Self-Compose Reasoning Structures (2024.02)