LLM Pre-Train - a zk67 Collection

zk67 's Collections

LLM Reasoning Papers

LLM Technical Report

LLM Post Training

LLM Pre-Train

updated Oct 8

Scaling Laws for Neural Language Models

Paper • 2001.08361 • Published Jan 23, 2020 • 6

Note OpenAI 2020 Scaling Law-1
Scaling Laws for Autoregressive Generative Modeling

Paper • 2010.14701 • Published Oct 28, 2020

Note OpenAI 2020 Scaling Law-2
Training Compute-Optimal Large Language Models

Paper • 2203.15556 • Published Mar 29, 2022 • 10

Note DeepMind 2022 Scaling Law-3 The Bitter Lesson. by Rich Sutton 2019 http://www.incompleteideas.net/IncIdeas/BitterLesson.html
A Survey on Data Selection for Language Models

Paper • 2402.16827 • Published Feb 26 • 4