Li Dong's picture

Li Dong

unilm

·

AI & ML interests

Language Model Pre-Training

Organizations

unilm's activity

upvoted a paper 29 days ago

Differential Transformer

Paper • 2410.05258 • Published Oct 7 • 165

upvoted 2 papers about 1 month ago

Data Selection via Optimal Control for Language Models

Paper • 2410.07064 • Published Oct 9 • 8

Self-Boosting Large Language Models with Synthetic Preference Data

Paper • 2410.06961 • Published Oct 9 • 15

upvoted a paper 3 months ago

DataComp-LM: In search of the next generation of training sets for language models

Paper • 2406.11794 • Published Jun 17 • 48

upvoted a paper 4 months ago

Direct Preference Knowledge Distillation for Large Language Models

Paper • 2406.19774 • Published Jun 28 • 21

upvoted a paper 5 months ago

BEiT: BERT Pre-Training of Image Transformers

Paper • 2106.08254 • Published Jun 15, 2021 • 2

upvoted 2 papers 6 months ago

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models

Paper • 2405.09220 • Published May 15 • 24

You Only Cache Once: Decoder-Decoder Architectures for Language Models

Paper • 2405.05254 • Published May 8 • 9

upvoted a paper 7 months ago

Multi-Head Mixture-of-Experts

Paper • 2404.15045 • Published Apr 23 • 59

upvoted a paper 8 months ago

Algorithmic progress in language models

Paper • 2403.05812 • Published Mar 9 • 18

upvoted 2 papers 9 months ago

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper • 2402.17764 • Published Feb 27 • 602

Towards Optimal Learning of Language Models

Paper • 2402.17759 • Published Feb 27 • 16

upvoted 8 papers about 1 year ago

TEAL: Tokenize and Embed ALL for Multi-modal Large Language Models

Paper • 2311.04589 • Published Nov 8, 2023 • 18

Does GPT-4 Pass the Turing Test?

Paper • 2310.20216 • Published Oct 31, 2023 • 17

Text Rendering Strategies for Pixel Language Models

Paper • 2311.00522 • Published Nov 1, 2023 • 10

Idempotent Generative Network

Paper • 2311.01462 • Published Nov 2, 2023 • 24

Levels of AGI: Operationalizing Progress on the Path to AGI

Paper • 2311.02462 • Published Nov 4, 2023 • 33

DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design

Paper • 2310.15144 • Published Oct 23, 2023 • 13

BitNet: Scaling 1-bit Transformers for Large Language Models

Paper • 2310.11453 • Published Oct 17, 2023 • 96

Large Language Model for Science: A Study on P vs. NP

Paper • 2309.05689 • Published Sep 11, 2023 • 20