Language Models are Few-Shot Learners

We show that scaling up language models greatly improves task-agnostic, few-shot performance. GPT-3, with 175B parameters, achieves strong performance on many NLP datasets without task-specific fine-tuning.