EVENT STREAM // NOVEMBER 2018
GPT-1 AND BERT DEMONSTRATE TRANSFER LEARNING
OpenAI's GPT and Google's BERT show that pre-trained language models can transfer knowledge to new tasks. The foundation for everything that followed.
SOURCES [2]
- Radford et al., "Improving Language Understanding by Generative Pre-Training" (OpenAI, 2018) — openai.com
- Devlin et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (arXiv:1810.04805) — arxiv.org