Reach and Limits of the Supermassive Model GPT-3

About This blog post

This blog post provides an explanation of GPT-3 [1]. The summary of the content is as follows.

• In GPT-2, the predecessor of GPT-3, the authors built a language model with a huge dataset and a huge network, and they got good results without having to train it in each task.
• In GPT-3, the authors built a language model with an even bigger dataset + an even bigger network, and got great results when the model see dozens of samples.