In this article, I present a total of 84 papers and articles published in 2020 that I found particularly interesting. For the sake of clarity, I divide them into 12 sections. My personal summary for 2020 is as follows.

In 2020, Transformer model made a huge leap forward. In natural language processing, GPT-3, a large scale Transformer model, has achieved high accuracy in many tasks. Using a large amount of data and a large number of parameters, it has surpassed Big Transfer, which had the highest accuracy in image classification.
Fractal image datasets that are free of discriminatory elements and…


  • A series of articles on MLOps with Pytorch Lightning is available. It covers many things, from models using W&B to CI/CD using GitHub actions.
  • Google Research has published a paper describing the methodology and details of their building segmentation task on the African continent. It is an excellent practical example and will be very helpful for engineering.

— — — — — — — — — — — — — — — — — — –

In the following sections, I will introduce various articles and papers not only on the above contents but also on the following five topics.


  • A hardware (camera) specific adversarial attack method has been presented. It seems to generate adversarial noise by using a surrogate model that reproduces the embedded hardware and taking differential values. As with physics simulations, using a surrogate model allows us to backprop even the input values, so it is interesting to see how much we can do with the surrogate model.

— — — — — — — — — — — — — — — — — — –

In the following sections, I will introduce various articles and papers not only on the above contents but also on…


— — — — — — — — — — — — — — — — —…


  • A study of unsupervised learning on a large amount of video data is presented, using Transformer to train well-designed tasks in both temporal and spatial directions. While handling 6 million pieces of data, the improvement in accuracy as the data grows has not reached the upper limit, and there is potential for further improvement in the future.
  • Knowledge distillation has traditionally been described as “making the student model smarter by learning the output distribution of the teacher model,” but a study have shown that the degree of agreement between the distribution of the teacher model and the distribution of the…


— — — — — — — —…


Week 31, 2021 (Aug 8~)


Summary for Week 31, 2021 (week of 8/1/2021)


Summary for the week of July 25, 2021


Summary for Week 26, 2021 (June 27~)

Akihiro FUJII

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store