Akira’s Machine Learning News — Issue #32

Akihiro FUJII
6 min readOct 31, 2021

Featured Paper/News in This Week.

  • Methods to improve the performance of zero-shot inference have been presented. Since GPT-3 zero-shot inference is used in many applications, any improvement in the performance of zero-shot inference may have a significant social impact.

— — — — — — — — — — — — — — — — — — –

In the following sections, I will introduce various articles and papers not only on the above contents but also on the following five topics.

  1. Featured Paper/News in This Week
  2. Machine Learning Use Case
  3. Papers
  4. Articles related to machine learning technology

— — — — — — — — — — — — — — — — — — –

1. Featured Paper/News in This Week

Improved performance of zero-shot inference by fine-tuning using templates.arxiv.org

[2109.01652] Finetuned Language Models Are Zero-Shot Learners
This is a study that proposes FLAN in which instruction-tune to improve the performance of zero-shot inference. Inspired by the fact that natural language tasks can be described in natural language, they use templates for fine-tuning. Results exceed GPT-3 for many tasks.

— — — — — — — — — — — — — — — — — — –

2. Machine Learning use case

AI and Rightswww.wired.com

AI has had a profound impact on society, but it has also been detrimental to a few. In order to prevent this from happening, it is suggested that a Bill of Rights is needed that outlines the “rights” of people about AI, including how it makes decisions.

Massive Disinformation through Large-Scale Language Modelsbreakingdefense.com

An article on the concerns of disinformation dissemination using large scale language models such as GPT-3. It claims that there are examples of strategic use on a national scale to manipulate information.

— — — — — — — — — — — — — — — — — — –

3. Machine Learning Papers

The state of ML outside of Big Techarxiv.org

[2110.02932] Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development
Unlike Google and other big tech companies, startups and non-tech companies do not have ample resources and may be excluded from the ML community. This study investigates what problems they face in this situation. It found that additional challenges arise in developing and validating fair and robust ML models, a trend that is particularly worrisome given the increasing popularity of ML and the rapid democratization

GAN rendering does not depend on the absolute coordinates of pixels.arxiv.org

[2106.12423] Alias-Free Generative Adversarial Networks
This paper proposes StyleGAN3, which solves the problem of GAN drawing depending on the absolute coordinates of pixels, by making CNNs translation invariant. It requires less computation than StyleGAN2, and can be generated with anti-aliasing.

VAE to learn causality of latent variablesarxiv.org

[2004.08697] CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models
The authors proposed CausalVAE, which can learn disentangle representation of latent representation in VAE. The learnable parameter A is set in the latent layer to learn the causal relationship between latent variables and learning is performed while applying constraints to reproduce the latent variables.

A method that can equally attack each class in object detectionarxiv.org

[2103.16148] Class-Aware Robust Adversarial Training for Object Detection
Although multiple objects appear in object detection, we proposed a method that can attack each class equally by normalizing each class. They also confirmed that its adversarial learning with adversarial noise improves the robustness against adversarial attacks.

A DeepFake detection method that prevents overfitting by looking at the entire facearxiv.org

[2104.06609] Representative Forgery Mining for Fake Face Detection
Deep Fake detection overfits by focusing only on a limited area of the face. Therefore, the authors propose a method to make the model pay attention to other parts of the face by applying a transformation that hides important regions during training. This method can be implemented in addition to conventional methods, and they have confirmed that it improves accuracy on various models and data sets.

— — — — — — — — — — — — — — — — — — –

4. Technical Articles

AI in 2021www.stateof.ai

A report on AI in 2021, including Transformer’s breakthrough as a general-purpose, domain-agnostic model, record-breaking startup investments, and the start of AI-powered military expansion.

Comparison of BigTransforer and CLIPblog.alexcg.net

A comparison of Google’s BigTransfer encoder and CLIP image encoder in meme search. The results show that BigTransfer is faster and may be slightly more accurate in meme search.

— — — — — — — — — — — — — — — — — — –

5. Other Topics

Google Lens will be soon available for Google Chrome!twitter.com

Google Lens, which allows you to search for images in the browser, will be soon available for Google Chrome. It seems to be able to search for similar images by selecting a rectangle on a web page.

— — — — — — — — — — — — — — — — — — –

Other Blogs

— — — — — — — — — — — — — — — — — — –

🌟I post weekly newsletters! Please subscribe!🌟

— — — — — — — — — — — — — — — — — — –

About Me

Manufacturing Engineer/Machine Learning Engineer/Data Scientist / Master of Science in Physics / http://github.com/AkiraTOSEI/

Twitter, I post one-sentence paper commentary.

--

--