ViT (Vision Transformer) is a novel artificial neural network architecture developed for computer vision tasks. The architecture was proposed by researchers at Google Research in October 2020. It is an improvement over existing architectures such as convolutional neural networks (CNNs) and other transformer-based models, and is capable of efficiently scaling to high-resolution image sizes.
The architecture consists of two components: (1) vision transformer (ViT) and (2) token embedding. The vision transformer uses an attention mechamism that allows the network to learn representations for different parts of an image, in contrast with a CNN which uses hand-crafted filters that are fixed. The attention mechanism also helps eliminate the need for large numbers of parameters, enabling ViT to train faster and more efficiently than a standard CNN.
The second component, token embedding, is used to represent the image data in a condensed form. A “token” is a datapoint within an image. Token embedding is used to embed this data into a lower dimensional space, allowing the ViT to handle high resolution images.
ViT is capable of achieving high accuracy on a range of vision tasks, including image classification, object detection, and instance segmentation. It can also be used for tasks such as natural language processing, making it a powerful addition for both computer vision and natural language processing (NLP).
The ViT architecture has revolutionized computer vision and NLP tasks, allowing for more efficient training times with higher accuracy. It has been adopted by many companies and organizations in a variety of applications. A few examples include Amazon’s DeepRacer and NVIDIA’s RTX GPUs.
