Vision-Language Models (VLMs) represent a frontier in artificial intelligence, creating systems that can see and reason about the visual world in tandem with human language. The field is incredibly diverse, with models specializing in distinct but related tasks. This guide provides a structured overview of the most prominent VLMs, categorized by their primary function.

Vision Transformers (ViTs) have become a popular choice for image recognition and related tasks, but they can be computationally expensive and memory-heavy. Below is a list of common (and often complementary) techniques to optimize Transformers—including ViTs—for more efficient training and inference. Alongside each category, I’ve mentioned some influential or representative papers.

Key ideas: 1. Perform attention **across rows** (the width dimension, $W$) for each of the $H$ rows, independently. 2. Then perform attention **across columns** (the height dimension, $H$) for each of the $W$ columns, independently. 3. Each step is effectively 1D self-attention, so the cost scales like $O(H \cdot W^2 + W \cdot H^2)$ instead of $O(H^2 W^2)$.

Traditional multi-object tracking (MOT) systems often follow a two-step pipeline: Detect objects in each frame independently. Associate detections across frames to form trajectories. This separation can lead to suboptimal solutions since detection and association are treated as separate problems. TrackFormer merges these steps by extending a Transformer-based detection architecture (inspired by DETR) to simultaneously detect and track objects. It does this by introducing track queries that carry information about previously tracked objects forward in time, allowing the network to reason about detection and association in a unified end-to-end manner.

The Detection Transformer (DETR) is a novel approach to object detection that leverages Transformers, which were originally designed for sequence-to-sequence tasks like machine translation. Introduced by Carion et al. in 2020, DETR simplifies the object detection pipeline by eliminating the need for hand-crafted components like anchor generation and non-maximum suppression (NMS).

Vision Transformers (ViTs) apply the Transformer architecture, originally designed for natural language processing (NLP), to computer vision tasks like image classification. ViTs treat an image as a sequence of patches (akin to words in a sentence) and process them using Transformer encoders.

Variational Autoencoders (VAEs): A Complete Tutorial

Python implementation of a Conditional Generative Adversarial Network (cGAN) using PyTorch.

Generative Adversarial Networks (GANs) are composed of two neural networks: * A **Generator (G)**: learns to generate fake samples $G(z)$ from random noise $z \sim p(z)$ * A **Discriminator (D)**: learns to classify samples as real (from data) or fake (from the generator) * These networks are trained in a two-player minimax game.

**Sub-pixel convolution** (also known as **pixel shuffle**) is a technique primarily used for **image super-resolution** and other upsampling tasks in deep learning. Instead of upsampling via interpolation or transposed convolution, it learns to generate a **high-resolution image** from a low-resolution feature map by **reorganizing the channels**.

This note explains how **features evolve across layers in deep neural networks** (especially CNNs), and how **fine-grained features** emerge and are preserved or enhanced for tasks like fine-grained classification, detection, and facial recognition.

Generalization issue with Distillation

This tutorial is written to provide an extensive understanding of the Mask R-CNN architecture by dissecting every individual component involved in its pipeline.

Single shot object detector