HighlightCurated by Aramai Editorial

Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks

Introduces CycleGAN for unpaired image-to-image translation, using adversarial and cycle-consistency losses to map between domains without paired data.

Image-to-image translation usually learns a mapping from aligned image pairs, but paired data is often unavailable. This work translates images from a source to a target domain without paired examples, using an adversarial loss to make outputs indistinguishable from the target distribution. Because that mapping is under-constrained, it adds an inverse mapping and a cycle-consistency loss so translating back recovers the original. Results span style transfer, object transfiguration, season transfer, and photo enhancement, outperforming prior methods.

Based on: Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks · IEEE International Conference on Computer Vision

HighlightCurated by Aramai Editorial

Large Language Models are Zero-Shot Reasoners

Shows that simply prepending 'Let's think step by step' turns large language models into strong zero-shot multi-step reasoners.

Large language models are known as strong few-shot learners, and chain-of-thought prompting with step-by-step exemplars reaches state-of-the-art on hard reasoning tasks. This paper shows LLMs are also capable zero-shot reasoners: simply adding 'Let's think step by step' before an answer elicits multi-step reasoning without hand-crafted examples. Using one prompt template, Zero-shot-CoT greatly outperforms standard zero-shot prompting on arithmetic, symbolic, and logical benchmarks, e.g. lifting MultiArith from 17.7% to 78.7% with a large InstructGPT model.

Based on: Large Language Models are Zero-Shot Reasoners · Neural Information Processing Systems

HighlightCurated by Aramai Editorial

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Proposes MTCNN, a three-stage cascaded multitask CNN framework that jointly performs face detection and alignment in real time.

Face detection and alignment are challenging under varied poses, illumination, and occlusion. The paper proposes a deep cascaded multitask framework that exploits the correlation between the two tasks to improve both. Three stages of deep convolutional networks predict face and landmark locations in a coarse-to-fine manner, aided by a new online hard sample mining strategy. The method beats prior techniques on challenging face detection benchmarks including WIDER FACE and on a facial landmark alignment benchmark, while keeping real-time performance.

Based on: Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks · IEEE Signal Processing Letters

HighlightCurated by Aramai Editorial

Attention U-Net: Learning Where to Look for the Pancreas

Attention U-Net adds trainable attention gates to CNNs so they focus on target structures, improving medical image segmentation efficiently.

The paper proposes attention gates (AGs) for medical imaging that automatically learn to focus on target structures of varying shapes and sizes, suppressing irrelevant regions while highlighting salient features. AGs remove the need for explicit external organ-localisation modules used in cascaded CNNs and integrate into standard architectures like U-Net with minimal overhead while boosting sensitivity and accuracy. Evaluated on two large CT abdominal datasets, Attention U-Net consistently improves on U-Net across datasets and training sizes.

Based on: Attention U-Net: Learning Where to Look for the Pancreas · arXiv.org

HighlightCurated by Aramai Editorial

LINE: Large-scale Information Network Embedding

Presents LINE, a scalable network embedding method that preserves local and global structure for graphs with millions of nodes.

LINE embeds very large information networks into low-dimensional vectors for visualization, node classification, and link prediction. Unlike prior methods that do not scale, it handles undirected, directed, and weighted networks via an objective preserving both local and global structure. An edge-sampling algorithm overcomes limitations of classical stochastic gradient descent, improving effectiveness and efficiency. Experiments on language, social, and citation networks embed millions of vertices and billions of edges in hours on one machine.

Based on: LINE: Large-scale Information Network Embedding · The Web Conference

HighlightCurated by Aramai Editorial

On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

Analyzes encoder-decoder neural machine translation models, showing performance drops with longer sentences and more unknown words.

Neural machine translation is a then-new approach to statistical machine translation built purely from neural networks, using an encoder that maps a variable-length sentence to a fixed-length representation and a decoder that generates the translation. The paper analyzes two models: an RNN Encoder-Decoder and a newly proposed gated recursive convolutional neural network. Quality is good for short sentences without unknown words but degrades rapidly as sentence length and unknown words grow; the gated model learns grammatical structure automatically.

Based on: On the Properties of Neural Machine Translation: Encoder–Decoder Approaches · SSST@EMNLP

HighlightCurated by Aramai Editorial

Neural Discrete Representation Learning

Introduces VQ-VAE, a generative model that learns discrete latent representations via vector quantisation, avoiding posterior collapse.

Learning useful representations without supervision remains a key challenge. The paper proposes the Vector Quantised-Variational AutoEncoder (VQ-VAE), a simple generative model whose encoder outputs discrete codes and whose prior is learned rather than static. By incorporating vector quantisation, the model avoids the posterior collapse that afflicts VAEs paired with powerful autoregressive decoders. Combined with an autoregressive prior, it generates high-quality images, videos, and speech, performs speaker conversion, and learns phonemes without supervision.

Based on: Neural Discrete Representation Learning · Neural Information Processing Systems

HighlightCurated by Aramai Editorial

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Scales vision and vision-language pretraining with a dual-encoder contrastive model trained on over one billion noisy image alt-text pairs.

Visual and vision-language representation learning still depends on costly curated datasets, which limits scale. This work instead uses over one billion noisy image alt-text pairs collected without expensive filtering. A simple dual-encoder aligns image and text via a contrastive loss, and the corpus scale offsets its noise to yield state-of-the-art representations. The model transfers strongly to ImageNet and VTAB, enables zero-shot classification, and sets new records on Flickr30K and MSCOCO retrieval.

Based on: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision · International Conference on Machine Learning

HighlightCurated by Aramai Editorial

BioBERT: a pre-trained biomedical language representation model for biomedical text mining

BioBERT adapts BERT to biomedical text by pretraining on large-scale biomedical corpora, boosting NER, relation extraction, and question answering.

As biomedical literature grows, text mining becomes increasingly important, but applying general-domain NLP models directly to biomedical text suffers from a word-distribution shift. BioBERT adapts the BERT language model by pretraining it on large-scale biomedical corpora while keeping nearly the same architecture across tasks. It substantially outperforms BERT and prior state-of-the-art on biomedical named entity recognition, relation extraction, and question answering. The authors release pretrained weights and fine-tuning code freely.

Based on: BioBERT: a pre-trained biomedical language representation model for biomedical text mining · Bioinform.

HighlightCurated by Aramai Editorial

Learning without Forgetting

Introduces Learning without Forgetting, which adds new tasks to a CNN using only new-task data while preserving performance on old tasks.

As tasks accumulate, storing and retraining on all prior data becomes infeasible, and old-task data may be unavailable when adding new CNN capabilities. Learning without Forgetting trains the network on only new-task data while preserving the original capabilities. It outperforms feature extraction and fine-tuning, and matches multitask learning that relies on the unavailable old data. Notably, it can even replace fine-tuning to improve new-task performance.

Based on: Learning without Forgetting · IEEE Transactions on Pattern Analysis and Machine Intelligence

HighlightCurated by Aramai Editorial

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

ALBERT introduces two parameter-reduction techniques and an inter-sentence coherence loss to scale BERT pretraining with less memory and faster training.

Scaling up model size in language representation pretraining tends to improve downstream performance but eventually runs into GPU/TPU memory limits and longer training times. ALBERT proposes two parameter-reduction techniques that cut memory use and speed up BERT training, allowing it to scale far better than the original. It also adds a self-supervised loss modeling inter-sentence coherence, which helps tasks with multi-sentence inputs. The best model sets new state-of-the-art results on GLUE, RACE, and SQuAD while using fewer parameters than BERT-large.

Based on: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations · International Conference on Learning Representations

HighlightCurated by Aramai Editorial

The Carbohydrate-Active EnZymes database (CAZy): an expert resource for Glycogenomics

The CAZy database is an expert, knowledge-based resource classifying enzymes that build and break down complex carbohydrates and glycoconjugates.

CAZy is a knowledge-based database specializing in enzymes that build and break down complex carbohydrates and glycoconjugates. As of September 2008 it described families of glycoside hydrolases, glycosyltransferases, polysaccharide lyases, carbohydrate esterases, and carbohydrate-binding modules, defined from characterized proteins and populated by similar sequences. Its classification reflects structural features, reveals evolutionary relationships, and frames mechanistic properties, aiding functional annotation in genome projects.

Based on: The Carbohydrate-Active EnZymes database (CAZy): an expert resource for Glycogenomics · Nucleic Acids Res.