Highlight

A Survey on Vision Transformer

Surveys transformer models adapted from NLP to computer vision, categorizing them by task and analyzing their advantages, disadvantages, and open challenges.

The transformer, a self-attention-based deep network first used in natural language processing, has increasingly been adapted to computer vision thanks to strong representation capabilities. On many benchmarks, transformer-based models match or surpass CNNs and RNNs while needing less vision-specific inductive bias. This paper reviews vision transformers by categorizing them across tasks, including backbone networks, high/mid-level vision, low-level vision, and video, and also covers efficient transformers for real devices, the self-attention mechanism, and future directions.

Based on: A Survey on Vision Transformer · IEEE Transactions on Pattern Analysis and Machine Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

A New Kind of Science

Analyzes a compiled US dataset of hazard losses from 1975-1998, examining temporal and spatial patterns of deaths, damages, and state disaster proneness.

A nationwide dataset of losses from 1975 to 1998 was compiled to assess disaster trends. Temporal patterns of deaths, injuries, and monetary damages are examined by year, together with a spatial assessment of statewide totals and explanations for major disasters. Losses are normalized by population, land area, and GDP, and an overall hazard score ranks states by proneness, from highest (Florida, Texas, California) to lowest. The work calls for new vulnerability science, a national hazard events and losses database, and a rethinking of how vulnerabilities are managed.

Based on: A New Kind of Science · Artificial Life

Curated by Aramai Editorial

Read summary →
Highlight

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Compares four methods for propagating gradients through stochastic or hard-nonlinear neurons, including the straight-through estimator.

Stochastic neurons and hard non-linearities are useful in deep learning but hard to back-propagate through. The paper compares four solution families: an unbiased REINFORCE-style estimator for binary stochastic neurons; a decomposition into a stochastic binary part plus a smooth differentiable approximation; injecting noise into a differentiable graph; and the straight-through estimator, which copies the output gradient to the sigmoid argument. The motivating setting is conditional computation, where sparse gaters switch off large parts of a network to cut compute.

Based on: Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

Simple online and realtime tracking

Introduces SORT, a simple online real-time multi-object tracker using the Kalman filter and Hungarian algorithm, driven by detection quality.

This paper presents a pragmatic approach to multiple object tracking that associates objects efficiently for online, real-time use. It identifies detection quality as a key factor in tracking performance, finding that changing the detector can improve tracking by up to 18.9%. Although it relies only on a rudimentary combination of familiar techniques—the Kalman filter and the Hungarian algorithm—the tracker attains accuracy comparable to state-of-the-art online trackers. Thanks to its simplicity, it updates at 260 Hz, over 20 times faster than other leading trackers.

Based on: Simple online and realtime tracking · International Conference on Information Photonics

Curated by Aramai Editorial

Read summary →
Highlight

Qwen Technical Report

Introduces Qwen, a series of large language models including base, chat (RLHF), coding, and math variants with strong tool-use and agent skills.

This report introduces Qwen, the first release in a series of large language models of varying sizes. It includes Qwen base pretrained models and Qwen-Chat, chat models fine-tuned with human alignment such as RLHF. Base models perform strongly across many downstream tasks, while chat models are highly competitive and show advanced tool-use and planning for agents, even rivaling larger models on tasks like using a code interpreter. Specialized Code-Qwen and Math-Qwen-Chat variants beat open-source models and trail proprietary ones only slightly.

Based on: Qwen Technical Report · arXiv.org

Curated by Aramai Editorial

Read summary →
Highlight

A Survey of Collaborative Filtering Techniques

A comprehensive survey of collaborative filtering recommender techniques across memory-based, model-based, and hybrid categories.

Collaborative filtering (CF), a leading approach to recommender systems, uses the known preferences of a group of users to predict unknown preferences of others. The survey introduces CF tasks and their main challenges—data sparsity, scalability, synonymy, gray sheep, shilling attacks, and privacy—with possible solutions. It groups CF methods into three categories: memory-based, model-based, and hybrid algorithms that combine CF with other techniques, with representative examples and analysis of their predictive performance. It offers a roadmap for research and practice.

Based on: A Survey of Collaborative Filtering Techniques · Advances in Artificial Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

Are Transformers Effective for Time Series Forecasting?

Questions Transformers for long-term time series forecasting and shows simple one-layer linear models (LTSF-Linear) often beat them.

Amid a surge of Transformer methods for long-term time series forecasting (LTSF), this work questions the approach. Transformers capture semantic correlations in long sequences, but forecasting needs temporal relations over ordered continuous points, and permutation-invariant self-attention loses temporal information despite positional encoding. To test this, the authors introduce LTSF-Linear, a set of one-layer linear models. On nine real datasets these simple models beat sophisticated Transformer methods in every case, often by a wide margin.

Based on: Are Transformers Effective for Time Series Forecasting? · AAAI Conference on Artificial Intelligence

Curated by Aramai Editorial

Read summary →
Highlight

Neural Graph Collaborative Filtering

Proposes NGCF, injecting collaborative signal into user/item embeddings by propagating them over the user-item bipartite graph.

User and item embeddings are central to modern recommenders, but methods from matrix factorization to deep learning map from pre-existing features and fail to encode the collaborative signal in user-item interactions. Neural Graph Collaborative Filtering (NGCF) integrates the user-item bipartite graph into the embedding process by propagating embeddings over it, modeling high-order connectivity. On three public benchmarks it significantly improves over state-of-the-art models like HOP-Rec and Collaborative Memory Network, with analysis confirming the value of embedding propagation.

Based on: Neural Graph Collaborative Filtering · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Curated by Aramai Editorial

Read summary →
Highlight

Verb Semantics and Lexical Selection

Proposes a verb semantics representation scheme for lexical selection in machine translation, tested on English and Chinese verbs.

This paper studies how verbs are semantically represented in computer systems and how that affects lexical selection in machine translation (MT). Examining English and Chinese verbs, the authors argue lexical selection must draw on sentence interpretation plus selection restrictions on a verb's arguments. They propose a novel representation scheme and compare it with selection-restriction representations in transfer-based MT, aligning it with knowledge-based MT. Experiments show the scheme lets inexact matches still yield correct lexical selection.

Based on: Verb Semantics and Lexical Selection · Annual Meeting of the Association for Computational Linguistics

Curated by Aramai Editorial

Read summary →
Highlight

Teaching Machines to Read and Comprehend

Creates large-scale supervised reading comprehension data and attention-based neural networks that read documents and answer questions.

Teaching machines to read natural language documents remains challenging, and machine reading systems can be tested by answering questions about documents they have seen, but large-scale training and test datasets for this evaluation have been missing. This work defines a new methodology that resolves this bottleneck and provides large-scale supervised reading comprehension data. Using it, the authors develop a class of attention-based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

Based on: Teaching Machines to Read and Comprehend · Neural Information Processing Systems

Curated by Aramai Editorial

Read summary →
Highlight

Convolutional neural networks: an overview and application in radiology

A review of convolutional neural network fundamentals and their application to radiological imaging tasks, plus challenges and mitigations.

Convolutional neural networks (CNNs) have become dominant in computer vision and are drawing interest across fields, including radiology. A CNN automatically learns spatial feature hierarchies through backpropagation using convolution, pooling, and fully connected layers. This review explains CNN basics and surveys their application to radiological tasks while discussing challenges and future directions. It also covers two obstacles—small datasets and overfitting—and techniques to mitigate them, aiming to augment radiologist performance and improve patient care.

Based on: Convolutional neural networks: an overview and application in radiology · Insights into Imaging

Curated by Aramai Editorial

Read summary →
Highlight

Geometric Deep Learning: Going beyond Euclidean data

Reviews geometric deep learning, extending neural networks to non-Euclidean data such as graphs and manifolds.

Many scientific fields study data with non-Euclidean structure, including social networks, sensor networks, brain functional networks, genetic regulatory networks, and meshed surfaces. Such geometric data are often large and complex, making them natural targets for machine learning, especially deep neural networks that have proven powerful in vision, NLP, and audio. However, these tools have been most successful on Euclidean or grid-like data where structural invariances are built into the networks, motivating methods that generalize deep learning to non-Euclidean domains.

Based on: Geometric Deep Learning: Going beyond Euclidean data · IEEE Signal Processing Magazine

Curated by Aramai Editorial

Read summary →