HighlightCurated by Aramai Editorial

Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads

A paper characterizing agent memory systems for long-horizon tasks.

The authors present a system-oriented taxonomy of agent memory systems, profiling harness, and characterization of ten representative systems. They uncover design choices' impact on cost across write and read paths and derive system recommendations.

Based on: Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads · arXiv

HighlightCurated by Aramai Editorial

MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory

A framework for evaluating multimodal agent memory capabilities.

The authors introduce MemEye, a visual-centric evaluation framework for multimodal agent memory. They propose two dimensions to measure memory capabilities: the granularity of decisive visual evidence and how retrieved evidence must be used. The framework is applied to 13 memory methods across 4 VLM backbones, revealing that current architectures struggle to preserve fine-grained visual details.

Based on: MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory · arXiv

HighlightCurated by Aramai Editorial

Compressing Observation History into Agent Memory: Distilling Transformers into Recurrent Transformers

A research paper proposing a distillation approach to transfer the compression strategy of full-history transformers to recurrent variants.

The authors propose a method to compress observation history into agent memory using a distillation approach. This allows training recurrent latent robotic memories with linear-time complexity while narrowing the performance gap to full-history transformers. The method is designed for long-horizon streaming vision and robotics applications, such as map-free pose estimation.

Based on: Compressing Observation History into Agent Memory: Distilling Transformers into Recurrent Transformers · arXiv

HighlightCurated by Aramai Editorial

Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach

This paper evaluates the use of large language models for grading command-line examinations.

The study assesses four frontier LLMs (GPT, Claude Opus, Gemini, and GLM) in approximating expert judgment when grading short Linux/bash command responses. The models were tested on real responses from second-year Computer Engineering students and found that question complexity is a reliable predictor of the difficulty LLMs face in grading accurately.

Based on: Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approach · arXiv

HighlightCurated by Aramai Editorial

From vectors to knowledge graphs: A comprehensive analysis of modern retrieval-augmented generation architectures

A study on modern retrieval-augmented generation architectures.

The paper analyzes the evolution of retrieval-augmented generation (RAG) models from vector-based representations to knowledge graph-based ones. It provides a comprehensive overview of the current state-of-the-art in RAG architectures and their applications. The authors discuss the benefits and limitations of using knowledge graphs in RAG models.

Based on: From vectors to knowledge graphs: A comprehensive analysis of modern retrieval-augmented generation architectures · Computer Science Review

HighlightCurated by Aramai Editorial

Synthesizing scientific literature with retrieval-augmented language models

A specialized language model for answering scientific queries and synthesizing literature.

The paper introduces OpenScholar, a retrieval-augmented language model that assists scientists in synthesizing literature. It outperforms other large language models on a challenging multi-paper synthesis task and achieves citation accuracy comparable to human experts. The model's data store, retriever, and self-feedback inference loop improve off-the-shelf language models.

Based on: Synthesizing scientific literature with retrieval-augmented language models · Nature

HighlightCurated by Aramai Editorial

Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use

A study on the performance of a knowledge-graph tool use recipe with large language models.

The authors test a standard recipe for using knowledge graphs with large language models, observing a 'peak-then-collapse' pattern in performance. They identify four recurring failure modes and argue that interface feedback is a key difference from other tools. The study also explores the effect of self-distillation as a mitigation strategy.

Based on: Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use · arXiv

HighlightCurated by Aramai Editorial

GRASP: Plan-Guided Graph Retrieval with Adaptive Fusion and Reranking on Semi-Structured Knowledge Bases

A plan-guided graph retrieval method for semi-structured knowledge bases.

The paper proposes GRASP, a plan-guided graph retrieval system that uses adaptive fusion and reranking to improve the accuracy of retrieving relevant information from semi-structured knowledge bases. The approach combines a planning module with a retrieval module to adaptively select relevant subgraphs and fuse their representations. This allows for more accurate and efficient retrieval of relevant information.

Based on: GRASP: Plan-Guided Graph Retrieval with Adaptive Fusion and Reranking on Semi-Structured Knowledge Bases · arXiv

HighlightCurated by Aramai Editorial

ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments

A taxonomy-guided benchmark for agentic academic paper search in open literature environments.

This paper proposes ScholarQuest, a taxonomy-guided benchmark for evaluating agentic academic paper search systems. The benchmark aims to assess the ability of such systems to retrieve relevant papers based on complex queries and taxonomic relationships. The authors evaluate ScholarQuest using various evaluation metrics and compare it with existing benchmarks.

Based on: ScholarQuest: A Taxonomy-Guided Benchmark for Agentic Academic Paper Search in Open Literature Environments · arXiv

HighlightCurated by Aramai Editorial

Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection

A mechanism-oriented taxonomy for indirect linguistic encoding in language models.

This paper proposes a comprehensive taxonomy of indirect linguistic encoding for coded language detection using large language models. The taxonomy focuses on mechanisms rather than surface forms, aiming to improve the accuracy and robustness of LLM-based coded language detection systems. The authors provide a detailed description of their approach and its potential applications.

Based on: Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection · arXiv

HighlightCurated by Aramai Editorial

When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation

A framework that calibrates knowledge graph retrieval-augmented generation models.

The paper proposes Ca2KG, a causality-aware calibration framework for KG-RAG. It integrates counterfactual prompting and a panel-based re-scoring mechanism to improve calibration while maintaining predictive accuracy. Experiments on two QA datasets demonstrate the effectiveness of Ca2KG.

Based on: When to Trust: A Causality-Aware Calibration Framework for Accurate Knowledge Graph Retrieval-Augmented Generation

HighlightCurated by Aramai Editorial

SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering

A schema-aware cumulative process reward model for knowledge graph question answering.

The paper proposes SCPRM, a schema-aware cumulative process reward model for knowledge graph question answering. It aims to improve the performance of KGQA systems by incorporating schema information and cumulative rewards. The authors evaluate SCPRM on several benchmark datasets and demonstrate its effectiveness compared to existing methods.

Based on: SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering · arXiv