Sharif Digital Repository

Hierarchical reinforcement learning facilitates learning in large and complex domains by exploiting subtasks and creating hierarchical structures using these subtasks. Subtasks are usually defined through finding subgoals of the problem. Providing mechanisms for autonomous subgoal discovery and skill acquisition is a challenging issue in reinforcement learning. Among the proposed algorithms, a few of them are successful both in performance and also efficiency in terms of the running time of the algorithm. In this paper, we study four methods for subgoal discovery which are based on graph partitioning. The idea behind the methods proposed in this paper is that if we partition the transition... 

Hierarchical reinforcement learning (HRL) has had a vast range of applications in recent years. Preparing mechanisms for autonomous acquisition of skills has been a main topic of research in this area. While different methods have been proposed to achieve this goal, few methods have been shown to be successful both in performance and also efficiency in terms of time complexity of the algorithm. In this paper, a linear time algorithm is proposed to find subgoal states of the environment in early episodes of learning. Having subgoals available in early phases of a learning task, results in building skills that dramatically increase the convergence rate of the learning process. © 2009 Springer... 

This paper models the cyber-social system as a cyber-network of agents monitoring states of individuals in a social network. The state of each individual is represented by a social node, and the interactions among individuals are represented by a social link. In the cyber-network, each node represents an agent, and the links represent information sharing among agents. The agents make an observation of social states and perform distributed inference. In this direction, the contribution of this paper is threefold: First, a novel distributed inference protocol is proposed that makes no assumption on the rank of the underlying social system. This is significant as most protocols in the... 

In this paper we define critical graphs as minimal graphs that support a given set of rates for the index coding problem, and study them for both the one-shot and asymptotic setups. For the case of equal rates, we find the critical graph with minimum number of edges for both one-shot and asymptotic cases. For the general case of possibly distinct rates, we show that for one-shot and asymptotic linear index coding, as well as asymptotic non-linear index coding, each critical graph is a union of disjoint strongly connected subgraphs (USCS). On the other hand, we identify a non-USCS critical graph for a one-shot non-linear index coding problem. In addition, we show that the capacity region of... 

Single-time-scale distributed estimation of dynamic systems via a network of sensors/estimators is addressed in this letter. In single-time-scale distributed estimation, the two fusion steps, consensus and measurement exchange, are implemented only once, in contrast to, e.g., a large number of consensus iterations at every step of the system dynamics. We particularly discuss the problem of failure in the sensor/estimator network and how to recover for distributed estimation by adding new sensor measurements from equivalent states. We separately discuss the recovery for two types of sensors, namely α and β sensors. We propose polynomial-order algorithms to find equivalent state nodes in graph... 

The hierarchical structure of real-world problems has resulted in a focus on hierarchical frameworks in the reinforcement learning paradigm. Preparing mechanisms for automatic discovery of macro-actions has mainly concentrated on subgoal discovery methods. Among the proposed algorithms, those based on graph partitioning have achieved precise results. However, few methods have been shown to be successful both in performance and also efficiency in terms of time complexity of the algorithm. In this paper, we present a SCC-based subgoal discovery algorithm; a graph theoretic approach for automatic detection of subgoals in linear time. Meanwhile a parameter tuning method is proposed to find the... 

In this paper, we study the problem of learning the structure of Bayesian networks from data, which takes a dataset and outputs a directed acyclic graph. This problem is known to be NP-hard. Almost most of the existing algorithms for structure learning can be classified into three categories: constraint-based, score-based, and hybrid methods. The K2 algorithm, as a score-based algorithm, takes a random order of variables as input and its efficiency is strongly dependent on this ordering. Incorrect order of variables can lead to learning an incorrect structure. Therefore, the main challenge of this algorithm is strongly dependency of output quality on the initial order of variables. The main...