Sharif Digital Repository

Hexapod robots gives us the ability to study walking robots without facing problems such as stability in many aspects. It has a great deal of flexibility in movement even if a leg becomes malfunctioned. Radially symmetric (hexagonal) hexapods have more flexibility in movement than rectangular leg alignments. Because of symmetry they can move in any direction and time efficiently. Inverse kinematic problem of this kind of hexapods is solved through a modular mobile view considering six degrees of freedom for the trunk. Then typical tripod and wave gaits are analyzed and simulated through the presented formulation. In Reinforcement Learning algorithm for walking it is important how to make... 

The concept of smart grid (SG) has been introduced to improve the operation of the power systems. In modern structures of power systems, different reasons prompt researchers to suggest integrated analysis of multi-carrier energy systems. Considering synergy effects of the couplings between different energy carriers and utilising intelligent technologies for monitoring and controlling of energy flow may change energy system management in the future. In this paper, we propose a new solution which is entitled ‘smart energy hub’ (SEH) that models a multi-carrier energy system in a SG. SEH solutions allow homeowners to manage their energy consumption to reduce their electricity and gas bill. We... 

A repeated inflation-unemployment game within the linear-quadratic frame-work of Barro and Gordon is studied assuming that the government would like to cheat optimally and the finite heterogeneous population of private agents attempts to learn the government's targets using a reinforcement learning algorithm. Private agents are heterogeneous in their initial expectations of inflation rate but are assumed to utilize an identical anticipatory reinforcement learning process, namely Q-learning. In our heterogeneous setting, the only way for the private agents to achieve a zero value for their loss function, is for all of them to correctly anticipate the Nash equilibrium. It is of particular... 

Numerous disciplines are engaged in studies involving motor control. In this study, we have used a single link system with a pair of muscles that are excited with alpha and gamma signals to achieve an oscillatory movement with variable amplitude and frequency. The system is highly nonlinear in all its physical and physiological attributes. The major physiological characteristics of this system are simultaneous activation of a pair of nonlinear muscle-like-actuators for control purposes, existence of nonlinear spindle-like sensors and Golgi tendon organ-like sensor, actions of gravity and external loading. Transmission delays are included in the afferent and efferent neural paths to account... 

In this study, we have used a single link system with a pair of muscles that are excited with alpha and gamma signals to achieve an oscillatory movement with variable amplitude and frequency. This paper proposes a reinforcement learning method with an Actor-Critic architecture instead of middle and low level of central nervous system (CNS). The Actor in this structure is a two layer feedforward neural network and the Critic is a model of the cerebellum. The Critic is trained by State-Action-Reward-State-Action (SARSA) method. The system showed excellent tracking capability and after 280 epochs the RMS error for position and velocity profiles were 0.02, 0.04 radian and radian/sec,... 

Previous researches have shown the success of using reinforcement learning in solving combinatorial optimization problems. The main idea of these methods is to learn (near) optimal evaluation function to improve local searches and find (near) optimal solutions. Stage algorithm introduced by Boyan & Moore, is one of the most important algorithm in this area. In the other hand fuzzy methods have been used in all fields of science to solve problems but still never used in combinatorial optimization problems. In this paper we focus on Bin Packing Problem. We introduce two basic fuzzy algorithms (ALM and IDS) and then solve our problem with these fuzzy algorithms. We run ALM and IDS algorithms on... 

Vehicular edge computing is a progressing technology which provides processing resources to the internet of vehicles using the edge servers deployed at roadside units. Vehicles take advantage by offloading their computationintensive tasks to this infrastructure. However, concerning time-sensitive applications and the high mobility of vehicles, cost-efficient task offloading is still a challenge. This paper establishes a computation offloading strategy based on deep Q-learning algorithm for vehicular edge computing networks. To jointly minimize the system cost including offloading failure rate and the total energy consumption of the offloading process, the vehicle tasks offloading problem is... 

Nowadays developing autonomous systems, which can act in various environments and interactively perform their assigned tasks, are intensively desirable. These systems would be ready to be applied in different fields such as medicine, controller robots and social life. Reinforcement learning is an attractive area of machine learning which addresses these concerns. In large scales, learning performance of an agent can be improved by using hierarchical Reinforcement Learning techniques and temporary extended actions. The higher level of abstraction helps the learning agent approach lifelong learning goals. In this paper a new method is presented for discovering subgoal states and constructing... 

In this paper the problem of automatically discovering subtasks and hierarchies in reinforcement learning is considered. We present a novel method that allows an agent to autonomously discover subgoals and create a hierarchy from actions. Our method identifies subgoals by partitioning local state transition graphs. Options constructed for reaching these subgoals are added to action choices and used for accelerating the Q-Learning algorithm. Experimental results show significant performance improvements, especially in the initial learning phase  

In this paper we proposed a new real-time learning method. The engine of this method is a fuzzy-modeling technique which is called ink drop spread (IDS). IDS method has good convergence and is very simple and away from complex formula. The proposed method uses a reinforcement learning approach by an actor-critic system similar to Generalized Approximate Reasoning based Intelligent Control (GARIC) structure to adapt the IDS by delayed reinforcement signals. Our system uses Temporal Difference (TD) learning to model the behavior of useful actions of a control system. It is shown that the system can adapt itself, commencing with random actions. © 2008 IEEE  

One of the challenging problems in artificial intelligence is credit assignment which simply means distributing the credit among a group, such as a group of agents. We made an attempt to meet this problem with the aid of the reinforcement learning paradigm. In this paper, expertness framework is defined and applied to the multi-agent credit assignment problem. In the expertness framework, the critic agent, who is responsible for distributing credit among agents, is equipped with learning capability, and the proposed credit assignment solution is based on the critic to learn to assign a proportion of the credit to each agent, and the used proportion should be learned by reinforcement...