OpenAI ROS Unit 2: Exercice 2.5c

In Exercice 2.5:
“c) Add a function in the Robot Environment file that changes the way in which the CartPole simulation is reset. For instance, make it reset to a position where the Pole is at an angle of 30º in respect to the Car”

creating a method (for example) init_at_30_pole() in the Robot Class environment --> CartPoleEnv, shall be called in the constructor or in the _env_setup method?

How can i call the method with the initial pole angle at 30° . It seems trivial but not for me :-).

Where is the correction of this exercise ?

thanks

Hi.
Its true , we might need to put solution if its necessary.

Here I explain how to do it:

  1. First have a look at the saty_up.py, file, in the mthod:

    def _set_init_pose(self):
         """
         Sets joints to initial position [0,0,0]
         :return:
         """
     
         self.check_publishers_connection()
     
         # Reset Internal pos variable
         self.init_internal_vars(self.init_pos)
         self.move_joints(self.pos)
    

Here we are initialising the self.pose and moving it to that place. This self.pose value is changed in the cartpole_env.py ( I show it next ), and we send that value through the move_joints (also defined in cartpole_env.py).

If you follow :

 init_internal_vars(self.init_pos)

Into the cartpole_env.py:

def _env_setup(self, initial_qpos):
    self.init_internal_vars(self.init_pos)
    self.set_init_pose()
    self.check_all_systems_ready()
    
def init_internal_vars(self, init_pos_value):
    self.pos = [init_pos_value]
    self.joints = None

def move_joints(self, joints_array):
    joint_value = Float64()
    joint_value.data = joints_array[0]
    rospy.logdebug("Single Base JointsPos>>"+str(joint_value))
    self._base_pub.publish(joint_value)

And this method:

 _set_init_pose

Is called each time we reset the simulation in the:

robot_gazebo_env.py

def _reset_sim(self):
    """Resets a simulation
    """
    if self.reset_controls :
        self.gazebo.unpauseSim()
        self.controllers_object.reset_controllers()
        self._check_all_systems_ready()
        **self._set_init_pose()**
        self.gazebo.pauseSim()
        self.gazebo.resetSim()
        self.gazebo.unpauseSim()
        self.controllers_object.reset_controllers()
        self._check_all_systems_ready()
        self.gazebo.pauseSim()
        
    else:
        self.gazebo.unpauseSim()
        
        self._check_all_systems_ready()
        **self._set_init_pose()**
        self.gazebo.pauseSim()
        self.gazebo.resetSim()
        self.gazebo.unpauseSim()
        
        self._check_all_systems_ready()
        self.gazebo.pauseSim()
    

    return True

def _set_init_pose(self):
    """Sets the Robot in its init pose
    """
    raise NotImplementedError()

There fore by changing it in the stay_up.py method, we are making that, when the system reset the simulation, it will place the configuration of the cartpole as we define in the higher level stap_up Task environment, using the middle level RobotEnv move_joints and so on.

This is made like this so that the person that designs the task, does have to know how the robot moves those joints or even when does this reset in the simulation.

So in your case you should change from:

def get_params(self):
    self.init_pos = rospy.get_param('/cartpole_v0/init_pos')

Its loading this parameter form the param server /cartpole_v0/init_pos. This is loaded and defined therefore in the file cartpole_qlearn_params.yaml

number_splits: 10 #set to change the number of state splits for the continuous problem and also the 
number of env_variable splits

**init_pos**: 0.0 # Position in which the base will start
wait_time: 0.1 # Time to wait in the reset phases

So after all these levels, the only thing you should change is a value in the parameter file that is lowered for the initial config. This is also made this way so that, you don’t have to even change code, just config parameters. This is ideal for testing different configurations when learning, to see which one works best.

Hope I solved your question. If you have any other question don’t hesitate to ask.

1 Like

I was having the same issue. When i was checking the code, i have noticed what @duckfrost said in his answer, nevertheless i still have a question. The move_joints function only appears to publish the position of the base joint, it seems like the the code was constructed with the intention to also publish information for the _pole_pub but i don’t see where the code does it, so with this in mind i tried to add these lines to the new _reset_sim function in the my_robot_gazebo_env.py code.

def _reset_sim(self):

        """Resets a simulation

        """ 

        **pole_joint_value = Float64()**

        **pole_joint_value.data = 30.0**

        if self.reset_controls :

            self.gazebo.unpauseSim()

            self.controllers_object.reset_controllers()

            self._check_all_systems_ready()

            self._set_init_pose()

            **self._pole_pub.publish(pole_joint_value)**

            self.gazebo.pauseSim()

            self.gazebo.resetSim()

            self.gazebo.unpauseSim()

            self.controllers_object.reset_controllers()

            self._check_all_systems_ready()

            self.gazebo.pauseSim()

            

        else:

            self.gazebo.unpauseSim()

            self._check_all_systems_ready()

            self._set_init_pose()

            **self._pole_pub.publish(pole_joint_value)**

            self.gazebo.pauseSim()

            self.gazebo.resetSim()

            self.gazebo.unpauseSim()

            

            self._check_all_systems_ready()

            self.gazebo.pauseSim()

        

        return True

I tried to make some other configuration using the same logic but none of them worked. They all where publishing the value 30.0 in the topic /cartpole_v0/pole_joint_velocity_controller/command every time the simulation started over but the pole just didn’t appear at a 30 degree angle. So I already tried the solution described here, and at first instance it appears to work but when i checked the topic /cartpole_v0/foot_joint_velocity_controller/command and after every reset the base moves to the position specified in the yaml file. I’m confused because the exercise says to change the location of the pole but not anything about the base location. So how can we change the initial position of the pole without changing the initial base position ? And also why the _pole_pub.publish seems to work but it doesn’t affect the actual location of the pole in the simulation and how we can use it.