Unit 7 - Custom Controller always dies

marcin.czeczko · January 23, 2023, 1:44pm

Does anyone finished with success unit 7.

Once I configure my_robot_bringup controllers are dying on startup. I found out the joint_state_broadcaster had wrong type, once fixed it, only rrbot_controller process is dying, but can’t find any meaningful message describing what went wrong.

Thanks

Additional information:

Following Unit 7 of the course it is impossible to launch my_robot with custom controller. Seems that Exercise 7.17.1 has wrong yaml definition: type of joint_state_broadcaster is: joint_state_controller/JointStateController , but based on previous units it should be rather joint_state_broadcaster/JointStateBroadcaster. rrbot_controller processes just fails - console says Died without any other explanation.

marcin.czeczko · January 24, 2023, 10:06am

Just to give more context. Here’s what I tried so far (Of course the code of custom controller compiles without any errors and warnings):

The exercise 7.17.1 says about creating rrbot_controllers_custom.yaml in rrbot_controller package and in the excerise 7.18.1 config is referenced in rrbot.xacro - it seems that CMakeLists.txt of the rrbot_controller package has missing install of the conf folder, so I added it as below

install(
  DIRECTORY
    config
  DESTINATION
    share/${PROJECT_NAME}/
)

Then I build whole workspace using colcon build and did source install/setup.bash
Then I started the my_robot using ros2 launch my_robot_bringup my_robot.launch.py

robot was successfully spawned in gazebo but control spawners throws those errors

[spawn_entity.py-1] /opt/ros/galactic/lib/python3.8/site-packages/rclpy/qos.py:307: UserWarning: DurabilityPolicy.RMW_QOS_POLICY_DURABILITY_TRANSIENT_LOCAL is deprecated. Use DurabilityPolicy.TRANSIENT_LOCAL instead.
[spawn_entity.py-1]   warnings.warn(
[INFO] [spawn_entity.py-1]: process has finished cleanly [pid 7195]
[INFO] [spawner-3]: process started with pid [7227]
[ERROR] [spawner-3]: process has died [pid 7227, exit code 1, cmd '/opt/ros/galactic/lib/controller_manager/spawner joint_state_broadcaster --controller-manager /controller_manager --ros-args'].
[INFO] [spawner-4]: process started with pid [7237]
[ERROR] [spawner-4]: process has died [pid 7237, exit code 1, cmd '/opt/ros/galactic/lib/controller_manager/spawner forward_position_controller -c /controller_manager --ros-args'].

So I realised that something is wrong with rrbot_controllers_custom.yaml and it seem that wrong joint_state_broadcaster type were used.
It was: joint_state_controller/JointStateController and I changed it to the proper one: joint_state_broadcaster/JointStateBroadcaster
After rebuilding and sourcing install script I ran ros2 launch my_robot_bringup my_robot.launch.py again. This time broadcaster is loaded, forward_position_controller dying - I assuming it is expected as the controller yaml doesn’t have config for that controller.

[spawn_entity.py-1] /opt/ros/galactic/lib/python3.8/site-packages/rclpy/qos.py:307: UserWarning: DurabilityPolicy.RMW_QOS_POLICY_DURABILITY_TRANSIENT_LOCAL is deprecated. Use DurabilityPolicy.TRANSIENT_LOCAL instead.
[spawn_entity.py-1]   warnings.warn(
[INFO] [spawn_entity.py-1]: process has finished cleanly [pid 10802]
[INFO] [spawner-3]: process started with pid [10834]
[spawner-3] [INFO] [1674554044.686684125] [spawner_joint_state_broadcaster]: Loaded joint_state_broadcaster
[spawner-3] [INFO] [1674554044.713396444] [spawner_joint_state_broadcaster]: Configured and started joint_state_broadcaster
[INFO] [spawner-3]: process has finished cleanly [pid 10834]
[INFO] [spawner-4]: process started with pid [10844]
[ERROR] [spawner-4]: process has died [pid 10844, exit code 1, cmd '/opt/ros/galactic/lib/controller_manager/spawner forward_position_controller -c /controller_manager --ros-args'].

So, it seems that excercise should mention about removing the spawner of forward_position_controller from my_robot_bringup/launch/my_robot.launch.py to avoid this error.
So, I tried to load rrbot_controller using ROS control CLI, but it always ends with error

user:~/ros2_ws$ ros2 control load_controller rrbot_controller
Error loading controller, check controller_manager logs

I couldn’t find any logs and actually stuck with this exercise. I tried to use controller_manager spawner in launch file but of course the same error. So it seems that there is something missing in the rrbot_controller but can’t spot it.
Please adivse.

rzegers · January 24, 2023, 10:55am

Hello @marcin.czeczko,

thank your very much for reporting here. You are correct on all your observations!

The rrbot_controllers_custom.yaml file incorrectly names the type joint_state_broadcaster/JointStateController whereas the correct type is joint_state_broadcaster/JointStateBroadcaster so I updated that part.
I can confirm that the CMake instructions for installing the rrbot_controllers_custom.yaml file where missing. I also updated the course to add that information.
The launch file from Unit 2 that spawn the robot loads a forward_position_controller, so it is necessary to stop that controller before starting the rrbot_controller/RRBotController. I added instruction to create a launch file from scratch rather that re-using the launch file from unit 2.

I am still having the same error as you have, so I will have to look into this matter more deeply.

In the meantime I would like to have a look at your robot model file. Could you please post the contents of this file here?:

~/ros2_ws/src/ros2_control_course/unit2/rrbot_unit2/urdf/rrbot.xacro

Regards,

Roberto

marcin.czeczko · January 24, 2023, 11:29am

Hi @rzegers,
Here’s my ~/ros2_ws/src/ros2_control_course/unit2/rrbot_unit2/urdf/rrbot.xacro

<?xml version="1.0"?>
<!-- Revolute-Revolute Manipulator -->
<robot name="rrbot" xmlns:xacro="http://www.ros.org/wiki/xacro">

  <!-- Constants for robot dimensions -->
  <xacro:property name="PI" value="3.1415926535897931"/>
  <!-- arbitrary value for mass -->
  <xacro:property name="mass" value="1"/>
  <!-- Square dimensions (widthxwidth) of beams -->
  <xacro:property name="width" value="0.1"/>
  <!-- Link 1 -->
  <xacro:property name="height1" value="2"/>
  <!-- Link 2 -->
  <xacro:property name="height2" value="1"/>
  <!-- Link 3 -->
  <xacro:property name="height3" value="1"/>
  <!-- Size of square 'camera' box -->
  <xacro:property name="camera_link" value="0.05"/>
  <!-- Space btw top of beam and the each joint -->
  <xacro:property name="axel_offset" value="0.05"/>

  <!-- Import all Gazebo-customization elements, including Gazebo colors -->
  <xacro:include filename="$(find rrbot_unit2)/urdf/rrbot.gazebo"/>
  <!-- Import Rviz colors -->
  <xacro:include filename="$(find rrbot_unit2)/urdf/materials.xacro"/>

  <!-- Used for fixing robot to Gazebo 'base_link' -->
  <link name="world"/>

  <joint name="fixed" type="fixed">
    <parent link="world"/>
    <child link="link1"/>
  </joint>

  <!-- Base Link -->
  <link name="link1">
    <collision>
      <origin xyz="0 0 ${height1/2}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height1}"/>
      </geometry>
    </collision>

    <visual>
      <origin xyz="0 0 ${height1/2}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height1}"/>
      </geometry>
      <material name="orange"/>
    </visual>

    <inertial>
      <origin xyz="0 0 ${height1/2}" rpy="0 0 0"/>
      <mass value="${mass}"/>
      <inertia ixx="${mass / 12.0 * (width*width + height1*height1)}" ixy="0.0" ixz="0.0" iyy="${mass / 12.0 * (height1*height1 + width*width)}" iyz="0.0" izz="${mass / 12.0 * (width*width + width*width)}"/>
    </inertial>
  </link>

  <joint name="joint1" type="continuous">
    <parent link="link1"/>
    <child link="link2"/>
    <origin xyz="0 ${width} ${height1 - axel_offset}" rpy="0 0 0"/>
    <axis xyz="0 1 0"/>
    <dynamics damping="0.7"/>
  </joint>

  <!-- Middle Link -->
  <link name="link2">
    <collision>
      <origin xyz="0 0 ${height2/2 - axel_offset}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height2}"/>
      </geometry>
    </collision>

    <visual>
      <origin xyz="0 0 ${height2/2 - axel_offset}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height2}"/>
      </geometry>
      <material name="black"/>
    </visual>

    <inertial>
      <origin xyz="0 0 ${height2/2 - axel_offset}" rpy="0 0 0"/>
      <mass value="${mass}"/>
      <inertia ixx="${mass / 12.0 * (width*width + height2*height2)}" ixy="0.0" ixz="0.0" iyy="${mass / 12.0 * (height2*height2 + width*width)}" iyz="0.0" izz="${mass / 12.0 * (width*width + width*width)}"/>
    </inertial>
  </link>

  <joint name="joint2" type="continuous">
    <parent link="link2"/>
    <child link="link3"/>
    <origin xyz="0 ${width} ${height2 - axel_offset*2}" rpy="0 0 0"/>
    <axis xyz="0 1 0"/>
    <dynamics damping="0.7"/>
  </joint>

  <!-- Top Link -->
  <link name="link3">
    <collision>
      <origin xyz="0 0 ${height3/2 - axel_offset}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height3}"/>
      </geometry>
    </collision>

    <visual>
      <origin xyz="0 0 ${height3/2 - axel_offset}" rpy="0 0 0"/>
      <geometry>
        <box size="${width} ${width} ${height3}"/>
      </geometry>
      <material name="orange"/>
    </visual>

    <inertial>
      <origin xyz="0 0 ${height3/2 - axel_offset}" rpy="0 0 0"/>
      <mass value="${mass}"/>
      <inertia ixx="${mass / 12.0 * (width*width + height3*height3)}" ixy="0.0" ixz="0.0" iyy="${mass / 12.0 * (height3*height3 + width*width)}" iyz="0.0" izz="${mass / 12.0 * (width*width + width*width)}"/>
    </inertial>
  </link>

  <!-- Hokuyo Laser 

  <joint name="hokuyo_joint" type="fixed">
    <axis xyz="0 1 0"/>
    <origin xyz="0 0 ${height3 - axel_offset/2}" rpy="0 0 0"/>
    <parent link="link3"/>
    <child link="hokuyo_link"/>
  </joint>

  <link name="hokuyo_link">
    <collision>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <geometry>
        <box size="0.1 0.1 0.1"/>
      </geometry>
    </collision>

    <visual>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <geometry>
        <mesh filename="$(find rrbot_description)/meshes/hokuyo.dae"/>
      </geometry>
    </visual>

    <inertial>
      <mass value="1e-5"/>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <inertia ixx="1e-6" ixy="0" ixz="0" iyy="1e-6" iyz="0" izz="1e-6"/>
    </inertial>
  </link>

  -->

  <joint name="camera_joint" type="fixed">
    <axis xyz="0 1 0"/>
    <origin xyz="${camera_link} 0 ${height3 - axel_offset*2}" rpy="0 0 0"/>
    <parent link="link3"/>
    <child link="camera_link"/>
  </joint>

  <!-- Camera -->
  <link name="camera_link">
    <collision>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <geometry>
        <box size="${camera_link} ${camera_link} ${camera_link}"/>
      </geometry>
    </collision>

    <visual>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <geometry>
        <box size="${camera_link} ${camera_link} ${camera_link}"/>
      </geometry>
      <material name="red"/>
    </visual>

    <inertial>
      <mass value="1e-5"/>
      <origin xyz="0 0 0" rpy="0 0 0"/>
      <inertia ixx="1e-6" ixy="0" ixz="0" iyy="1e-6" iyz="0" izz="1e-6"/>
    </inertial>
  </link>

  <!-- generate an optical frame http://www.ros.org/reps/rep-0103.html#suffix-frames
   so that ros and opencv can operate on the camera frame correctly -->
  <joint name="camera_optical_joint" type="fixed">
    <!-- these values have to be these values otherwise the gazebo camera image won't
     be aligned properly with the frame it is supposedly originating from -->
    <origin xyz="0 0 0" rpy="${-pi/2} 0 ${-pi/2}"/>
    <parent link="camera_link"/>
    <child link="camera_link_optical"/>
  </joint>

  <link name="camera_link_optical"></link>

  <ros2_control name="MyRobotSystem" type="system">
    
    <hardware>
      <plugin>gazebo_ros2_control/GazeboSystem</plugin>
    </hardware>
    
    <joint name="joint1">
      <command_interface name="position">
        <param name="min">-6.28</param>
        <param name="max">6.28</param>
      </command_interface>
      <state_interface name="position"/>
      <state_interface name="velocity"/>
      <state_interface name="effort"/>
    </joint>

    <joint name="joint2">
      <command_interface name="position">
        <param name="min">-6.28</param>
        <param name="max">6.28</param>
      </command_interface>
      <state_interface name="position"/>
      <state_interface name="velocity"/>
      <state_interface name="effort"/>
    </joint>
    
  </ros2_control>

  <!-- Gazebo's ros2_control plugin -->
  <gazebo>
    <plugin filename="libgazebo_ros2_control.so" name="gazebo_ros2_control">
      <robot_sim_type>gazebo_ros2_control/GazeboSystem</robot_sim_type>
      <parameters>$(find rrbot_controller)/config/rrbot_controllers_custom.yaml</parameters>
    </plugin>
  </gazebo>

</robot>

marcin.czeczko · January 30, 2023, 1:09pm

Hi @rzegers - any luck with this issue ?

Thanks

rzegers · January 31, 2023, 11:02am

Hello @marcin.czeczko,

your .xacro file looks good.

I made some changes to the course material and it should be working now.

First change is to the yaml file: type of joint_state_broadcaster is joint_state_broadcaster/JointStateBroadcaster and not joint_state_controller/JointStateController and modify the CMakeLists.txt to install that .yaml file.

Second change is to create a totally new launch file that also spawns the robot to Gazebo.

Third change: I added a timer to delay the start of the controllers, in the launch file I changed this:

    RegisterEventHandler(
        event_handler=OnProcessExit(
            target_action=spawn_entity,
            on_exit=[joint_state_broadcaster_spawner],
        )
    ),

to this:

    RegisterEventHandler(
        OnExecutionComplete(
            target_action=spawn_entity,
            on_completion=[
                LogInfo(
                    msg='Spawn finished, waiting 10 seconds to start controllers.'),
                TimerAction(
                    period=10.0,
                    actions=[joint_state_broadcaster_spawner],
                )
            ]
        )
    ),

Finally I changed the course unit to use Gazebo Client instead of Gazebo’s web version, so now we must start Gazebo from the command line.

With these changes the custom controller loaded successfully.

The course material is updated with these changes. I think only change for you is to modify the launch file with the new version shown in the course. You also need to launch Gazebo now but this is also described at the end of the unit. Please let me know if you have any issues.

Apologies for the inconvenience,

Roberto

system · February 7, 2023, 11:03am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.