Installation

You can install all the dependencies using Conda as follows:

git clone https://github.com/catarinaopires/gym-duckietown

cd gym-duckietown
conda create --name gym-duckietown python=3.8

conda activate gym-duckietown

pip install -e .

pip install pyglet==1.5.11  # otherwise the simulator crashes

You will need to activate your Conda environment before running any of the scripts:

conda activate gym-duckietown

Simple Usage - manual_control

There is a simple UI application which allows you to control the simulation or real robot manually. The manual_control.py application will launch the Gym environment, display camera images, and send actions (keyboard commands) back to the simulator or robot. You can specify which map file to load with the --map-name argument:

./manual_control.py --env-name Duckietown-udem1-v0

Other ways of running can be found in the repository, including training a reinforcement learning agent.

Study - manual_control

Taking a look into the manual_control.py file, we will learn how to use and customize the simulation. Note the examples are only an excerpt of the important parts of the manual_control.py file.

1. Create environment

We first need to create the Duckietown environment. To allow the customization of the simulation through the command line, we are going to add a parser of the needed parameters (with default values).

parser = argparse.ArgumentParser()

parser.add_argument("--env-name", default=None)

parser.add_argument("--map-name", default="udem1")

parser.add_argument("--distortion", default=False, action="store_true")

parser.add_argument("--camera_rand", default=False, action="store_true")

parser.add_argument("--draw-curve", action="store_true", help="draw the lane following curve")

parser.add_argument("--draw-bbox", action="store_true", help="draw collision detection bounding boxes")

parser.add_argument("--domain-rand", action="store_true", help="enable domain randomization")

parser.add_argument("--dynamics_rand", action="store_true", help="enable dynamics randomization")

parser.add_argument("--frame-skip", default=1, type=int, help="number of frames to skip")

parser.add_argument("--seed", default=1, type=int, help="seed")

args = parser.parse_args()

if args.env_name and args.env_name.find("Duckietown") != -1:

    env = DuckietownEnv(

        seed=args.seed,

        map_name=args.map_name,

        draw_curve=args.draw_curve,

        draw_bbox=args.draw_bbox,

        domain_rand=args.domain_rand,

        frame_skip=args.frame_skip,

        distortion=args.distortion,

        camera_rand=args.camera_rand,

        dynamics_rand=args.dynamics_rand,

    )

else:

    env = gym.make(args.env_name)

Then, we need to reset the simulation and render it.

obs = env.reset()
env.render()

2. Change agent's state

With the simulation created, we can now put the agent moving. First, we need to create a regular update function to continuously change the agent's state.

Note that the Agent needs to be implemented.

agent = Agent()

def update(dt):

    """

    This function is called at every frame to handle

    movement/stepping and redrawing

    """

    img = cv.cvtColor(obs, cv.COLOR_RGB2BGR)

    # action = [forward velocity, steering angle]

    action = agent.action(img) # customizable function

    obs, reward, done, info = env.step(action)

    print("step_count = %s, reward=%.3f" % (env.unwrapped.step_count, reward))

    if done:

        print("done!")

        env.reset()

        env.render()

    env.render()

pyglet.clock.schedule_interval(update, 1.0 / env.unwrapped.frame_rate)

3. Run app

Then, we can put the app running.

# Enter main event loop

pyglet.app.run()

env.close()

Customization

Depending on the desired capabilities of the agent, we can decide the action the agent needs to take in every update instance.

We can create an agent, and decide the next action according to the current aspects of the map, given by the observation and converting it into an image of a known color space to later perform a computer vision algorithm (e.g. color segmentation).

For example, we might want to make an autonomous agent that decides at each intersection what direction to follow and stop when detecting the stop sign. In the following example, we have a simplified agent that illustrates these conditions.

A simple agent could be:

class Agent:

    def __init__(self):

        self.actions = {"left": [0.2, 1.0], "right": [0.2, -1.0], "forward": [0.3, 0.0]}

    def action(self, img):

        """

        This function returns [forward velocity, steering angle]

        """

        if stop_sign_detected(img): # stop

            return [0.0, 0.0]

        else:

    if intersection_detected(img):

turn = get_next_direction(img) # "left", "right" or "foward"

             return self.actions[turn]