Course_OpenAIBaselines_Unit0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/robotignite_logo_text.png\" width=\"700\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using OpenAI with ROS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unit 0: Introduction to the Course"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background:green;color:white;\">SUMMARY</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Estimated time to completion: <b>10 minutes</b><br><br>\n",
    "This unit is an introduction to the Using OpenAI with ROS Course. You'll have a quick preview of the contents you are going to cover during the course, and you will also view a practical demo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background:green;color:white;\">END OF SUMMARY</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What's this course about?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OpenAI-Gym makes comparing different AI learning algorithms much easier and robust. It allows easy bench-marking for AI and gives access to other people's algorithms to speed up the AI development process. It also provides an infrastructure that allows the re-usability of AI learning algorithms with any agent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But there's still one field in which working with OpenAI can be a little bit difficult and cumbersome: when trying to use OpenAI with ROS and Gazebo simulations. Within this field, there's not yet a common way of working, and each example has many particularities, which makes it difficult to reuse code and compare different projects. \n",
    "\n",
    "That's why we (<a href=\"http://www.theconstructsim.com\">**The Construct**</a>) have decided to create a structure of environments that will allow you to separate and organize each layer in a particular way that will make developing for OpenAI with ROS much easier to understand and make it easier reuse code. We have called this structure the **OpenAI ROS structure**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Within this course, you are going to learn how to set up this OpenAI ROS structure for different robots, and how to perform different kinds of trainings and learning algorithms with these robots."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Do you want to have a taste?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the proper introductions made, it is time to actually start. And... as we always do in the Robot Ignite Academy, let's start with practice! In the following example, you will be launching a training session, the goal of which is to train the CartPole how to stay upright. So... let's go!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background:#EE9023;color:white;\">Demo 1.1</p>\n",
    "<br>\n",
    "\n",
    "a) For this demo, we are going to be using the Qlearn algorithm (which you will set up in the next unit) in order to train our CartPole environment. To start the session, execute the following command.<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"float:left;background: #407EAF\">\n",
    "<tr>\n",
    "<th>\n",
    "<p class=\"transparent\">Execute in WebShell #1</p>\n",
    "</th>\n",
    "</tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "roslaunch cartpole_v0_training start_training.launch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should now see the Cartpole trying to maintain its upright position in the simulation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/cartpole_gif.gif\" width=\"600\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"color:red;\">**NOTE**: Whenever you want to finish with the Demo, hit **Ctrl+C** in the Web Shell where you launched the training session, in order to stop the training.</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"background:#EE9023;color:white;\">End of Demo 1.1</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What will you learn with this course?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basically, during this course, you will address the following topics:\n",
    "\n",
    "* Basic Concepts of the OpenAI ROS structure\n",
    "* Set up the OpenAI ROS structure for a CartPole environment\n",
    "* Train the Cartpole with the qlearn algorithm\n",
    "* Set up the OpenAI ROS structure for a Moving Cube environment\n",
    "* Train the Cube with the qlearn algorithm\n",
    "* Modifying the learning algorithm: DeepQ\n",
    "* Set up the OpenAI ROS structure for a Fetch Robot\n",
    "* Training Fetch robot with the HER algorithm from OpenAI baselines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How will you learn it?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will learn through hands-on experience from day one! Here you can see some of the simulations you are going to use during the course:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**CartPole Sim**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/cartpole.png\" width=\"500\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Cube Sim**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/cubli_rosds.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Fetch Robot**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/fetch.png\" width=\"400\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Hopper**:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/hopper.png\" width=\"400\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Minimum requirements for the course"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to be able to fully understand the contents of this course, it is highly recommended to have the following knowledge:\n",
    "\n",
    "* Basic ROS. You can get this knowledge by following the **ROS in 5 Days** course.\n",
    "* Basic Python.\n",
    "* Basic Unix shell knowledge.\n",
    "* Basic OpenAI knowledge. You can get this knowledge by following the **OpenAI Gym for Robotics 101** course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Special Thanks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This course wouldn't have been possible if it weren't for the OpenAI organization that generated this magnificent infrastructure. For more about them, please follow the links:\n",
    "<br><br>\n",
    "<img src=\"img/openai_logo_text.png\" width=\"500\"/>\n",
    "OpenAI: https://gym.openai.com/<br>\n",
    "OpenAI Gym Basic Tutorials: https://gym.openai.com/docs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Also, a big shout out to the guys at Erle Robotics for this magnificent OpenAI-Gym Add-on for Gazebo. Please go to their website and git for the code used and more amazing robotics products.<br>\n",
    "<img src=\"img/erle_logo_text.png\" width=\"600\"/>\n",
    "<br><br>\n",
    "ErleRobotics : http://erlerobotics.com/blog/<br>\n",
    "Gym Gazebo : https://github.com/erlerobot/gym-gazebo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* This course wouldn't have been possible without the knowledge and work of the <a href=\"http://www.ros.org/\">ROS Community</a>, <a href=\"https://www.osrfoundation.org/\">OSRF</a>, and <a href=\"http://gazebosim.org/\">Gazebo Team</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/ros_logo.jpg\" width=\"400\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"img/gazebo_logo.png\" width=\"200\"/>"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}