Book Review: Deep Reinforcement Learning Hands-On

Reinforcement learning (RL) is a hugely popular area of deep learning, and many data scientists are exploring this AI technology to broaden their skillet to include a number of important problem domains like chatbots, robotics, discrete optimization, web automation and much more.

As a result of this wide-spread interest in RL, there are many available educational resources specifically tailored to this class of deep learning – boot camps, training certificates, educational specializations, etc.

But if you’re a data scientist who has been programming in Python (with object oriented features) for a while, and has some experience with other forms of deep learning using a framework like TensorFlow, then maybe this new book, “Deep Reinforcement Learning Hands-On,” by Maxim Lapan from Packt, might be a great way to kick-start yourself into becoming productive with RL.

Reinforcement Learning RL development is being driven by a number of large companies and research groups, including Google, Microsoft, and Facebook.

RL requires considerable investment in research as the field is growing to enable data scientists to be able to take prescribed methods and apply them to a problem domain.

This is similar to how the states of computer vision and NLP were several years ago.

A good portion of the AI research papers appearing on the arXiv.

org pre-print server deal with RL.



push(function() { googletag.

display(div-gpt-ad-1439400881943-0); }); As a journalist, I’m seeing that the field of RL is attracting lots of attention, both from researchers and practitioners.

Fortunately, this hands-on book helps readers to understand RL methods using real-life problems, and make the exciting RL field accessible to a much wider audience than just research groups or large AI companies.

The emphasis of the book is to enable readers to feel that modern RL is an approachable, practical, and useful aspect of machine learning.

The book will help readers understand RL methods using real-life, relatable problems.

Information about recent research is widely available via arXiv.

org and research blogs, but it is too specialized and grounded with mathematics for most data scientists to easily understand.

If readers are new to this field, these papers might not be the best source of information, at least not early on.

This book gets around such complexities by addressing the obvious lack of practical and structured information about RL methods and approaches.

New in the Second Edition The second edition is devoted to the very latest RL tools and techniques, focusing on new innovations in this emerging field.

It includes six new chapters that give readers the hands-on ability to code intelligent learning agents to perform a range of practical tasks.

New topics include robotics and multiple agent learning.

The chapter on trust regions has been updated with the new SAC algorithm.

The author has also developed a library for easily interfacing the OpenAI environment which was used all through the first edition but is now properly documented.

After a clear definition of RL and associated nomenclature, the Gym library and its classes are explained.

Gym is a rich library used widely by RL practitioners and the book’s code depends on it.

A straight-forward and impressive RL application using Gym is given and explained carefully.

Fun topics include – using the TextWorld environment from Microsoft Research to solve text-based interactive fiction games, solving discrete optimization problems (showcased using Rubik’s Cube), and learning to apply RL methods to the robotics domain.

The book then dives into an important aspect of RL methods: advanced exploration.

Several modern exploration techniques are described, implemented, and compared.

Readers will also discover basic methods applied to a multi-agent environment.

All examples have been updated for PyTorch 1.


A good introduction to PyTorch is given in Chapter 3.

PyTorch Ignite has also been introduced to make the RL coding more concise.

To demonstrate the breadth of coverage of the subject, here are the chapters included in the book: Chapter 1 – What is Reinforcement Learning?Chapter 2 – OpenAI GymChapter 3 – Deep Learning with PyTorchChapter 4 – The Cross-Entropy MethodChapter 5 – Tabular Learning and the Bellman EquationChapter 6 – Deep Q-NetworksChapter 7 – Higher-Level RL LibrariesChapter 8 – DQN ExtensionsChapter 9 – Ways to Speed up RL MethodsChapter 10 – Stocks Trading Using RLChapter 11 – Policy Gradients Chapter 12 – The Actor-Critic MethodChapter 13 – Asynchronous Advantage Actor-CriticChapter 14 – Training Chatbots with RLChapter 15 – The TextWorld EnvironmentChapter 16 – Web NavigationChapter 17 – Continuous Action SpaceChapter 18 – RL in RoboticsChapter 19 – Trust Regions – TRPO, PPO, and ACKTRChapter 20 – Black-Box Optimization in RLChapter 21 – Advanced ExplorationChapter 22 – Beyond Model-Free Chapter 23 – AlphaGo ZeroChapter 24 – RL in Discrete OptimizationChapter 25 – Multi-agent RL The book is targeted toward readers with a fluency in Python.

Basic deep learning approaches should be familiar to readers and some practical experience in DL will be helpful.

This book is a complete introduction to deep reinforcement learning (RL) and requires no background in RL.

This is a very comprehensive book covering a range of RL techniques.

With nearly 800 pages, it will also weigh down your bookshelf a bit.

It comes with a GitHub repo containing all the code described in the book.

Great place to start building your RL skillet.

Contributed by Daniel D.

Gutierrez, Editor-in-Chief and Resident Data Scientist for insideBIGDATA.

In addition to being a tech journalist, Daniel also is a consultant in data scientist, author, educator and sits on a number of advisory boards for various start-up companies.

  Sign up for the free insideBIGDATA newsletter.


Leave a Reply