General Books on Reinforcement Learning

Start with the reading question

Reinforcement learning is a field focused on training agents to make decisions in complex environments. Its reading material is not one uniform collection: the source distinguishes general books, books with a control or operations research perspective, surveys, special journal issues, and a volume covering recent developments. The most useful first step is therefore not to choose a famous title at random. First identify what you want the reading to accomplish.

Trace a reading decision

The diagram represents a reading decision rather than a ranking of resources. A general book is not automatically better than a survey, and a special issue is not automatically a replacement for either one. Each category answers a different reading need identified by the source.

What a survey contributes

The source characterizes surveys as overview-oriented references. Their contribution is to help a reader see a field at a higher level instead of approaching reinforcement learning only through one book or one individual research paper. In a reading plan, a survey can therefore serve as an orientation point: it helps you understand the range of material before deciding which more specialized or detailed sources deserve closer reading.

General books for broad coverage

When your purpose is broad coverage of reinforcement learning, the source points to books by Szepesvári (2010), Bertsekas and Tsitsiklis (1996), Kaelbling (1993a), and Masashi Sugiyama and colleagues (2013). These are the source's named options for beginning with a general treatment rather than starting from a narrower disciplinary perspective or a curated journal collection.

Perspective-specific alternatives

The source separately names books associated with a control or operations research perspective: Si et al. (2004), Powell (2011), Lewis and Liu (2012), and Bertsekas (2012). This category is useful when your reading purpose is not simply broad coverage, but understanding reinforcement learning through one of those perspectives.

Machine Learning special issues

The source identifies three Machine Learning special issues focused on reinforcement learning: Sutton (1992), Kaelbling (1996), and Singh (2002). A special issue is a curated journal resource, so its role in a reading plan differs from that of a general book. It points the reader toward a collection associated with a particular research focus rather than presenting the category as one broad introductory book.

• Sutton (1992), identified by the source as a Machine Learning special issue focused on reinforcement learning.
• Kaelbling (1996), identified by the source as a Machine Learning special issue focused on reinforcement learning.
• Singh (2002), identified by the source as a Machine Learning special issue focused on reinforcement learning.

A worked selection example

Mistakes in resource selection

• Treating every named publication as a general book.

  The source distinguishes general books from special issues focused on reinforcement learning.

  Fix: Keep the general-book references separate from Sutton (1992), Kaelbling (1996), and Singh (2002), which the source identifies as special issues.

• Assuming that a perspective-specific book is the neutral starting point for every reader.

  The source assigns the control or operations research books to a specific perspective and names a different group for broad coverage.

  Fix: Use the general-book list when broad coverage is the immediate goal; use the perspective-specific list when that disciplinary lens is the goal.

• Inventing survey citations from an incomplete list.

  The source says surveys are overview-oriented references but does not name individual surveys in the excerpt.

  Fix: Report only the survey category from this source, and seek a fuller bibliography for named survey publications.

• Using the recent-development volume as if it were the only kind of reinforcement learning resource.

  The source assigns that volume to an overview of recent developments, not to the general-books category.

  Fix: Match the volume to the recent-development goal and choose another category when the purpose differs.

Practice the category match

1. A correct matching is: (1) general books, (2) control or operations research books, (3) surveys, (4) Machine Learning special issues, and (5) the volume edited by Wiering and van Otterlo (2012).

Reading plan takeaway

• Surveys are presented in the source as overview-oriented references, but the supplied excerpt does not name individual surveys.
• For broad coverage, the source names books by Szepesvári, Bertsekas and Tsitsiklis, Kaelbling, and Masashi Sugiyama and colleagues.
• For a control or operations research perspective, the source names Si, Powell, Lewis and Liu, and Bertsekas.
• The Machine Learning special-issue references are Sutton (1992), Kaelbling (1996), and Singh (2002).
• The source recommends the volume edited by Wiering and van Otterlo (2012) for an overview of recent developments.