MATH4406: Control Theory

Control Theory: MATH4406 / MATH7406

Taught by: Yoni Nazarathy , Julia Kuhn (tutor), Brendan Patch (tutor)

Semester 2, 2014.

This is the 2014 course web-site.
The current course web-site (2016) is here.

Overview:

This is a course designed to introduce several aspects of mathematical control theory with a focus on Markov Decision Processes (MDP), also known as Discrete Stochastic Dynamic Programming. In this edition of the course (2014), the course mostly follows selected parts of Martin Puterman’s book, “Markov Decision Processes”. After understanding basic ideas of dynamic programming and control theory in general, the emphasis is shifted towards mathematical detail associated with MDP. The course begins with a background unit, ensuring students have enough knowledge of probability and Markov chains. This unit is followed by 9 additional units, of which 6 units focus on MDP and the other 3 units are aimed to giving the student a broader view of mathematical problems associated with control theory.

This web-page contains a detailed plan of the course as well as links to homework (HW) assignments and other resources. The course profile page, available to UQ students can be accessed through here. Some HW assignments may require computation. In that case, MATLAB is a natural tool for control theory, yet other software packages may be used as well (see software).

Key books and resources:

· [Put94] Puterman ML. Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994. UQ Library link (online).

· [AstKum14] Åström KJ, Kumar PR. "Control: A perspective." Automatica 50.1 (2014): 3-43. Link to paper.

· [BaurRied11] Bäuerle N, Rieder U. Markov Decision Processes with Applications to Finance. 2011. UQ Library link (online).

· [Ber00] Bertsekas DP. Dynamic Programming and Optimal Control. Vol 1. And Vol 2. Belmont, Mass: Athena Scientific; 2000. UQ Library link.

· [KaeLitCass98] Kaelbling LP, Littman ML, Cassandra AR, Planning and acting in partially observable stochastic domains. Artificial intelligence 101.1 (1998): 99-134. Link to paper.

· [Lue79] Luenberger DG. Introduction to Dynamic Systems: Theory, Models, and Applications. New York: Wiley; 1979. UQ Library link.

· [Mon82] Monahan GE. A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. Management Science 28.1 (1982): 1-16. Link to paper.

· [PerMar10] Albertos Pérez P, Mareels I. Feedback and Control for Everyone. Berlin: Springer Berlin Heidelberg; 2010. UQ Library link (online).

Other useful books and resources:

· The 2012 Edition of this course.

· [AntMich07] Antsaklis PJ, Michel AN. A Linear Systems Primer. Boston, Mass: Birkhäuser Boston; 2007. UQ Library link (online).

· [Kir70] Kirk DE. Optimal Control Theory: An Introduction. Englewood Cliffs, N.J: Prentice-Hall; 1970. UQ Library link.

· [KwaSiv91] Kwakernaak H, Sivan R, Modern Signals and Systems. UQ Library link

· [PolWil98] Jan Willem Polderman, Jan C. Willems, Introduction to Mathematical Systems Theory. Vol 26. New York; Berlin: Springer-Verlag; 1998. UQ Library link.

· [SivKwa72] Sivan R, Kwakernaak H. Linear Optimal Control Systems. New York: Wiley Interscience; 1972. UQ Library link On-line through IEEE

· [Son90] Sontag ED. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Vol 6. New York ; Berlin: Springer-Verlag; 1990. UQ Library link.

Schedule:

· The course is split up into 10 units (0—9). Unit 0 is probability background. The other units are about control.

· There are 13 weeks and 4 hourly meetings per week, they are referenced as x.y, where x is in {1,…,13} and y is in {1,2,3,4}. Note that units are NOT directly mapped to weeks. It is often that the HW and/or quiz associated with a part, is handled a few weeks later.

· There are 4 in class quizzes (45 min each), 7 HW assignments and an additional “course summary” assignment.

· Meetings of the form x.4 are typically (but not always) tutorial meetings where students may either ask questions about the HW or participate in a quiz.

· The teaching format varies based on the material and the speed and depth in which it is to be covered.

Control Theory: MATH4406 / MATH7406

Taught by: Yoni Nazarathy , Julia Kuhn (tutor), Brendan Patch (tutor)

Semester 2, 2014.

This is the 2014 course web-site. The current course web-site (2016) is here.

Overview:

Key books and resources:

· [Put94] Puterman ML. Markov Decision Processes: Discrete Stochastic Dynamic Programming, 1994. UQ Library link (online).

· [AstKum14] Åström KJ, Kumar PR. "Control: A perspective." Automatica 50.1 (2014): 3-43. Link to paper.

· [BaurRied11] Bäuerle N, Rieder U. Markov Decision Processes with Applications to Finance. 2011. UQ Library link (online).

· [Ber00] Bertsekas DP. Dynamic Programming and Optimal Control. Vol 1. And Vol 2. Belmont, Mass: Athena Scientific; 2000. UQ Library link.

· [KaeLitCass98] Kaelbling LP, Littman ML, Cassandra AR, Planning and acting in partially observable stochastic domains. Artificial intelligence 101.1 (1998): 99-134. Link to paper.

· [Lue79] Luenberger DG. Introduction to Dynamic Systems: Theory, Models, and Applications. New York: Wiley; 1979. UQ Library link.

· [Mon82] Monahan GE. A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. Management Science 28.1 (1982): 1-16. Link to paper.

· [PerMar10] Albertos Pérez P, Mareels I. Feedback and Control for Everyone. Berlin: Springer Berlin Heidelberg; 2010. UQ Library link (online).

Other useful books and resources:

· The 2012 Edition of this course.

· [AntMich07] Antsaklis PJ, Michel AN. A Linear Systems Primer. Boston, Mass: Birkhäuser Boston; 2007. UQ Library link (online).

· [Kir70] Kirk DE. Optimal Control Theory: An Introduction. Englewood Cliffs, N.J: Prentice-Hall; 1970. UQ Library link.

· [KwaSiv91] Kwakernaak H, Sivan R, Modern Signals and Systems. UQ Library link

· [PolWil98] Jan Willem Polderman, Jan C. Willems, Introduction to Mathematical Systems Theory. Vol 26. New York; Berlin: Springer-Verlag; 1998. UQ Library link.

· [SivKwa72] Sivan R, Kwakernaak H. Linear Optimal Control Systems. New York: Wiley Interscience; 1972. UQ Library link On-line through IEEE

· [Son90] Sontag ED. Mathematical Control Theory: Deterministic Finite Dimensional Systems. Vol 6. New York ; Berlin: Springer-Verlag; 1990. UQ Library link.

Schedule:

· The course is split up into 10 units (0—9). Unit 0 is probability background. The other units are about control.

· There are 13 weeks and 4 hourly meetings per week, they are referenced as x.y, where x is in {1,…,13} and y is in {1,2,3,4}. Note that units are NOT directly mapped to weeks. It is often that the HW and/or quiz associated with a part, is handled a few weeks later.

· There are 4 in class quizzes (45 min each), 7 HW assignments and an additional “course summary” assignment.

· Meetings of the form x.4 are typically (but not always) tutorial meetings where students may either ask questions about the HW or participate in a quiz.

· The teaching format varies based on the material and the speed and depth in which it is to be covered.

This is the 2014 course web-site.
The current course web-site (2016) is here.