Stochastic Modelling for Engineers
(last updated by Yoni Nazarathy: August 11, 2011)

This subject is designed to give engineering students both the basic tools in understanding probabilistic analysis and the ability to apply stochastic models to engineering applications. Electrical, robotic, biomedical and telecommunications engineering students shall find this subject a useful gateway for the understanding of analysis, design and control methods applied to random signals and random packets arising in communication, computation and life support systems. Industrial, mechanical and civil engineering students shall find this subject useful in presenting the basic ideas of reliability theory, stochastic modelling of logistic processes and basic concepts in risk analysis.

The subject assumes no prior knowledge of probability, yet requires a basic understanding of single and multi-variable calculus as well as basic linear algebra. All needed mathematical concepts are reviewed. The students are also to perform computational and simulation tasks. Here is some more information regarding software.

The subject is designed to be taught over 48 lecture hours (12 weeks). It is composed of 28 sections labelled 0a, 0b, 0c, 1a, 1b etc… Sections 1a and 1b are covered in the first week. Sections 2a and 2b are covered in the second week etc… (with a few exceptions). The assessment is composed of a 2 hour mid-term class test covering only Sections 1 through 4. In addition there are 6 homework projects which can be handed in either individually or in pairs. The mid-term test counts for 20% of the grade. The assignments are 80% of the grade. The assignment part of the grade is taken as the average of the best 4 assignments (thus essentially students are required to hand in only 4 assignments – but handing in more assignments is likely to improve the grade).

Recommended book:

The subject follows the following book closely:

· [LeGar08] A. Leon-Garcia, Probability, Statistics and Random Processes for Electrical Engineering, 3rd Edition 2008. (Previous editions can also be used).

Complementary Literature:

· [Gub06] J. A. Gubner, Probability and Random Processes for Electrical and Computer Engineer, 2006.

· [Ro10] S. M. Ross, Introduction to Probability Models, 10th Edition, 2010. (Previous editions are also OK).

· [Kul99] V. G. Kulkarni, Modelling, Analysis, Design, and Control of Stochastic Systems, 1999.

· [HoPoSt72] P. G. Hoel, S. C. Port, C. J. Stone, Introduction to Stochastic Processes, 1972.

· [Pap91] A. Papoulis, Probability, Random Variables and Stochastic Processes, 3rd Edtion, 1991. (Previous editions are also OK).

· [Dur99] R. Durret, Essentials of Stochastic Processes, 1999.

· [DeGr12] M. H. DeGroot, M. J. Schervish, Probability and Statistics, 4th Edition, 2012. (Previous editions are also OK).

Section	Outline	[LeGar08] Sections	Home Assignments	Comments and additional links
0a	Mathematical background: Sets, Counting, Series and Summations.	-	-	Math background notes (partially complete) Demo: InteractiveVennDiagrams Demo: BinomialTheorem Demo: ComparingBasicNumericalIntegrationMethods Demo: ConvolutionSum Wiki: Inclusion Exclusion Principle Wiki: List of trigonometric identities Wiki: Lists of integrals
0b	Mathematical background: Basic integration review, Complex Numbers, Fourier Transforms.	-
0c	Mathematical background: Convolutions, linear time invariant systems, frequency response.	-
1a	Introduction: Stochastic Modelling – an uncertain world – scalar uncertain results.	1.1 1.2 1.3	-	Job Shop Simulation Demo: ScalingsOfAGIG1QueueRealization Demo: LawOfLargeNumbersDiceRollingExample
1b	Introduction: Stochastic Modelling – an uncertain world – random signals, random packets and other models.	1.4 1.5
2a	Probability spaces: Generating Uniforms (simulation), Events, probability function, basic axioms.	2.7 2.1 2.2	HW Project #1 40 Probability Questions HW1partialSolution.pdf	Demo: MersenneTwisterAndFriends Demo: ApproximatingPiByTheMonteCarloMethod Demo: ConditionalProbability Wiki: Birthday Problem
2b	Probability spaces: Independence, conditional probability, law of total probability, Bayes’ rule.	2.4 2.5
3a	Random Variables: Discrete random variables, PMFs, CDFs, expectations: mean, moments, variance, Generation (simulation).	3.1 3.2 4.1 3.3		From Class: Aug18Class.nb From Class: Aug23Class.nb From Class: Aug24Class.nb From Class:Sep7Class.nb Demo: ConnectingTheCDFAndThePDF Demo: ChebyshevsInequalityAndTheWeakLawOfLargeNumbers
3b	Random Variables: Continuous random variables, PDFs (density), expectations: mean, moments, variance, Markov inequality.	4.1 4.2 4.3 4.6
3c	Random Variables: functions of random variables, inverse probability transform, Random number generation, solving basic problems by basic simulation.	4.5 4.9 3.6
4a	Useful (families) of discrete distributions: Discrete uniform, binomial, geometric, hyper-geometric, negative binomial, Poisson.	3.5		From Class: Aug31Class.nb Demo: BinomialProbabilityDistribution Demo: GeometricDistribution Demo: PoissonDistribution Demo: TheHypergeometricDistribution Demo: TheNegativeBinomialDistribution Demo: IllustratingTheUseOfDiscreteDistributions
4b	Useful (families) of continuous distributions: Uniform, exponential, normal (Gaussian), gamma, beta, Cauchy.	4.4		From Class: ClassSep8.nb Demo: TheContinuousUniformDistribution Demo: TheExponentialDistribution Demo: TheNormalDistribution Demo: AreaOfANormalDistribution Demo: TheGammaDistribution Demo: BetaDistribution NORMALTABLE
Test on Sections 1-4 test with solution.pdf
5a	Joint distributions: Joint distribution of pairs (discrete and continuous). Random vectors. Independence. Joint moments. Joint Gaussian.	5.1 6.1 5.2 5.4 5.5 5.9	HW Project #2 Joint Distributions	From Class: Sep15Class.nb From Class: ClassOfSep16.nb From Class: FromClassOct5.nb From Class: Oct7 .nb From Class: Oct12 .nb Wiki: Box Muller transform Demo: DiscreteMarginalDistributions Demo: VisualizingCorrelations Demo: JointDensityOfBivariateGaussianRandomVariables Demo: TheBivariateNormalAndConditionalDistributions Demo: JointDensityOfTrivariateGaussianRandomVariables
5b	Joint distributions: Conditional distributions, expectation and variance. Conditional distribution of Gaussians. Mean vector and covariance matrix. Jointly Gaussian random vectors.	5.7 5.9 6.3.1 6.4
5c	Joint distributions: Transformations and generation.	5.8 5.10 6.2 6.32 6.41
6a	Estimation: MAP and ML Estimators, Minimum MSE Linear Estimators and Minimum MSE Estimators.	6.5	HW Project #3 Estimation	Link to course notes of Nahum Shimkin on Estimation
6b	Estimation: Estimation using a vector observations, examples, parameter estimation, ML parameter estimation, Cramer-Rao (in brief).	6.5 8.2 8.3
7a	Transforms, Sums and long term averages: Transforms of random variables, distribution of i.i.d sums, using the DFT to calculate the distribution.	4.7 7.1 7.6	HW Project #4 Basic Random Processes	Demo: ConvolutionOfTwoDensities Demo: ConvolutionsOfShiftedDensities
7b	Transforms, Sums and long term averages: Law of large numbers, central limit theorem, renewal counting processes.	7.2 7.3 7.5
8a	Random process basics: Definitions, distribution, mean function, covariance function, multiple processes, simple examples (Binomial process etc..).	9.1 9.2 9.3		Demo: SimulatingThePoissonProcess
8b	Random processes basics: Poisson and related processes, Gaussian processes, Brownian motion (in brief), generating random processes.	9.4 9.5 9.10
9a	Signal Level Processes: Stationary processes (and wide sense stationary), cyclostationary processes, time averages and ergodic theorems (non-rigorous).	9.6 9.8	HW Project #5 Signal Processes HW Project #6 Optimum Linear Systems and the Kalman Filter	Demo: AutoregressiveMovingAverageSimulationFirstOrder
9b	Signal Level Processes: Fourier series and Karhunen-Loeve expansions.	9.9 6.3.4 9.9.1 9.10.2
10a	Signal Processing Basics: Power spectral density, response of linear systems to signals, sampling and modulation (in brief).	10.1 10.2 10.3		Demo: FrequencySpectrumOfANoisySignal
10b	Signal Processing Basics: Optimum linear systems, the Kalman filter.	10.4 10.5		Wiki: Wiener_filter Wiki: Kalman_filter
11a Cancelled	Packet Level Processes: Idea of Markov processes, discrete time Markov chains, steady state, brief description of classes and recurrence properties.	11.1 11.2 11.3	HW Project #6 Markov Chains and Queues cancelled	Demo: TransitionMatricesOfMarkovChains Demo: FiniteStateDiscreteTimeMarkovChains Moshe Haviv's Book on Queues
11b Cancelled	Packet Level Processes: Continuous time Markov chains, birth-death chains with queueing examples, numerical techniques (steady state solutions and simulation).	11.4 11.6 12.3 12.4
12a Cancelled	Queuing Theory: Terminology, Little’s formula, M/M/1 FCFS waiting time distribution.	12.1 12.2 12.32		Demo: SimulatingTheMM1Queue
12b cancelled	Queuing Theory: M/G/1 queue (in brief), Jackson Networks (in brief), simulation and data analysis of queuing systems.	12.6 12.9 12.10