You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: lectures/markov_chains_II.md
+7-8Lines changed: 7 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -71,7 +71,7 @@ that
71
71
The stochastic matrix $P$ is called **irreducible** if all states communicate;
72
72
that is, if $x$ and $y$ communicate for all $(x, y)$ in $S \times S$.
73
73
74
-
```{prf:example}
74
+
````{prf:example}
75
75
:label: mc2_ex_ir
76
76
For example, consider the following transition probabilities for wealth of a
77
77
fictitious set of households
@@ -81,8 +81,6 @@ fictitious set of households
81
81
:align: center
82
82
```
83
83
84
-
85
-
86
84
We can translate this into a stochastic matrix, putting zeros where
87
85
there's no edge between nodes
88
86
@@ -97,7 +95,7 @@ $$
97
95
98
96
It's clear from the graph that this stochastic matrix is irreducible: we can eventually
99
97
reach any state from any other state.
100
-
```
98
+
````
101
99
102
100
We can also test this using [QuantEcon.py](http://quantecon.org/quantecon-py)'s MarkovChain class
103
101
@@ -110,7 +108,7 @@ mc = qe.MarkovChain(P, ('poor', 'middle', 'rich'))
110
108
mc.is_irreducible
111
109
```
112
110
113
-
```{prf:example}
111
+
````{prf:example}
114
112
:label: mc2_ex_pf
115
113
116
114
Here's a more pessimistic scenario in which poor people remain poor forever
@@ -122,7 +120,7 @@ Here's a more pessimistic scenario in which poor people remain poor forever
122
120
123
121
This stochastic matrix is not irreducible since, for example, rich is not
124
122
accessible from poor.
125
-
```
123
+
````
126
124
127
125
Let's confirm this
128
126
@@ -279,7 +277,7 @@ In any of these cases, ergodicity will hold.
279
277
280
278
### Example: a periodic chain
281
279
282
-
```{prf:example}
280
+
````{prf:example}
283
281
:label: mc2_ex_pc
284
282
285
283
Let's look at the following example with states 0 and 1:
@@ -301,7 +299,8 @@ The transition graph shows that this model is irreducible.
301
299
```
302
300
303
301
Notice that there is a periodic cycle --- the state cycles between the two states in a regular way.
304
-
```
302
+
````
303
+
305
304
Not surprisingly, this property
306
305
is called [periodicity](https://stats.libretexts.org/Bookshelves/Probability_Theory/Probability_Mathematical_Statistics_and_Stochastic_Processes_(Siegrist)/16%3A_Markov_Processes/16.05%3A_Periodicity_of_Discrete-Time_Chains).
0 commit comments