@@ -69,7 +69,7 @@ Possible interpretations of the graph include
69
69
70
70
* Minimum cost for supplier to reach a destination.
71
71
* Routing of packets on the internet (minimize time).
72
- * Etc ., etc.
72
+ * etc ., etc.
73
73
74
74
For this simple graph, a quick scan of the edges shows that the optimal paths are
75
75
@@ -91,7 +91,7 @@ For large graphs, we need a systematic solution.
91
91
92
92
Let $J(v)$ denote the minimum cost-to-go from node $v$, understood as the total cost from $v$ if we take the best route.
93
93
94
- Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example
94
+ Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example.
95
95
96
96
``` {figure} /_static/lecture_specific/short_path/graph2.png
97
97
@@ -128,7 +128,7 @@ the function $J$ satisfies
128
128
J(v) = \min_{w \in F_v} \{ c(v, w) + J(w) \}
129
129
```
130
130
131
- This is known as the * Bellman equation* , after the mathematician Richard Bellman.
131
+ This is known as the ** Bellman equation** , after the mathematician [ Richard Bellman] ( https://en.wikipedia.org/wiki/Richard_E._Bellman ) .
132
132
133
133
The Bellman equation can be thought of as a restriction that $J$ must
134
134
satisfy.
@@ -209,7 +209,7 @@ Q = np.array([[inf, 1, 5, 3, inf, inf, inf],
209
209
210
210
Notice that the cost of staying still (on the principle diagonal) is set to
211
211
212
- * np.inf for non-destination nodes --- moving on is required.
212
+ * ` np.inf ` for non-destination nodes --- moving on is required.
213
213
* 0 for the destination node --- here is where we stop.
214
214
215
215
For the sequence of approximations $\{ J_n\} $ of the cost-to-go functions, we can use NumPy arrays.
@@ -226,18 +226,14 @@ i = 0
226
226
227
227
while i < max_iter:
228
228
for v in nodes:
229
- # minimize Q[v, w] + J[w] over all choices of w
230
- lowest_cost = inf
231
- for w in nodes:
232
- cost = Q[v, w] + J[w]
233
- if cost < lowest_cost:
234
- lowest_cost = cost
235
- next_J[v] = lowest_cost
236
- if np.equal(next_J, J).all():
229
+ # Minimize Q[v, w] + J[w] over all choices of w
230
+ next_J[v] = np.min(Q[v, :] + J)
231
+
232
+ if np.array_equal(next_J, J):
237
233
break
238
- else:
239
- J[:] = next_J # Copy contents of next_J to J
240
- i += 1
234
+
235
+ J[:] = next_J # Copy contents of next_J to J
236
+ i += 1
241
237
242
238
print("The cost-to-go function is", J)
243
239
```
@@ -420,11 +416,7 @@ The minimization step is vectorized to make it faster.
420
416
421
417
``` {code-cell} python3
422
418
def bellman(J, Q):
423
- num_nodes = Q.shape[0]
424
- next_J = np.empty_like(J)
425
- for v in range(num_nodes):
426
- next_J[v] = np.min(Q[v, :] + J)
427
- return next_J
419
+ return np.min(Q + J, axis=1)
428
420
429
421
430
422
def compute_cost_to_go(Q):
0 commit comments