@@ -75,7 +75,7 @@ Possible interpretations of the graph include
75
75
76
76
* Minimum cost for supplier to reach a destination.
77
77
* Routing of packets on the internet (minimize time).
78
- * Etc ., etc.
78
+ * etc ., etc.
79
79
80
80
For this simple graph, a quick scan of the edges shows that the optimal paths are
81
81
@@ -97,7 +97,7 @@ For large graphs, we need a systematic solution.
97
97
98
98
Let $J(v)$ denote the minimum cost-to-go from node $v$, understood as the total cost from $v$ if we take the best route.
99
99
100
- Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example
100
+ Suppose that we know $J(v)$ for each node $v$, as shown below for the graph from the preceding example.
101
101
102
102
``` {figure} /_static/lecture_specific/short_path/graph2.png
103
103
@@ -134,7 +134,7 @@ the function $J$ satisfies
134
134
J(v) = \min_{w \in F_v} \{ c(v, w) + J(w) \}
135
135
```
136
136
137
- This is known as the * Bellman equation* , after the mathematician Richard Bellman.
137
+ This is known as the ** Bellman equation** , after the mathematician [ Richard Bellman] ( https://en.wikipedia.org/wiki/Richard_E._Bellman ) .
138
138
139
139
The Bellman equation can be thought of as a restriction that $J$ must
140
140
satisfy.
@@ -215,7 +215,7 @@ Q = np.array([[inf, 1, 5, 3, inf, inf, inf],
215
215
216
216
Notice that the cost of staying still (on the principle diagonal) is set to
217
217
218
- * np.inf for non-destination nodes --- moving on is required.
218
+ * ` np.inf ` for non-destination nodes --- moving on is required.
219
219
* 0 for the destination node --- here is where we stop.
220
220
221
221
For the sequence of approximations $\{ J_n\} $ of the cost-to-go functions, we can use NumPy arrays.
@@ -232,18 +232,14 @@ i = 0
232
232
233
233
while i < max_iter:
234
234
for v in nodes:
235
- # minimize Q[v, w] + J[w] over all choices of w
236
- lowest_cost = inf
237
- for w in nodes:
238
- cost = Q[v, w] + J[w]
239
- if cost < lowest_cost:
240
- lowest_cost = cost
241
- next_J[v] = lowest_cost
242
- if np.equal(next_J, J).all():
235
+ # Minimize Q[v, w] + J[w] over all choices of w
236
+ next_J[v] = np.min(Q[v, :] + J)
237
+
238
+ if np.array_equal(next_J, J):
243
239
break
244
- else:
245
- J[:] = next_J # Copy contents of next_J to J
246
- i += 1
240
+
241
+ J[:] = next_J # Copy contents of next_J to J
242
+ i += 1
247
243
248
244
print("The cost-to-go function is", J)
249
245
```
@@ -426,11 +422,7 @@ The minimization step is vectorized to make it faster.
426
422
427
423
``` {code-cell} python3
428
424
def bellman(J, Q):
429
- num_nodes = Q.shape[0]
430
- next_J = np.empty_like(J)
431
- for v in range(num_nodes):
432
- next_J[v] = np.min(Q[v, :] + J)
433
- return next_J
425
+ return np.min(Q + J, axis=1)
434
426
435
427
436
428
def compute_cost_to_go(Q):
0 commit comments