21 terms

BFS Vs. Dijkstra's

If you want to find paths with the minimum number of edges, then breadth-first search is the way to go. In other words, BFS is great for unweighted graphs. Dijkstra's algorithm generalizes BFS, but with weighted edges.

Shortest path

We call a path from vertex u to vertex v whose weight is minimum over all paths from u to v a shortest path, since if weights were distances, a minimum- weight path would indeed be the shortest of all paths u to v.

Precondition for Dijkstra's

Dijkstra's algorithm, is guaranteed to find shortest paths **only when all edge weights are nonnegative**When edge weights are required to be nonnegative, Dijkstra's algorithm is often the algorithm of choice.

Why are negative weights a problem?

There is a situation that we have to watch out for when edge weights can be negative. Suppose that there is a cycle whose total weight is negative. Then you can keep going around and around that cycle, decreasing the cost each time, and getting a path weight of − ∞. Shortest paths are undefined when the graph contains a negative-weight cycle.

What does Dijkstra's Algorithm do?

Dijkstra's algorithm finds a shortest path from a source vertex s to all other vertices. It keeps track of 2 things for each vertex v in a graph:

--> the weight of a shortest path from the source to v (v.dist)

--> the back-pointer for v (predecessor v.pred) on each shortest path.

--> the weight of a shortest path from the source to v (v.dist)

--> the back-pointer for v (predecessor v.pred) on each shortest path.

Sending runners Out vs. Dijkstra's

Dijkstra's algorithm works slightly differently. It treats all edges the same, so that when it considers the edges leaving a vertex, it processes the adjacent vertices together, and in no particular order.For example, when Dijkstra's algorithm processes the edges leaving vertex s, it declares that y.dist = 4, t.dist = 6, and y.pred and t.pred are both s—**so far**r* (please see Week7#2 for images). When Dijkstra's algorithm later considers the edge (y, t), it decreases the weight of the shortest path to vertex t that it has found so far, so that t.dist goes from 6 to 5 and t.pred switches from s to y.

Implementing Dijkstra's algorithm: What key data structure is needed?

Dijkstra's algorithm maintains a **min-priority queue** of vertices, with their dist values as the keys.t repeatedly extracts from the min-priority queue the vertex u with the minimum dist value of all those in the queue, and then it examines all edges leaving u. If v is adjacent to u and taking the edge (u, v) can decrease v's dist value, then we put edge (u, v) into the shortest-path tree (for now, at least), and adjust v.dist and v.pred. Let's denote the weight of edge (u, v) by w(u,v). We can encapsulate what we do for each edge in a relaxation step, with the following pseudocode (See crib sheet)

key aspect of while-loop

In the simulation with runners, once a vertex receives dist and pred values, they cannot change, but here a vertex could receive dist and pred values in one iteration of

NFINITY

the while-loop, and a later iteration of the while-loop for some other vertex u could change these values. For example, after edge (y, x) is relaxed in part (c) of the figure, the value of x.dist decreases from ∞ to 13, and x.pred becomes y. The next iteration of the while-loop in (part (d)) relaxes edge (t, x), and x.dist decreases further, to 8, and x.pred becomes t. In the next iteration (part (e)), edge (z, x) is relaxed, but this time x.dist does not change, because its value, 8, is less than

z.dist + w(z,x), which equals 12.

NFINITY

the while-loop, and a later iteration of the while-loop for some other vertex u could change these values. For example, after edge (y, x) is relaxed in part (c) of the figure, the value of x.dist decreases from ∞ to 13, and x.pred becomes y. The next iteration of the while-loop in (part (d)) relaxes edge (t, x), and x.dist decreases further, to 8, and x.pred becomes t. In the next iteration (part (e)), edge (z, x) is relaxed, but this time x.dist does not change, because its value, 8, is less than

z.dist + w(z,x), which equals 12.

When plementing Dijkstra's algorithm, you must take care that...

the min-priority queue is updated whenever a dist value decreases. For example, the implementation in the textbook uses a HeapAdaptablePriorityQueue, which has a method replaceKey. This method updates the min-priority queue's internal state to reflect the change in a vertex's key. The textbook also relies on a map from Vertex objects to Entry objects, which locate the vertex in the min-priority queue. It's a bit complicated.

Abstraction*

You might wonder, since the min-priority queue is going to be implemented by a min-heap, why not just have the Dijkstra's algorithm code keep track of the index of each vertex's index in the array that implements the heap? The problem is that the premise violates our goal of abstraction. Who is to say that a min-heap implements the min-priority queue? In fact, we could just use an unsorted array. Or any other implementation of a min-priority queue. The Dijkstra's algorithm code should know only that it's using a min-priority queue, and not how the min-priority queue is implemented.

Correctness of Algorithm

How do we know that, at termination (when the min-priority queue is empty because every vertex has been dequeued), v.dist is in fact the shortest-path weight for every vertex v? We rely on a loop invariant argument. We state a property that pertains to a loop, which we call the loop invariant, and we have to show three things about it:

1. The loop invariant is true before we enter the loop.

2. If the loop invariant is true upon starting an iteration of the loop, it remains true upon starting

the next iteration.

3. Theloopinvariant,combinedwiththereasonthatweexittheloop,yieldsthepropertythatwe

want to show.

1. The loop invariant is true before we enter the loop.

2. If the loop invariant is true upon starting an iteration of the loop, it remains true upon starting

the next iteration.

3. Theloopinvariant,combinedwiththereasonthatweexittheloop,yieldsthepropertythatwe

want to show.

Run time depends on

Type of data structure we use for priority queue:

- unsorted array

- min-heap

-Fibonacci heap

- unsorted array

- min-heap

-Fibonacci heap

Unsorted array for min-PQ run time

Θ(n ^2)

Why

We can use an unsorted array for the min-priority queue. Each insert and decreaseKey operation takes Θ(1) time. Each extractMin operation takes time O(q), where q is the number of vertices in the min-priority queue at the time. We know that q ≤ n always, and so we can say that each extractMin operation takes O(n) time. But wait—what about when q is small? The way to think about it is to look at the time for all n extractMin operations altogether. You can do the analysis that results in an arithmetic series in n, or you can do it in a simpler way. We have n operations, each taking O(n) time, and so the total time for all extractMin operations is O(n2). If we look at just the first n/2 extractMin operations, each of them has to examine at least n/2 vertices to find the one with the smallest dist value, and so just the first half of the extractMin operations take Ω(n2) time. Hence the total time for the n extractMin operations is Θ(n2), as is the total time for Dijkstra's algorithm.

What does it mean for a graph to be **dense**?

We say that a graph is dense if m = Θ(n^2), that is if on average every vertex n has Θ(n) neighbors.

Min-heap for min-PQ

What if we use a min-heap for the min-priority queue? Now each insert, extractMin, and decreaseKey operation takes O(lg n) time, and so the total time for Dijkstra's algorithm is O((n + m) lg n). If we make the reasonable assumption that m ≥ n − 1 (otherwise, we don't even have a connected graph), then that running time is O(m lg n). That's always better than using an unsorted array, right? No. We say that a graph is dense if m = Θ(n2), that is if on average every vertex has Θ(n) neighbors. In a dense graph, Dijkstra's algorithm runs in time O(n2 lg n), which is worse than using an unsorted array. It is somewhat embarrassing that the "clever" heap is beaten by the "naive" unsorted array.

Fibonacci-heap for min-PQ

This embarrassment lasted for decades, until the invention of a data structure called a Fibonacci heap that implements extractMin in O(lg n) amortized time (amortized over all the operations) and insert and decreaseKey in Θ(1) amortized time. With a Fibonacci heap, Dijkstra's algorithm runs in time O(n lg n + m), which is at least as good as using either an unsorted array or a min-heap. Fibonacci heaps are a little tricky to implement, and their hidden constant factors are a little worse than those for binary heaps, but they're not as hard to implement as some people seem to think.

A*

...

What is the idea behind A*?

Sometimes we are looking for the shortest path to a **known goal**. In these cases, we can use a generalization of Dijkstra's algorithm called A* search to find the shortest path without taking as much time as normal Dijkstra's

Main difference A* Vs Dijkstra

The key in the PQ for A** is always **distance so far plus the estimate of the distance remaining *

For this to find the shortest path we need two things.

1. the estimate of distance remaining must be **admissible**. This means that **we always underestimate the true remaining distance or get it exactly.** (i.e. "at least: it could be more but it certainly isn't less")

Because of the triangle inequality the Euclidean distance is an underestimate of the true travel distance. In fact, the ultimate underestimate of the remaining distance is 0. This leads to normal Dijkstra, because then we take the shortest distance traveled next.

2. the cost estimate is monotone. That means that if we extend the path (thus increasing the distance travelled so far), the total estimate is never less that the estimate before we extended the path. This is the generalization of "no negative edges." Our Euclidean distance plus distance so far estimate satisfies this, because if moving to an adjacent vertex increases the distance so far by d it cannot reduce the Euclidean distance by more than d. (It would reduce by exactly d if you were moving directly toward the goal.) Thus the sum will not go down. Using just Euclidean distance (without distance so far) would fail to be monotone, so would not be not guaranteed to give shortest paths.

Because of the triangle inequality the Euclidean distance is an underestimate of the true travel distance. In fact, the ultimate underestimate of the remaining distance is 0. This leads to normal Dijkstra, because then we take the shortest distance traveled next.

2. the cost estimate is monotone. That means that if we extend the path (thus increasing the distance travelled so far), the total estimate is never less that the estimate before we extended the path. This is the generalization of "no negative edges." Our Euclidean distance plus distance so far estimate satisfies this, because if moving to an adjacent vertex increases the distance so far by d it cannot reduce the Euclidean distance by more than d. (It would reduce by exactly d if you were moving directly toward the goal.) Thus the sum will not go down. Using just Euclidean distance (without distance so far) would fail to be monotone, so would not be not guaranteed to give shortest paths.