First set of lecture notes for exam studying

### Properties of Algorithms

Correct - always gives the right answer

Terminable - no infinite loops

Robust - handle unexpected inputs

Complexity - easily translates into code

Adaptable - easy to adapt

Efficient - low use of resources

### Why worst case?

Easier to analyze

Crucial to applications requiring an upper bound on performance

Average case is impractical (must know inputs and results)

### Primitive Operations

Evaluating an expression

Assigning a value to a variable

Indexing into an array

Calling a method

Returning from a method

### Measuring Algorithm Run Time (Actual vs # Ops)

Actual Time: problematic because it depends on hardware, software, and programmer.

Number of Operations: independent of hardware and software.

### Important Functions

Constant

Logarithmic: doubling input size takes one more unit of time

Fractional power: sqrt(n)

Linear

Linear Log: n log(n)

Quadratic

Polynomial

Exponential

Factorial

### Limit Rule

To see if F(x) is O(G(x)) you can take the limit of F(x)/G(x). If...

0 then F is O(g) but G is not O(f)

Infinity then G is O(f) but F is not O(g)

Finite then F is O(g) and G is O(f)

### Big Omega (define)

A lower bound on the growth rate of a function. If f is big Omega of g then f is asymptotically greater than or equal to g.

Ex. 5n^2 is big omega of n^2 as well as big omega of n.

### Recurrence relation

Used to analyze runtime of recursive algorithms.

A function F: N -> R+ defined by

1. base case: f(0), f(1), etc.

2. expression for f(n) in terms of previous values if not base case

### Closed form solution (define+find)

Does not depend on previous values of n.

1. Take recurrence relation and expand it.

2. Once you have a pattern, simplify to closed form.

### linear recursion (define)

When a method is defined so that it makes at most one recursive call each time it is invoked.

### recursion trace (define)

A tool for analyzing and visualizing recursion algorithms that involves drawing a box for each instance of the method and labeling it with the parameters.

### merge sort (pseudocode)

MergeSort (Array, first, last)

if (first < last -1) then

middle <- (first + last) / 2

MergeSort (T, first, middle)

MergeSort (T, middle + 1, last)

Merge(T, first, middle, middle+1, last)

Time Complexity: O(n log n)

### NP-Complete

No efficient algorithm has been found, but

the existence of efficient algorithms hasn't been ruled out.

ex. TSP, factorization of large primes

### stack (define)

Stores arbitrary objects in LIFO order

Methods:

Push() and Pop()

may have Top(), Size(), and isEmpty()

Exceptions: empty and full

### Logarithm Rules

log ac = log a + log c

log a/c = log a - log c

log a^c = c log a

log (of b) a = log (of d) a / log (of d) b

b ^ log a = a ^ log b

### queue (define)

Stores arbitrary objects in FIFO order

Methods:

enqueue(obj)

obj dequeue()

may have front(), size(), isEmpty()

Exception: empty and full

### array based queue

array of size N is circular fashion

two variables to track front and end -- f is front element, r is index immediately past the rear element

size: return (N - f + r)

isempty: return (f = r)

enqueue and dequeue are const

### list (define)

an ordered sequence of arbitrary objects -- not necessarily array implementation

Methods:

lookUp(i)

remove(i)

insert(o)

insertBefore(i,o)

May have: size(), isEmpty()

Implemented as contiguous array or linked memory.

### contiguous-memory list

Benefits:

random access to each element in O(1) time

no memory overhead - only list items stored

Problems:

array sizes are fixed - memory waste, not for dynamic apps

insert/remove in middle - massive data movement

### linked-list (vs contiguous)

Benefits:

insert/delete - no data mvmt

dynamic allocation of memory on as needed benefit

Problems:

no random access, ie. time is O(n)

### singly linked list

Nodes w/ two pieces:

1. an element

2. a pointer to next node

Head pointer

Tail pointer

### singly linked list w/ dummy

Dummy head = no element stored in head (null value) but head always points to the dummy.

Advantages:

all changes take place after the head

good for recursion

makes code more uniform

### duplicate list using dummy method **

duplicate(node p):

newHead <- newNode(dummy, duplicateAfter(p))

return newHead

duplicateAfter(node p)

if next(p) = null then

return null

else

return newNode(element(node(p)), duplicateAfter(next(p)))

### tail recursion vs general recursion

tail recursion:

recursive call is the last operation

easily expressed with while loops

ex. removeAfter()

general recursion:

additional code is called after recursion returns

can be expressed as a programmer defined stack

ex. duplicateAfter()

### locality of reference

If something is accessed, it is likely to be accessed again in the future (so cache it or move to front). MTF strategy uses this.

### binary search

T(n) = O(log n) or {1 when n=1, 1+T(n/2)}

Insert

Base case: low > high

Search mid, then mid+1 to high or mid-1 to low.

### skip list (define)

a series of lists such that

1. each list S sub i contains the special keys -infinity and + infinity

2. list S sub 0 contains all the keys in the dictionary in increasing order

3, each list is a subsequence of the previous list, with S sub max only containing -infinity and + infinity

4. height of the list is log(n)

Lookup/Insert/Delete is O(log n)

### skip list - remove

worst case log (n) log (n) log (n) --> O(log n)

log(n) to find, log(n) to remove, log(n) to remove lists

### randomized algorithm for skip list

Generate random 0-1

0 < r < 1/2 means add to S0

1/2 < r < 3/4 means add to S0, S1

3/4 < r < 47/8 means add to S0, S1, S2

Prob[new node added to list Si] = 1/2^i

### randomized algorithms (define)

Uses random numbers to control execution

Running time depends on outcomes of random number generator

Properties of good RNG: unbiased, independent

Good average case running time

Worst case is very large but occurs with low probability

### search table (define)

Dictionary implemented by means of sorted array

lookup w/ binary search -> O(log n)

insert (must shift entries) -> O(n)

remove (must shift entries) -> O(n)

Applications: dictionaries of small size, lookups very frequent, insertion and removal rare

### doubly linked list vs singly linked list

tradeoff between speed and performance

easy insertions/deletions

backtracking is easy

no need for pointers to keep position while operating

BUT O(n) extra pointers