Properties of Algorithms

Correct - always gives the right answer

Terminable - no infinite loops

Robust - handle unexpected inputs

Complexity - easily translates into code

Adaptable - easy to adapt

Efficient - low use of resources

Terminable - no infinite loops

Robust - handle unexpected inputs

Complexity - easily translates into code

Adaptable - easy to adapt

Efficient - low use of resources

Algorithm Efficiency

Time efficient

Space efficient

Often it is a tradeoff

Space efficient

Often it is a tradeoff

Algorithm (define)

step by step procedure for solving a problem in finite amount of time

Harmonic Series

H(sub n) = 1 + 1/2 + 1/3 + ... + 1/n =~ ln n + Euler's constant (0.577...)

Why worst case?

Easier to analyze

Crucial to applications requiring an upper bound on performance

Average case is impractical (must know inputs and results)

Crucial to applications requiring an upper bound on performance

Average case is impractical (must know inputs and results)

Primitive Operations

Evaluating an expression

Assigning a value to a variable

Indexing into an array

Calling a method

Returning from a method

Assigning a value to a variable

Indexing into an array

Calling a method

Returning from a method

Measuring Algorithm Run Time (Actual vs # Ops)

Actual Time: problematic because it depends on hardware, software, and programmer.

Number of Operations: independent of hardware and software.

Number of Operations: independent of hardware and software.

Important Functions

Constant

Logarithmic: doubling input size takes one more unit of time

Fractional power: sqrt(n)

Linear

Linear Log: n log(n)

Quadratic

Polynomial

Exponential

Factorial

Logarithmic: doubling input size takes one more unit of time

Fractional power: sqrt(n)

Linear

Linear Log: n log(n)

Quadratic

Polynomial

Exponential

Factorial

Limit Rule

To see if F(x) is O(G(x)) you can take the limit of F(x)/G(x). If...

0 then F is O(g) but G is not O(f)

Infinity then G is O(f) but F is not O(g)

Finite then F is O(g) and G is O(f)

0 then F is O(g) but G is not O(f)

Infinity then G is O(f) but F is not O(g)

Finite then F is O(g) and G is O(f)

Big Theta (define)

A tight upper bound that indicates that two functions are big O of each other.

Big Omega (define)

A lower bound on the growth rate of a function. If f is big Omega of g then f is asymptotically greater than or equal to g.

Ex. 5n^2 is big omega of n^2 as well as big omega of n.

Ex. 5n^2 is big omega of n^2 as well as big omega of n.

Recurrence relation

Used to analyze runtime of recursive algorithms.

A function F: N -> R+ defined by

1. base case: f(0), f(1), etc.

2. expression for f(n) in terms of previous values if not base case

A function F: N -> R+ defined by

1. base case: f(0), f(1), etc.

2. expression for f(n) in terms of previous values if not base case

Closed form solution (define+find)

Does not depend on previous values of n.

1. Take recurrence relation and expand it.

2. Once you have a pattern, simplify to closed form.

1. Take recurrence relation and expand it.

2. Once you have a pattern, simplify to closed form.

Power Recursion Algorithm

Know this well. Why is it O(log(n))?

linear recursion (define)

When a method is defined so that it makes at most one recursive call each time it is invoked.

recursion trace (define)

A tool for analyzing and visualizing recursion algorithms that involves drawing a box for each instance of the method and labeling it with the parameters.

merge sort (pseudocode)

MergeSort (Array, first, last)

if (first < last -1) then

middle <- (first + last) / 2

MergeSort (T, first, middle)

MergeSort (T, middle + 1, last)

Merge(T, first, middle, middle+1, last)

Time Complexity: O(n log n)

if (first < last -1) then

middle <- (first + last) / 2

MergeSort (T, first, middle)

MergeSort (T, middle + 1, last)

Merge(T, first, middle, middle+1, last)

Time Complexity: O(n log n)

NP-Complete

No efficient algorithm has been found, but

the existence of efficient algorithms hasn't been ruled out.

ex. TSP, factorization of large primes

the existence of efficient algorithms hasn't been ruled out.

ex. TSP, factorization of large primes

Undecidable problems

No algorithm exists to solve these problems

ex. the Halting problem

ex. the Halting problem

stack (define)

Stores arbitrary objects in LIFO order

Methods:

Push() and Pop()

may have Top(), Size(), and isEmpty()

Exceptions: empty and full

Methods:

Push() and Pop()

may have Top(), Size(), and isEmpty()

Exceptions: empty and full

Logarithm Rules

log ac = log a + log c

log a/c = log a - log c

log a^c = c log a

log (of b) a = log (of d) a / log (of d) b

b ^ log a = a ^ log b

log a/c = log a - log c

log a^c = c log a

log (of b) a = log (of d) a / log (of d) b

b ^ log a = a ^ log b

queue (define)

Stores arbitrary objects in FIFO order

Methods:

enqueue(obj)

obj dequeue()

may have front(), size(), isEmpty()

Exception: empty and full

Methods:

enqueue(obj)

obj dequeue()

may have front(), size(), isEmpty()

Exception: empty and full

circular buffer ()

positions n-1 and 0 are adjacent

think position x mod n

think position x mod n

array based queue

array of size N is circular fashion

two variables to track front and end -- f is front element, r is index immediately past the rear element

size: return (N - f + r)

isempty: return (f = r)

enqueue and dequeue are const

two variables to track front and end -- f is front element, r is index immediately past the rear element

size: return (N - f + r)

isempty: return (f = r)

enqueue and dequeue are const

list (define)

an ordered sequence of arbitrary objects -- not necessarily array implementation

Methods:

lookUp(i)

remove(i)

insert(o)

insertBefore(i,o)

May have: size(), isEmpty()

Implemented as contiguous array or linked memory.

Methods:

lookUp(i)

remove(i)

insert(o)

insertBefore(i,o)

May have: size(), isEmpty()

Implemented as contiguous array or linked memory.

contiguous-memory list

Benefits:

random access to each element in O(1) time

no memory overhead - only list items stored

Problems:

array sizes are fixed - memory waste, not for dynamic apps

insert/remove in middle - massive data movement

random access to each element in O(1) time

no memory overhead - only list items stored

Problems:

array sizes are fixed - memory waste, not for dynamic apps

insert/remove in middle - massive data movement

linked-list (vs contiguous)

Benefits:

insert/delete - no data mvmt

dynamic allocation of memory on as needed benefit

Problems:

no random access, ie. time is O(n)

insert/delete - no data mvmt

dynamic allocation of memory on as needed benefit

Problems:

no random access, ie. time is O(n)

singly linked list

Nodes w/ two pieces:

1. an element

2. a pointer to next node

Head pointer

Tail pointer

1. an element

2. a pointer to next node

Head pointer

Tail pointer

singly linked list w/ dummy

Dummy head = no element stored in head (null value) but head always points to the dummy.

Advantages:

all changes take place after the head

good for recursion

makes code more uniform

Advantages:

all changes take place after the head

good for recursion

makes code more uniform

duplicate list using dummy method **

duplicate(node p):

newHead <- newNode(dummy, duplicateAfter(p))

return newHead

duplicateAfter(node p)

if next(p) = null then

return null

else

return newNode(element(node(p)), duplicateAfter(next(p)))

newHead <- newNode(dummy, duplicateAfter(p))

return newHead

duplicateAfter(node p)

if next(p) = null then

return null

else

return newNode(element(node(p)), duplicateAfter(next(p)))

tail recursion vs general recursion

tail recursion:

recursive call is the last operation

easily expressed with while loops

ex. removeAfter()

general recursion:

additional code is called after recursion returns

can be expressed as a programmer defined stack

ex. duplicateAfter()

recursive call is the last operation

easily expressed with while loops

ex. removeAfter()

general recursion:

additional code is called after recursion returns

can be expressed as a programmer defined stack

ex. duplicateAfter()

locality of reference

If something is accessed, it is likely to be accessed again in the future (so cache it or move to front). MTF strategy uses this.

binary search

T(n) = O(log n) or {1 when n=1, 1+T(n/2)}

Insert

Base case: low > high

Search mid, then mid+1 to high or mid-1 to low.

Insert

Base case: low > high

Search mid, then mid+1 to high or mid-1 to low.

recursive implementation of power

FILL THIS IN

interpolation search

worst case -> Theta(n)

average case -> Theat(log log n)

average case -> Theat(log log n)

skip list (define)

a series of lists such that

1. each list S sub i contains the special keys -infinity and + infinity

2. list S sub 0 contains all the keys in the dictionary in increasing order

3, each list is a subsequence of the previous list, with S sub max only containing -infinity and + infinity

4. height of the list is log(n)

Lookup/Insert/Delete is O(log n)

1. each list S sub i contains the special keys -infinity and + infinity

2. list S sub 0 contains all the keys in the dictionary in increasing order

3, each list is a subsequence of the previous list, with S sub max only containing -infinity and + infinity

4. height of the list is log(n)

Lookup/Insert/Delete is O(log n)

skip list - remove

worst case log (n) log (n) log (n) --> O(log n)

log(n) to find, log(n) to remove, log(n) to remove lists

log(n) to find, log(n) to remove, log(n) to remove lists

randomized algorithm for skip list

Generate random 0-1

0 < r < 1/2 means add to S0

1/2 < r < 3/4 means add to S0, S1

3/4 < r < 47/8 means add to S0, S1, S2

Prob[new node added to list Si] = 1/2^i

0 < r < 1/2 means add to S0

1/2 < r < 3/4 means add to S0, S1

3/4 < r < 47/8 means add to S0, S1, S2

Prob[new node added to list Si] = 1/2^i

skip list - insert

worst case log (n) log (n) const --> O(log n)

randomized algorithms (define)

Uses random numbers to control execution

Running time depends on outcomes of random number generator

Properties of good RNG: unbiased, independent

Good average case running time

Worst case is very large but occurs with low probability

Running time depends on outcomes of random number generator

Properties of good RNG: unbiased, independent

Good average case running time

Worst case is very large but occurs with low probability

search table (define)

Dictionary implemented by means of sorted array

lookup w/ binary search -> O(log n)

insert (must shift entries) -> O(n)

remove (must shift entries) -> O(n)

Applications: dictionaries of small size, lookups very frequent, insertion and removal rare

lookup w/ binary search -> O(log n)

insert (must shift entries) -> O(n)

remove (must shift entries) -> O(n)

Applications: dictionaries of small size, lookups very frequent, insertion and removal rare

link inversion

pointer on each node that gets reversed on way through - zig zag alorgithms

doubly linked list vs singly linked list

tradeoff between speed and performance

easy insertions/deletions

backtracking is easy

no need for pointers to keep position while operating

BUT O(n) extra pointers

easy insertions/deletions

backtracking is easy

no need for pointers to keep position while operating

BUT O(n) extra pointers

list backtracking

1. Traverse list twice

2. Insert into stack as you go (third from last means put all in stack and pop 3)

3. List Inversion (reverse pointers on your way through)

2. Insert into stack as you go (third from last means put all in stack and pop 3)

3. List Inversion (reverse pointers on your way through)