44 terms

CSC316 - Lecture Notes - Test 1

First set of lecture notes for exam studying
Properties of Algorithms
Correct - always gives the right answer
Terminable - no infinite loops
Robust - handle unexpected inputs
Complexity - easily translates into code
Adaptable - easy to adapt
Efficient - low use of resources
Algorithm Efficiency
Time efficient
Space efficient
Often it is a tradeoff
Algorithm (define)
step by step procedure for solving a problem in finite amount of time
Harmonic Series
H(sub n) = 1 + 1/2 + 1/3 + ... + 1/n =~ ln n + Euler's constant (0.577...)
Why worst case?
Easier to analyze
Crucial to applications requiring an upper bound on performance
Average case is impractical (must know inputs and results)
Primitive Operations
Evaluating an expression
Assigning a value to a variable
Indexing into an array
Calling a method
Returning from a method
Measuring Algorithm Run Time (Actual vs # Ops)
Actual Time: problematic because it depends on hardware, software, and programmer.
Number of Operations: independent of hardware and software.
Important Functions
Logarithmic: doubling input size takes one more unit of time
Fractional power: sqrt(n)
Linear Log: n log(n)
Limit Rule
To see if F(x) is O(G(x)) you can take the limit of F(x)/G(x). If...

0 then F is O(g) but G is not O(f)
Infinity then G is O(f) but F is not O(g)
Finite then F is O(g) and G is O(f)
Big Theta (define)
A tight upper bound that indicates that two functions are big O of each other.
Big Omega (define)
A lower bound on the growth rate of a function. If f is big Omega of g then f is asymptotically greater than or equal to g.

Ex. 5n^2 is big omega of n^2 as well as big omega of n.
Recurrence relation
Used to analyze runtime of recursive algorithms.

A function F: N -> R+ defined by

1. base case: f(0), f(1), etc.
2. expression for f(n) in terms of previous values if not base case
Closed form solution (define+find)
Does not depend on previous values of n.

1. Take recurrence relation and expand it.
2. Once you have a pattern, simplify to closed form.
Power Recursion Algorithm
Know this well. Why is it O(log(n))?
linear recursion (define)
When a method is defined so that it makes at most one recursive call each time it is invoked.
recursion trace (define)
A tool for analyzing and visualizing recursion algorithms that involves drawing a box for each instance of the method and labeling it with the parameters.
merge sort (pseudocode)
MergeSort (Array, first, last)
if (first < last -1) then
middle <- (first + last) / 2
MergeSort (T, first, middle)
MergeSort (T, middle + 1, last)
Merge(T, first, middle, middle+1, last)

Time Complexity: O(n log n)
No efficient algorithm has been found, but
the existence of efficient algorithms hasn't been ruled out.

ex. TSP, factorization of large primes
Undecidable problems
No algorithm exists to solve these problems

ex. the Halting problem
stack (define)
Stores arbitrary objects in LIFO order

Push() and Pop()
may have Top(), Size(), and isEmpty()
Exceptions: empty and full
Logarithm Rules
log ac = log a + log c
log a/c = log a - log c
log a^c = c log a
log (of b) a = log (of d) a / log (of d) b
b ^ log a = a ^ log b
queue (define)
Stores arbitrary objects in FIFO order

obj dequeue()
may have front(), size(), isEmpty()
Exception: empty and full
circular buffer ()
positions n-1 and 0 are adjacent

think position x mod n
array based queue
array of size N is circular fashion
two variables to track front and end -- f is front element, r is index immediately past the rear element

size: return (N - f + r)
isempty: return (f = r)

enqueue and dequeue are const
list (define)
an ordered sequence of arbitrary objects -- not necessarily array implementation

May have: size(), isEmpty()

Implemented as contiguous array or linked memory.
contiguous-memory list
random access to each element in O(1) time
no memory overhead - only list items stored

array sizes are fixed - memory waste, not for dynamic apps
insert/remove in middle - massive data movement
linked-list (vs contiguous)
insert/delete - no data mvmt
dynamic allocation of memory on as needed benefit

no random access, ie. time is O(n)
singly linked list
Nodes w/ two pieces:
1. an element
2. a pointer to next node

Head pointer
Tail pointer
singly linked list w/ dummy
Dummy head = no element stored in head (null value) but head always points to the dummy.

all changes take place after the head
good for recursion
makes code more uniform
duplicate list using dummy method **
duplicate(node p):
newHead <- newNode(dummy, duplicateAfter(p))
return newHead

duplicateAfter(node p)
if next(p) = null then
return null
return newNode(element(node(p)), duplicateAfter(next(p)))
tail recursion vs general recursion
tail recursion:
recursive call is the last operation
easily expressed with while loops
ex. removeAfter()

general recursion:
additional code is called after recursion returns
can be expressed as a programmer defined stack
ex. duplicateAfter()
locality of reference
If something is accessed, it is likely to be accessed again in the future (so cache it or move to front). MTF strategy uses this.
binary search
T(n) = O(log n) or {1 when n=1, 1+T(n/2)}
Base case: low > high
Search mid, then mid+1 to high or mid-1 to low.
recursive implementation of power
interpolation search
worst case -> Theta(n)
average case -> Theat(log log n)
skip list (define)
a series of lists such that
1. each list S sub i contains the special keys -infinity and + infinity
2. list S sub 0 contains all the keys in the dictionary in increasing order
3, each list is a subsequence of the previous list, with S sub max only containing -infinity and + infinity
4. height of the list is log(n)

Lookup/Insert/Delete is O(log n)
skip list - remove
worst case log (n) log (n) log (n) --> O(log n)
log(n) to find, log(n) to remove, log(n) to remove lists
randomized algorithm for skip list
Generate random 0-1
0 < r < 1/2 means add to S0
1/2 < r < 3/4 means add to S0, S1
3/4 < r < 47/8 means add to S0, S1, S2

Prob[new node added to list Si] = 1/2^i
skip list - insert
worst case log (n) log (n) const --> O(log n)
randomized algorithms (define)
Uses random numbers to control execution
Running time depends on outcomes of random number generator
Properties of good RNG: unbiased, independent
Good average case running time
Worst case is very large but occurs with low probability
search table (define)
Dictionary implemented by means of sorted array

lookup w/ binary search -> O(log n)
insert (must shift entries) -> O(n)
remove (must shift entries) -> O(n)

Applications: dictionaries of small size, lookups very frequent, insertion and removal rare
link inversion
pointer on each node that gets reversed on way through - zig zag alorgithms
doubly linked list vs singly linked list
tradeoff between speed and performance

easy insertions/deletions
backtracking is easy
no need for pointers to keep position while operating

BUT O(n) extra pointers
list backtracking
1. Traverse list twice
2. Insert into stack as you go (third from last means put all in stack and pop 3)
3. List Inversion (reverse pointers on your way through)