Scheduled maintenance: Saturday, August 7 from 5PM to 6PM PDT
Home
Subjects
Create
Search
Log in
Sign up
Upgrade to remove ads
Only $2.99/month
Rule Classifiers
STUDY
Flashcards
Learn
Write
Spell
Test
PLAY
Match
Gravity
Terms in this set (44)
binary versus multiclass
for binary class problems, the primary class of interest is usually considered the positive class - not true for imbalanced data usually
multi class problems can be decomposed into
several binary class problems
decision trees can handle
multi class problems, as can most of the methods we consider
deterministic classifier produces
a discrete-valued label
probabilistic classifier produces
a probably for each possible class (sums to 1.0)
-can produce a single label by assigning the class with highest probability - or can factor for cost sensitive learning
what is an example of a probabilistic classifier
naive bayes
are decision trees probabilistic
no but they do generate a probability estimate as a side effect of how they work
global classifier
fits a single model to entire dataset - one size fits all may not work well when relationship between attributes and class varies over space instnace
local model
partitions the input space into smaller regions and fit a model to examples in each region
which is less susceptible to overfitting
global models
is a linear model local or global
global
is decision tree local or global
local
generative models learn
the distributions of classes
discriminative models learn
the boundaries between classes and do not need to learn distributions
naive bayes is generative or discriminative
generative
decision tree and neural networks are generative or discriminative
discriminative as well as everythign else
LHS of a rule
rule antecedent or condition
RHS of a rule
consequent
coverage of a rule
fraction of records that satisfy antecedent of rule
accuracy of rule
fraction of records that satisfy both antecedent and consequent
mutually exclusive rules
classifiers contain mutually exclusive rules if the rules are independent of each other
-every record is covered by at most one rule
exhaustive
each record covered by at least one rule
rules are not mutually exclusive
a record may trigger more than one rule
solution - ordered rule set, unordered rule set - use voting schemes
rules are not exhaustive
record may not trigger any rules
solution - use default class
ordered rule set
rules are rank ordered according to priority
- known as decision list
if no rules fired it is assigned
the default class
rule based ordering
individual rules are ranked based on their quality
class based ordering
rules that belong to the same class appear together
direct method
extract rules directly from data
RIPPER, CN2, holtes 1R
indirect method
extract rules form other classification models
decision trees, neural networks
sequential covering
1) start from empty rule
2) grow rule using learn one rule function
3) remove training records covered by the rule
4) repeat 2 and 3 until stoppign
instance elimination needed why
otherwise next rule is identical to previous
why do we remove positive instances
ensure that next rule different
remove negative instances
prevent underestimating accuracy of rule
foils information gain
first order inductive learner
RIPPER 2-class problem
choose one of the classes as pos/one neg
-learn rules for pos class (gen to specific)
-neg will be default class
RIPPER multi-class problem
order the classes according to increasing class prevalence
-learn the rule set for the smallest class first, treat rest as negative
-repeat with next smallest pos class
RIPPER growing rule
-start from empty rule
-add conjuncts as long as they improve FOILS info gain
-stop when rule no longer covers negative examples
-prune the rule using incremental reduced error pruning
RIPPER measure for pruning
V=(p-n)/(p+n)
p - number of pos examples covered by the rule in val set
n - same with neg
RIPPER pruning method
delete any final sequence of conditions that maximizes V
RIPPER building a rule set
use seq covering algorithm
- finds best rule that covers current set of positive examples
-eliminate both pos and neg examples covered by rule
Indirect method: C4.5 rules
extract rules from an unpruned decision tree
for each rule r:A->y
-prune if one of the alternate rules has lower pessimistic error rate
-repeat until we can no longer improve
indirect method: C4.5 rules ordering
class ordering
-each subset is a collection of rules with same rule consequent
- compute description length for each subset
advantages of rule based classifiers
has characteristics similar to decision trees
- highly expressive
-easy to interpret
-performance comparable to DT
- can handle redundant attributes
better suited for handling imbalanced classes
-harder to handle missing Vals in test set
You might also like...
Grammar rules
20 terms
WEEK 6 (Chapter 8) Categorisation
23 terms
Statistics Chapter 5
18 terms
Stats Ch. 6
28 terms
Other sets by this creator
ensembles
28 terms
class imbalance
22 terms
Nearest Neighbor
22 terms
Decision Trees
58 terms
Other Quizlet sets
Psych: Modules 9 and 10 Study Guide
66 terms
Chapters 1,2,3 Intro to Sociology Vocab
76 terms
Marketing 3
68 terms
T-6 BOLDFACE EMERGENCY PROCEDURES
10 terms