What is meant by " a design of computer experiments?"
We are doing computer modeling design, not real design. Technically we are not doing a DOE. Real design has error associated with the measurements of a real experiment. Doing a modeling and simulation exercise the results are repeatable unless they are stochastic.
What are design inputs and outputs called?
A set of experiments, runs or trials designed to maximize information and minimize experimental effort
non-deterministic, usually some type of time dependency.
same input results in varying/different outputs
same input = same output everytime
varying inputs = varying outputs with different distributions
repeating trials or measurements; useful if the results have inherent noise. Typically not used in simulation
non-independence of input variables.
Implies zero correlation between experimental factors. A purely orthogonal design is used to maintain the independence of independent variables
Arranging experimental units into groups that are similar to one another to reduce known but irrelevant sources of variability and focus on the estimation of study parameters
Define Factorial Designn
Allows the effect of several factors and their interactions to be determined with the same number of trials as are needed to determine any single effect
-Factorial designs are used in place of one-variable-at-a-time studies
What are the four types of uses for DOEs
Comparative, Screening/Characterization, Modeling, Optimizing
Describe a Comparative use for a DOE
the engineer is interested in assessing whether a change in a single factor has resulted in a change/improvement to the process as a whole
Describe a Screening/Characterization use for a DOE
the engineer is interested in "understanding" the process as a whole in the sense that he/she wishes (after design and analysis) to have in hand a ranked list of most important to least important that affect the process. Directly links to ANOVA and is very sensitivie to the range of the variables.
Describe a Modeling use for a DOE
the engineer is interested in functionally modeling the process with the output being a good-fitting (high predictive power) mathematical function, and to have good (maximal accuracy) estimates of the coefficients in that function. Creating the equation
Describe a Optimizing use for a DOE
the engineer is interested in determining optimal settings of the process factors; that is to determine for each factor the level of that factor which optimizes the process response. Use the eqn(s) to get the best answer
List 6 types of classical DOEs and 3 types of modern DOEs.
Taguchi Orthogonal Arrays, Full Factorial, Latin Hyper Cube, Box Behnken, Central Composite, Face-Centered Central Composite.
Space Filling Designs: Uniform, Sphere Packing, Latin Hypercube
When you start you experiment are the variables independent or dependent?
independent to start
What can you use to check for variable dependence and when would you do this?
correlation matrix, after you've run the experiment to check and see if any of the factor combinations and settings have introduced dependence
Full Factorial, Taguchi Experiment, Central Comoposite and Box-Benhken have what design characterisic in common
they are all considered balanced or orthognal designs
D-Optimal design is a Saturated DOE what does that mean?
the number of eqns needed equals the number of cases that have to be run so there is no runs to quantify error because you believe that the eqn is exact
What are the advantages and disadvantages of a Full Factorial Design, what is the eqn?
Advantages: Every extreme and mid points are considered, which reduces error, 3^n number of runs, Orthogonal design
Disadvantages: Excessively high number of cases to test, Limited to 2nd order or less functions
Draw the Full Factorial Design
Draw an orthogonal arraycorrelation matrix and scatterplot matrix for a full factorial design with 3 variables ranging from 1-3.
What is a 3-D space called?
hyper space (hyper cube)
What are the advantages and disadvantages of a Latin Hyper Cube?
A: Rich sampling of interior of design space, balances maximum spacing and uniformity, highly accurate on interior, greater than 2nd order polynomial funcitons, generally a good choice if you know nothing about the problem.
D: Possible high independent variable correlation (very rarely orthogonal) and poor accuracy on edges/corners of design space
Draw the Latin Hyper Cube
What are the advantages and disadvantages of a Central Composite/Face Centered Central Composite and what is the eqn?
2^n + 2n +1
A: Extremes of the design space are considered, fewer cases than full factorial, orthogonal, eliminates extrapolation
D: Large Design Space can result in many non-converged solutions b/c it covers the corners of the space, but M&S tools may not be able to run there, limited to 2nd order modeling applications
Draw the Central Composite/Face Centered Central Composite Design
What are the advantages and disadvantages of the Box-Behnken Design and what is the eqn?
A: Better convergence of analysis tools, fewer executions required ("saturated" = smallest possible number of cases), orthogonal design
D: Extrapolation to extremes of design space introduces error for non-linear design spaces, poor coverage of the corners of the space, limited to specific pre-formulated 2nd order designs
Draw the Box-Behnken Design
What are the advantages and disadvantages of the Space Filling Design: Sphere Parking and what is the eqn?
Number of runs is user specified, no eqn
A: Goal is maximum separation of design points, highly accurate on interior, greater than 2nd order polynomial functions, generally a good choice if you know nothing about the problem
D: Usually not orthogonal designs, pooer coverage of the center of the space
What are the advantages and disadvantages of the Space Filling Design: Uniform and wat is the eqn?
A: Goal is equal separation of design points, highly accurate on interior, greater than 2nd order polynomial functions, generally a good choice if you know nothing about the problem
D: usually not orthogonal designs, poor coverage in regions of the space
Where does classical DOE believe the optimum will occur?
at the extremes of the design space so they put the points at the extreme of the design space
How does engineering design differ from classical DOE?
We have no reason to believe that the optimum will reside at the extremes, and we are not interested in the optimum, we want the eqn so we get the model that will evaluate the entire interior of the space versus the extremes. So all these methods are inferior to waht we are trying to do.
When do you use Effect Screening?
when you have an unmanagable design space (too many variables) or the computational process is very expensive even though you have a reasonable # of variables.
What doe Effect Screening do for us?
it determines the sensitivity of a response to various inputs and screens out those reqs that don't contribute significantly to the VARIABILITY OF THE RESPONSE.
Note: This method is highly sensitive to the ranges of the variable inputs.
What is Surface Fitting?
yields a surrogate model (non-linear response surface model) which gives the response as a function of input parameters.
For the designs that are not orthogonal what do we have to ask our selves?
can we live with the correlation
For the designs that are not well covered over the entire space what do we have to ask ourselves?
can we live with the empty spaces in the scatter plot matrix
What is experimental design?
Experimental design consists of purposeful changes to the inputs to a process in order to observe the corresponding changes in the outputs.
Besides the different types of DOEs what other factors is your DOE choice depend on?
-The number of variables or factors
-Speed (or execution time) of the analysis tools
-The overall accuracy desired
-The behavior of the response
-Convergence behavior of the modeling tool(s)
What is the idea behind fractional factorial designs?
Fractional factorial designs are also used in screening tests and when the higher order interactions can be ignored.
What is The Sparsity-of-Effects Principle?
When a system has several variables, the system or process is likely to be driven primarily by some of the lower level interactions and some of the main effects. To verify do ANOVA.
What is the Projection Property?
Fractional factorial designs can be projected into higher resolution designs for a subset of the original factors. If one factor has negligible effect on the response, then the DoE can be collapsed into a higher resolution DoE for the remaining factors.
What is Sequential Experimentation?
It is possible to combine the runs of two or more fractional factorials to assemble a larger design and estimate factor effects and interactions of interest.
When might someone do sequential experimentation?
When funds/resources are limited and you can only do so much at a time. Or it's years later and the problem has changed a bit. So it's important to keep all your previous runs, document (ranges, assumptions, baseline) and save them under configuration control.
What does higher resolution imply?
That more interactions are accounted for.
What is aliasing structure in general?
It describes what things (main effects and interactions) are confounded with each other.
What is a resolution three design?
A fractional factorial design which allows main effects to be confounded with two-factor interactions but does not allow main effects to be confounded with each other. Main effects are aliased two level interactions, but not each other. Taguchi is a resolution three design.
What is a resolution four design?
A fractional factorial design which allows confounding among two-factor interactions but does not allow any two-factor interactions to be confounded with a main effect. Two level interactions are aliased together.
What is a resolution five design?
An experimental fractional factorial design which allows all main effects and two-factor interactions to be independently estimated; that is, without confounding. Level one is aliased with level 4 and higher. Level two is aliased with level three and higher.
What resolution is our quadratic RSEs?
Resolution five. The main variables and two level interactions are not confounded (completely independent)
What is a saturated design?
It is possible to construct resolution III designs for investigating up to k=N-1 factors in only N runs, where N is a multiple of 4
•When k=N-1, the design is saturated
Solving the minimum number of cases that correspond to the number of unknowns that you have.
Give some examples of useful saturated designs.
2^3-1 (one-half design): Used to study 3 factors using 4 runs
-2^7-4 (one-sixteenth design): Used to study 4 factors using 8 runs
-2^(15-11): Used for studying 15 factors in 16 runs
-2^(31-26): Used for studying 31 factors in 32 runs
What are two types of correlation and what are they used for?
R^2: How well the eqn developed describes the response
Correlation: Starting w/ all independent variables when the combinations and interactions have been joined are they independent (the interactions that is) with each other and the other main variables we started with. If not you now have correlation.
What is the difference between aliasing and confounding.
Aliasing is when you didn't run enough experiments to separate the effects. Confounding is random things you couldn't control, but you can block them in intelligent ways.
When do we run a saturated design versus a least squares analysis.
Taguchi could only handle linear problems. We have no reason to believe the model is linear. A quadratic model can handle main effects plus two level interactions. If you assume the model is correct and it's just a quadratic a saturated DOE and you only run the min number of experiments which equals the number of unknowns (intercept, main effects, interactions, and squared main effects). If you are not sure that your eqn is exact than you run additional experiments to quantify your error and that is what we use least squares for.
What is the eqn for determining the number of runs for a D-Optimal / Saturated Design?
Describe how the eqn for D-Optimal Design is derived
For a quadratic RSE = b0 + sum(bixi) + sum(biixi^2) + sumsum(bijxixj). b0 = 1, sum(bixi) = n, sum(biixi^2) = n, sumsum(bijxixj) = n(n-1)/2. 1+n+n+n(n-1)/2
What is a minimal design?
Designs containing exactly 2^k runs
If I am only looking at one response and it is monotomic, where the solution lie in regards to my solution space?
at the extremes
If I have multiple responses and a pareto frontier with constraints where will the slution lie in regards to my solution space?
on the interior, very rarely at the extremes
What is model verification?
does the model do what I think it should do?
What is model validation?
how well do I predict using a known set of data.
What does each word in "Response Surface Methodology" mean?
Response - Outocome
Surface - looks like a surface in 2D, in higher order looks like a hyperspace
Methodology - The process uses (DOE is part of the process)
What is RSM?
a collection of mathematical and statistical techniques useful for:
-Developing, improving, and optimizing processes
-Design and development of new products
-Improvement of existing product designs
When is RSM used?
RSM is applied in situations where several process variables (independent / control variables) potentially influence a performance measure (response), but the underlying relationship is unknown
What are the main elements of RSM?
RSM consists of:
-Experimental strategy for exploring the space of process / independent variables
-Empirical statistical modeling to create an approx relationship b/t the response and the independent variables
-Optimization methods for finding independent variable values that yield desirable response values
What is a mechanistic model versus empirical model.
Mechanistic: The relationship b/t the variable of interest y and the predictor variables [x1, ...,xk] is known exactly, based on the underlying scientific principles
Empirical: The relationship b/t the variable of interest y and the predictor variables [x1, ...,xk] is NOT known or not fully understood, the experimenter approximates the unknown function
Give an example of mechanistic model and an emperical model
Mechanistic: Deformation of a beam of known geometry and material. Amount of lift generated for a given geometry and flight conditions
Emperical: (Any value obtained from historical data), Life cycle cost metrics, Empty weight
Once you complete the ANOVA and parameter screening do you still need orthogonality (your regressing now)?
no, b/c you've already committed to going with this set of variables, so violating orthogonality is ok but how much violation is okay?
I have violated orthogonality, I don't care about screening, I'm only interested in regression do I care how much I'm willing to violate orthogonality?
Tyler will find out and present.
What is the definition for interactions?
Interaction is the dependence of one factor‟s effect on another factor
What is a contour plot?
The contour plot is a display of the response surface projected onto a the input variable plane where all points with the same response value are connected by contour lines.
What determines the amount of curvature on a counter plot?
Strength of interaction, interactions twist the surface
What determines whether a counter plot is concave up or down?
Reinforcing Interaction: concave up
Conflicting Interaction: concave down
What type of DOEs are good for industrial/physical experiments, and what is bad about them for our design problems?
Classical DOEs: Full Factorial, Fractional Fractorial, Box Behken, Central Composite. They typically put points at the extreme and they can crash our computers.
What DOEs are more suitable for modeling trends from computer codes
Modern DOEs: Space Filling Uniform, Sphere Packing, Latin Hypercube, Quasi-Monte Carlo
Why do we use low order polynomial models in RSM?
•They are flexible, •Their parameters are easy to estimate, •There is a lot of empirical evidence that they work
The philosophy of using polynomial approximation for the true function is based on what principle?
Taylor Series Expansion
What are the phases of an RSM study for optimization?
-Factor screening (phase zero)
-Seeking the region of the optimum with steepest ascent (phase 1)
-Determination of optimum conditions (phase 2)
What is the region of operability?
Region of the independent variable space where the sequential phases of RSM take place
What is the region of interest or region of experimentation?
Small region within Operability range, around a point of interest A
What is the 2nd Order Multivariate Linear Regression Model that we use?
RSE = b0 + sum(bixi) + sum(biixi^2) + sumsum(bijxixj) + error.
Why do call our regression model linear?
B/c we are solving for the coefficients which are linear. So even though the model has quadratic terms what we are solving for is only linear so it is a linear model.
What are the terms in the 2nd order RSE multivariate linear regression model?
Intercept, x_i = independent variables (regressors), b_i = coefficients (partial regressors)
Why are the regression coefficients called partial regressors?
because βi measures the expected change in y when xi is changed and all other regressors are held constant
What resolution is our 2nd order polynomial multivariate linear regression model?
resolution five, main effects are confounded with fourth order terms.
Describe a resolution 6 design (what is confounded with what)?
Resolution 6, main effects are confounded with 5th order, two level effects confounded with 4th order, three level effects confounded with three levels