| a) Scoping Design | b) Screening Design | c) Optimisation Design | d) Robustness Design |
Definition
Design of experiments (DOE) is defined as a branch of applied statistics that deals with planning, conducting, analyzing, and interpreting controlled tests to evaluate the factors that control the value of a parameter or group of parameters. DOE is a powerful data collection and analysis tool that can be used in a variety of experimental situations.
It allows for multiple input factors to be manipulated, determining their effect on a desired output (response). By manipulating multiple inputs at the same time, DOE can identify important interactions that may be missed when experimenting with one factor at a time. All possible combinations can be investigated (full factorial) or only a portion of the possible combinations (fractional factorial).
A strategically planned and executed experiment may provide a great deal of information about the effect on a response variable due to one or more factors. Many experiments involve holding certain factors constant and altering the levels of another variable. This “one factor at a time” (OFAT) approach to process knowledge is, however, inefficient when compared with changing factor levels simultaneously.
Many of the current statistical approaches to designed experiments originate from the work of R. A. Fisher in the early part of the 20th century. Fisher demonstrated how taking the time to seriously consider the design and execution of an experiment before trying it helped avoid frequently encountered problems in analysis. Key concepts in creating a designed experiment include blocking, randomization, and replication.
- Blocking: When randomizing a factor is impossible or too costly, blocking lets you restrict randomization by carrying out all of the trials with one setting of the factor and then all the trials with the other setting.
- Randomization: Refers to the order in which the trials of an experiment are performed. A randomized sequence helps eliminate effects of unknown or uncontrolled variables.
- Replication: Repetition of a complete experimental treatment, including the setup.
A well-performed experiment may provide answers to questions such as:
- What are the key factors in a process?
- At what settings would the process deliver acceptable performance?
- What are the key, main, and interaction effects in the process?
- What settings would bring about less variation in the output?
A repetitive approach to gaining knowledge is encouraged, typically involving these consecutive steps:
- A screening design that narrows the field of variables under assessment.
- A “full factorial” design that studies the response of every combination of factors and factor levels, and an attempt to zone in on a region of values where the process is close to optimization.
- A response surface designed to model the response.
“What Is Design Of Experiments (DOE)? | ASQ”. 2022. asq.org. https://asq.org/quality-resources/design-of-experiments.
Who
I am not endorsing Penn State’s Design of Experiments course, which is part of their Online Master of Applied Statistics program. Nevertheless, I am impressed by the curriculum for STAT 503 and the prerequisites classes needed.
STAT 503 – Design of Experiments
STAT 502 – Analysis of Variance and Design of Experiments
STAT 501 – Regression Methods
STAT 500 – Matrix Algebra
The Course Overview states: “Statistics is often taught as though the design of the data collection and the data cleaning have already been done in advance. However, as most practicing statisticians quickly learn, typically problems that arise at the analysis stage, could have been avoided if the experimenter had consulted a statistician before the experiment was done and the data were conducted. This course is created to provide an understanding of how experiments should be designed so that when the data are collected, these shortcomings are avoided.”
The textbook that is used for the course is cited here. The link allows you to download a PDF of the 8th Edition.
Montgomery, D. C. Design and Analysis of Experiments, 10th Edition. Hoboken, NJ:
John Wiley & Sons, 2019.
If you enjoy statistics (I did), then you may want to consider learning DOE.
What
Design of experiments has been applied successfully in diverse fields such as agriculture (improved crop yields have created grain surpluses), the petrochemical industry (for highly efficient oil refineries), and Japanese automobile manufacturing (giving them a large market share for their vehicles). These developments are due in part to the successful implementation of design of experiments. The reason to use design of experiments is to implement valid and efficient experiments that will produce quantitative results and support sound decision making.3
Design of Experiments (DoE) is a method used to analyze the relationship between a particular independent variable and the dependent variable when there are multiple independent variables affecting the dependent variable. I will try to explain this definition and the concept through multiple examples. 5
Why
Use DOE when more than one input factor is suspected of influencing an output. For example, it may be desirable to understand the effect of temperature and pressure on the strength of a glue bond.
DOE can also be used to confirm suspected input/output relationships and to develop a predictive equation suitable for performing what-if analysis.2
Experimental Design
I wish I had a large bag of cash for every time a scientist brought me the results of an important experiment and asked me to analyse the data only to find a fatal flaw in the experimental design that rendered the results useless. I would have, errr, a lot of bags of cash right now.
Actually, no I wouldn’t, because I would have happily returned the large bag of cash so they could find another statistician to miraculously ‘rescue’ their experiment…
Here’s an example of a flawed experiment. See if you can spot the fatal error:
A psychologist conducted an experiment to determine whether music had an impact on problem solving. His design was to get his subjects to solve puzzles under the following conditions (in this order):
- In silence
- Listening to classical music
- Listening to jazz
He measured how long it took to complete each of the tasks and summarised the results.
Spotted the flaw yet? Well, the results were as follows:
- The 1st puzzle took the longest
- The 2nd puzzle was completed in a shorter time
- The 3rd puzzle was completed quickest
Conclusion? Music had a positive effect on the ability to solve problems, and people were better at solving problems when listening to jazz.
Absolute rubbish! The flaw was that the experiment captured 2 different effects and there was no way to distinguish between them. The music may have had an impact on problem solving, but the results were contaminated because the subjects were becoming more adept at solving the puzzles.
How would you change the experiment to eliminate the ‘learning effect’?
(Hint: the answer rhymes with ‘shmandomise’…).1
See Theoretical Knowledge Vs Practical Application.
How
Many of the References and Additional Reading websites and Videos will assist you with the design of experiments.
As some professors say: “It is intuitively obvious to even the most casual observer.“
References
1 “Statistics – The Last Dark Art”. 2016. Chi-Squared Innovations. https://www.chi2innovations.com/blog/discover-stats-blog-series/statistics-last-dark-art/#t-1591631358650.
2 “What Is Design Of Experiments (DOE)? | ASQ”. 2022. asq.org. https://asq.org/quality-resources/design-of-experiments.
3 Telford, Jacqueline K. 2022. jhuapl.edu. https://www.jhuapl.edu/Content/techdigest/pdf/V27-N03/27-03-Telford.pdf.
4 Bahuguna. “What Is Design Of Experiments?” 2022. Medium. https://medium.com/geekculture/what-is-design-of-experiments-48555d1587db.
Additional Reading
“5.1. Design And Analysis Of Experiments In Context — Process Improvement Using Data”. 2022. learnche.org. https://learnche.org/pid/design-analysis-experiments/design-analysis-experiments.
“Design Of Experiments”. 2022. woodm.myweb.port.ac.uk. http://woodm.myweb.port.ac.uk/q/DoE.pdf.
“Design Of Experiments – Wikipedia”. 2020. en.wikipedia.org. https://en.wikipedia.org/wiki/Design_of_experiments.
“Design Of Experiments (DOE) Tutorial”. 2022. moresteam.com. https://www.moresteam.com/toolbox/design-of-experiments.cfm.
“How To Perform A Design Of Experiments (DOE) | QI Macros”. 2022. qimacros.com. https://www.qimacros.com/lean-six-sigma-articles/how-to-perform-a-design-of-experiments/.
“Statistics I”. 2022. faculty.elgin.edu. https://faculty.elgin.edu/dkernler/statistics/index.html.
Videos
Introduction to experiment design. Explanatory and response variables. Control and treatment groups.
A well planned DOE can get masses of process knowledge, make money and smash your competition!! It should take a day to plan it correctly, here’s a video to show how to do it.
Learn about experimental designs, completely randomized designs, randomized block designs, blocking variables, and the matched pairs design.
Mathematical Mysteries 