How to Test Anything, Part 1: Maximize Learning, Minimize Unexpected Consequences

How to Test Anything, Part 1: Maximize Learning, Minimize Unexpected Consequences
c
c
Step 1: Write down what you want to test, and list all the expected outcomes and their relative probabilities.
freshly ground black pepper
Step 2: List all possible variations. They may be obvious, or they may be nearly invisible. If they are hidden, check all the places that might be relevant.
bowl
Step 3: Choose a variation, either at random or based on judgment. First identify preconceptions that may bias the test. If you know the result, you have once raised a barrier to learning (your own inability to see reality).
ensure
Step 4: Measure the metric(s) of interest, and document your reasoning. Then measure everything else. Document your reasoning for everything else.
define neural networks
Step 5: Document again. You need to be reminded that you remembered to measure what you thought you were measuring.
need
Step 6: Analyze the results. Learn to tell when you are learning as fast as possible, and when you need a better hypothesis or additional variations of the test.
need
Step 7: Revise your model. Your models, not reality, are the unbiased sources of your learning. One of us (DHH) has suggested that the world does indeed need more anthropomorphized quadratic equations.
shrimp
Step 8: If there are connections between what you are testing and other areas of interest, consider parallel tests in those areas.
serves 4
Step 9: Share what you have learned as widely as possible. You can take steps to reduce the risk of negative outcomes, if you learn enough.
coarse salt
Step 10: Set up a process to follow the most important results even after you have forgotten about them. If no one is interested, announce the results and explain why they are important to people who care.
¾ cup fresh lemon juice