Is There a Difference: Using a Paired T Test To See Significance in a Change Process

Reading a Paired T Test Example and how to apply it can be helpful for practitioners of Lean and Six Sigma.

Those involved in continuous improvement have one thing in common: our aim is to improve how things are done. That means that one common phenomena in processes is that there was a way of performing a task and a new way of performing a task. In any change process worth measuring, it’s important to determine through hypothesis testing whether there was a statistically significant change in the process. One common way to see if there was a statistically significant change in the process is to use the Paired T-Test.

When to Use a Paired T-Test

Suppose you run a call center. One metric most call centers monitor is Handle Time – or the total time a customer service representative is on the phone. Now, let’s assume you have 3 CSR’s and their average handle times were as follows:

• CSR1: 434 Seconds
• CSR2: 567 Seconds
• CSR3: 123 Seconds

Now, say you decide that the current process contains several steps that you consider to be waste; you identify those steps, and you make changes in the customer service process. After the change, those same CSR’s now have the following average handle times:

• CSR1: 987 Seconds
• CSR2: 276 Seconds
• CSR3: 877 Seconds

Null Hypothesis

But before you make any claims on the change process, it’s important to understand the Null Hypothesis in the case of a Paired T-Test.

For this example, our Null Hypothesis is that:

• There is no difference in the mean handle times before the process change and after the process change.

Looking only at the before and after results doesn’t guarantee that there was a statistically significant change in the process, however. We use the Paired T-Test for that.
Let’s use another example.

Order Fulfillment Picking Process Change

Picking is a process in Order Fulfillment. Now, suppose you manage the outbound department and Picking was a process you managed. Now, imagine that your pickers are measured by the Pick Rate, or the number of items picked per hour.

Because you are a diligent and motivated manager with skills in continuous improvement, you decide that as a team you would improve the picking process, with the hope of increasing Pick Rate with no change in labor. Below are the results of the process change:

In the table above, we have pick rates for the pickers before the process change and after the process change. We also calculate the mean and standard deviations before and after. Then, using excel, we simply do the standard calculations for a Paired T-Test, which gives us the results below:

The data shows that we can reject the Null Hypothesis. In other words, we can reasonably conclude that the process change resulted in a statistically significant change in the mean pick rates of the pickers before and after.

Paired T-Test Examples

There are many example in business where you could use the Paired T-Test. Here are just a few:

• New drug efficacy (pain before and after, weight before and after, etc.)
• Logistics – change in the mean time-in-transit from supplier to customer (change in route, trucker rest times, pedal acceleration, etc.)

Think about your processes. I’m sure you can think of a few examples on your own. Here, you can download the Paired T-Test Excel spreadsheet.