Paired marble mass difference detection

Pairs of marbles were used to setup and demonstrate the use of the paired TTEST function. Each student was given two visually similar marbles. The students were asked to decide which marble was lighter, which marble was heavier.

Which is heavier?

In terms of in-class procedure, at 8:00 I wrote the marble masses on a pad of paper and then entered them into a G Suite spreadsheet. At 9:00 I set the scale next to the keyboard and had the students come up, putting the light one on first, the heavy one on second, and directly entered the values into the spreadsheet. This proved more efficient.

Though the marbles look the same, there is a weight difference. After 16 students in the 8:00 class the paired t-test p-value stood at 0.07, failure to reject the null hypothesis. The students could not detect the difference. Although not appropriate due to the failed t-test, the effect size was actually medium to large. I had hope that the 9:00 class would bring home a p-value under 0.05. 

With the two sections combined, 38 students brought in a paired t-test p-value of 0.02, below our chosen alpha of 0.05. The null hypothesis was rejected. Increase the sample size until one gets a significant difference. Yet is this just playing the sample size game until one achieves significance? The estimated effect size came in at 0.5, a medium effect size, suggesting that there is a difference there to be measured.

I would be remiss if I did not note that a Chi squared test on the students ability failed to achieve significance. Removing four students who had identical mass marbles, 22 of the remaining 34 correctly identified the heavier marble. 12 students failed to identify the heavier marble. In a Chi squared test the chi-squared statistic did not exceed the critical value. Thus while the demonstration led to rejection of the null hypothesis, the matter remains open for further data on whether this was perhaps just a chance result. For those who want to play with the data, the data spreadsheet is available in G Suite.

Using Google's Statistics add-in one can analyze the marble pairs as (x,y) data. The result also casts doubt on the ability of the students to distinguish with a slope of zero being possible for the data.

Perhaps increasing the sample size will eventually bring in a significant results for the Chi square test and ultimately a paired data test for a non-zero slope. Perhaps. The experiment will simply have to be repeated.