I know the test is being done in the spirit of good fun and ab testing and the resultant opinions is a goldmine of data but there are some points to be refined.
1. Unless you get two releases from the original master in both 16 and 24 bit youre method (if youre downgrading the file from 24 to 16bit) may introduce so many artifacts in the end result that run the gamut from jitter to how okay is your power supply in NOT interfering with your rips or downgrading.
2. If a user notices a lesser file size that user will form a bias which would color his opinion
3. If your user listens to those files with:
A. Vintage Gear
B. An untreated room
C. A certain threshold or noise floor within his listening space that he doesnt know about
Your results will likely be colored as the advantages of extended range will likely be very subtle if at all felt.
That being said weve listened to 16 vs 24 in a studio with an outboard Protools HD system and if it werent for the file size we all prefer the 24bit version. What we do is imagine a single point source for a specific instrument and we listen multiple times to the track until we adequately have an idea of how that instrument's sound occupies an imaginary space. When we listen to the other track and the imaginary boundary we put the instrument in, extends or becomes more spherical usually thats the higher res file.
That technique wont serve you though if your houses noise floor is high. Youd just hear the ref or the light humming instead of hearing another layer of range.
But again, i believe that 24 bit has a place in the life of music appreciators which includes most consumers because we dont listen with our ears alone. Our head vibrates and our skin "hears" in a tactile manner and 24 bits advantages may rely on the realm of psychoacoustics. Also, a subwoofer and a supertweeter actually makes sense with 24bit files.
Sorry. Insomnia got the best of me
