Is A/B testing guitars and amps fundamentally flawed?

[JGinNJ]

There are also differences in the short-term and long-term preferences. When I was buying my first expensive flat-top, I A/B'ed mahogany and rosewood (back and sides) guitars. In the stores rosewood guitars always sounded better. Their sounds were rich and complex whereas mahogany guitars were dull and dry in comparison. So I bought a rosewood guitar. Some time down the line I also bought a mahogany guitar. I now actually prefer mahogany guitars. I find that rosewood's richness gets in the way too much. Mahogany allows you to make the music shine with your playing. That's my opinion.

This speaks to what I was going to comment. That is, it's not really A/B testing, it's A or B versus some ideal you have in your head. The one that comes closest to that is the one you're going to prefer.

I've found going to a store with a variety of instruments (and archtops, specifically) to be incredibly helpful for comparison shopping, and narrowing down what I do and don't like about various guitars.

[skykomishone]

I'm not sure flawed is the right term. It certainly isn't a controlled scientific experiment. However, I doubt many of us really believe that buying a guitar is purely an objective endeavor, rather one based on our own experience, tastes, and preferences. I do like the 'psychoacoustic' reference. And also like to think of it like wine tasting.

Economists like to use the phrase 'ceteris paribus', or, everything else constant, when trying to compare results of changing a variable of interest on outcomes. That would be hard with guitars. I bought a classical guitar once from a luthier where he had me on one side of a curtain while my son played a series of guitars that I had picked out of the builder's collection to identify my preferred one. He then asked us to change positions, where I played and my son picked, then we compared ranking. We both picked the same guitar. Man, that was a relief! Not very scientific, but seemed appropriate. I bought the guitar. I wonder if I tried to repeat that 15 years later, if we'd reach the same decision.

[Spook410]

Put "psycho" before anything and it suddenly sounds scientific and universal. Too many variables. Don't overthink. Learn to hear the big things (easier said than done) then trust yourself. Of course, I prefer diversification. Buy enough gear and guitars and on any given day or in any given situation, you're bound to find something you like.

[skykomishone]

Come to think about it, I frequently go Psycho-acoustic when I play! The reference reminded me of an old friend, jazzoid, composer, pianist, and arranger extraordinaire by the name of Patrick Kelly, who has a big band in Cincinnati called the PsychoAcoustic Orchestra. I think I'll run this by him and get his thoughts.

[ruger9]

i didn't say it wasn't a real thing. I just mean that peoples' claims about the HUGE difference is extremely over-hyped. My celestion G12H-75 creambacks sound pretty much the same 150+ hours and several loud jams later.

You've read the whole thread and know I don't disagree, right?

[ruger9]

Put "psycho" before anything and it suddenly sounds scientific and universal. Too many variables. Don't overthink. Learn to hear the big things (easier said than done) then trust yourself. Of course, I prefer diversification. Buy enough gear and guitars and on any given day or in any given situation, you're bound to find something you like.

I was just thinking about this today. I got a new delay pedal today. Now, I've spent many years owning many dozens of delay pedals, and could bore you with all kinds of minutiae about how they are "so different". They are not. With a few obvious exceptions, for example comparing a rack mount studio digital delay with an old Echoplex that has warble in it, a delay is a delay.

but basically, an analog delay is an analog delay. Add modulation to it and it becomes even more homogenized. I LOVE analog (or analog-voiced digital) delay with modulation (think: DMM/tape echoes). And I've owned them all. But really, in the end, while sitting in my home I may find I prefer an old big box Deluxe Memory Man to a Carbon Copy, they're the same thing. Really. It's not as if the DMM is red and the CC is white. They are both pink. Slightly different shades of pink. And having a preference for hot pink or bubblegum pink is absolutely valid, but they're both still pink.

[Namelyguitar]

Compare by picking up a guitar in your hands. Close your eyes, and notice the way it feels (mainly the fretting hand) and the way it sounds when played. The specs or characteristics, sound, and feel can be noticeably different from one to the next. Materials, craftmanship, setup, and design all affect that "feel."

[Cunamara]

Put "psycho" before anything and it suddenly sounds scientific and universal.

Except there is a legitimate and rigorous scientific field of psychophysics (of which psychoacoustics is a subset) that has been studying the principles and processes governing perception for over 150 years.

Psychophysics - Wikipedia

And, yes, subjective A/B testing is fundamentally flawed. One may readily find that one's preference of A > B is confounded by a preference for B > C and then C > A.

[Spook410]

Except there is a legitimate and rigorous scientific field of psychophysics (of which psychoacoustics is a subset) that has been studying the principles and processes governing perception for over 150 years. Psychophysics - Wikipedia And, yes, subjective A/B testing is fundamentally flawed. One may readily find that one's preference of A > B is confounded by a preference for B > C and then C > A.

I'm sure it's a field filled with significant and important insights into many interesting things. But I'm thinking that what is going on at Guitar Center on a Saturday afternoon has more to do with critical listening skills and/or real, not imagined, acoustics.

[jzucker]

I'm sure it's a field filled with significant and important insights into many interesting things. But I'm thinking that what is going on at Guitar Center on a Saturday afternoon has more to do with critical listening skills and/or real, not imagined, acoustics.

guitar center and critical listening skills? Can't tell if you're joking or not!

[Spook410]

guitar center and critical listening skills? Can't tell if you're joking or not!

Have you been to a Guitar Center on a Saturday afternoon? It is at once, the triumph of our musical culture's ability to inspire and an irrefutable indication of it's imminent downfall. A triumph of Dadaism. Pollack for the ears. As for A/B testing I still say use your ears and don't overthink.

[One_Note]

I think a lot of tonal perception can be affected by mood or ear fatigue. I've left off playing thinking I had a really great tone, only to come back the next day at the same settings and it sounds too bright or too mid-rangy.

[mje]

Anther term for ear fatigue is TTS, or temporary threshold shift. A few minutes at moderate volume can play havoc with your hearing sensitivity.

[medblues]

Fletcher Munson Curve: A Must-Know for Audio Recording

[medblues]

I notice a drastic pitch shift when I put on headphones - almost a whole semitone. Does anybody know more about this? : audioengineering

"someone playing next to a really loud brass musician might have trouble staying in tune, because everything may sound sharp to them, when in reality the pitch may be fine."

[JCat]

How many times have you picked up a guitar (or played through an amp) and then when you switch to another guitar (or amp) at first it sounded like it was lacking something. But then after playing the second guitar for a while and switched back to the first one, this time you found that the first one is lacking something that you got used to hearing from the second one.
The most typical example of this is if the first gear is warm and smooth and the second one is bright and clear. Second one at first sounds a bit harsh and lacks lows. When you go back to the first one this time it sounds muddy to you.
Our mind adapts to the sound we are hearing. It's like it internally adjusts the EQ to make what we hearing optimal. Then you switch to another gear, it sounds a bit off at first, but after a short while the mind re-adjust its EQ. Say it tunes-in to the highs a bit more, cuts a bit of the lows etc.
That's probably an evolutionary trait that configures our hearing so we are able to process aural cues (threats, food sources, cries of lost offsprings etc.) under various imperfect environmental conditions.
Another flaw is our implicit belief that we should have a innate preference for one gear over another, we just need to put in the time to make the right choice. But that's often not true.
What are your thoughts on this?

I think maybe your idea of "A/B-testing" is a bit narrow?

There are many situations where A/B testing is useful, for example when troubleshooting noise issues and signal strength. It's applicable for guitars, amps, speakers, pedals, cables, power sources, tubes etc. In this context the objective is often binary; it's either right or wrong, e.g hum over or below a threshold, hiss level acceptable or not etc.

In your example there is no right or wrong, it's an entirely subjective exercise where your preference may change based on what you had for breakfast or the color of your shirt.

If I have to concentrate hard to hear a difference, it's not worth sweating.

But here's an audio technician pro tip:

When listening closely for fine details, take frequent pauses in order to reset and recalibrate your ears. It's surprising how fast the ears adapt to the environment, after a little while the tired ear will accept a sound that the fresh ear would reject.

[medblues]

I think maybe your idea of "A/B-testing" is a bit narrow?

There are many situations where A/B testing is useful, for example when troubleshooting noise issues and signal strength. It's applicable for guitars, amps, speakers, pedals, cables, power sources, tubes etc. In this context the objective is often binary; it's either right or wrong, e.g hum over or below a threshold, hiss level acceptable or not etc.

In your example there is no right or wrong, it's an entirely subjective exercise where your preference may change based on what you had for breakfast or the color of your shirt.

If I have to concentrate hard to hear a difference, it's not worth sweating.

But here's an audio technician pro tip:

When listening closely for fine details, take frequent pauses in order to reset and recalibrate your ears. It's surprising how fast the ears adapt to the environment, after a little while the tired ear will accept a sound that the fresh ear would reject.

I like that saying, especially when sweating can mean $$$$$$$$$$

[vinlander]

Also don't underestimate the "post purchase rationalization" syndrome in an A/B test...
The more money spent on something the less objectivity!

[ruger9]

If I have to concentrate hard to hear a difference, it's not worth sweating.

.

Couldn't agree more. It took me dozens and dozens of ODs, over several years, to truly figure that one out LOL. I can quickly discern "yes or no" when evaluating if I like a piece of gear... deciding whether or not it's a "keeper" (intent: forever, or at least several years) is a bit tougher. I only have ONE piece of gear that I KNOW I'll have when I die, and it's been my #1 for 25 years now, so there's more to it than it being a "keeper" from the beginning, it's a part of me at this point.

Everything else is just "fun".

[mad dog]

Interesting discussion. It's only in pedal testing that expectation and memory seems to not get in the way. Much easier to hear the differences.

Speaker break in is different. Some speakers seem ready to go out of the box, sounding about the same to start as they ever will. Others change rather noticeably over a short period of time. How much my perception on this is colored by expectation and acclimatization I cannot be sure of course. I've noticed it too in shopping for used speakers. Often people are selling barely used, essentially new speakers. Seems they didn't like that sound out of the box. I'll buy based on using that same speaker before. It'll sound flat and sorta strange when I get it, compared to what I remember of the broken in ones. At first. Unless it really is in my mind ...

Apart from the idea of A/B testing, it seems to me there is a learning curve with guitars and amps. I've learned to be patient. And to play with the controls. Some of the biggest mistakes I've made with new to me amps is setting them as I set others, then being disappointed at first. Sold more than one in that way, only to realize later that there was a tonal sweet spot I'd missed first time around.

[ruger9]

Apart from the idea of A/B testing, it seems to me there is a learning curve with guitars and amps. I've learned to be patient. And to play with the controls. Some of the biggest mistakes I've made with new to me amps is setting them as I set others, then being disappointed at first. Sold more than one in that way, only to realize later that there was a tonal sweet spot I'd missed first time around.

I've experienced this with all gear- pedals, amps, and guitars.

[JaxJaxon]

I allready know that one guitar will sound diffrent from another. But as for amps the only way to be shure is to play the same guitar the same way each time you plug into a diffrent amp and you have to be able to do this while you stop and adjust the amps controals.So for me to accomplish this I use a good looper and play it into each amp even use a Y cable spliter to run the signal into bouth amps at the same time and turn one up and the other amp down to see how each compares.

[KirkP]

I allready know that one guitar will sound diffrent from another. But as for amps the only way to be shure is to play the same guitar the same way each time you plug into a diffrent amp and you have to be able to do this while you stop and adjust the amps controals.So for me to accomplish this I use a good looper and play it into each amp even use a Y cable spliter to run the signal into bouth amps at the same time and turn one up and the other amp down to see how each compares.

That’s a variation of what’s known as “reamping” in the recording business.
Reamp Basics - Radial Engineering
I agree that’s a good way to A/B test.

A looper can also be used for sound check, allowing you to walk around to hear how your tone is holding up through the room. I’ve thought of doing that before performances, but I never have.

[sgosnell]

I just walk around with my guitar. The wireless system makes it simple. I mostly do it to check volume, as well as tone, and it's easy to make changes on the fly.