Working with the human ear: the proper evaluation of audio equipment incl. amplifiers

[A.S.]

It doesn't begin to be anything approaching easy.

I have a question, however. Is the discrimination of relative levels of quality perhaps different different from the ability to distinguish (or more accurately, not distinguish) physical properties such as colour by reference to memory or some internal standard?

To take the example of your wife and the blouse. I can see how your wife might find a blouse in the store and think (for example) that it's the exact shade of blue she needs to match some other article of clothing she owns, only to find it's not so when she makes it home.

However, if the comparison is not of a specific physical property (such as exact colour), but rather the overall quality level of the blouse (natural fabric versus synthetic, fine versus coarse, machine made versus hand-stitched, etc.), I wonder if the internal comparison would have been more accurate? Is it possible that, although we may not be able to discriminate accurately in terms of objective measurement (without instruments), we may be able to gauge quality relatively accurately even without direct and immediate comparison?

"Quality" - a fascinating topic, bigger than the entire HUG archive itself. Have you by chance read Zen and the art of motorbike maintenance by Robert Pirsig? This book is about trying to get to the heart of the term 'quality'. It's both impossible to define and yet universally understood. As I recall Pirsig commenting (or if he didn't, apologies) that were an Aborigine living an isolated life in the jungle presented with a well preserved 60s car or a brand new Porche and asked to pick the one in his cultural terms of 'best quality', he and the sophisticated New York city dweller would both pick the Porche. But asked to define why and to explain in detail and they'd both struggle using available language to define this illusive 'quality' issue.

Actually, regarding my wife, it is merely a matter of colour match for her: she would have pre-selected in the store the shape and feel that she wanted. She would not have bought the whatever if she was in doubt about it's 'quality' for her needs.

Ok, back to the supermarket shelves. It's really difficult isn't it. Yes, unlike music, the image is flat, two dimensional. You could look at it for an hour or a year and the contents of these shelves will not change. Even though the image is static, we're still having great difficulty identifying the difference. Consider how much more complex music is. In effect, it's 5D: it has loudness, it has pitch, it has rhythm, it has tempo and it has a frequency spectrum: a huge amount of asynchronous data for our brain to absorb and interpret.

Supermarket shelves - 2 second exposure, 2 second gap.

Any easier?

[A.S.]

As a lay person, I went through a long journey to finally settle with my current cables, Cd players , amplifiers and Harbeth for my musical enjoyment. But if I were to start my journey with a proper scientific mind and conducted a ABX test on each equipment then I would be still listening to some mediocre speakers and equipments because on many occasions I just failed under the DBT ABX tests.

Under the scientifically recognized ABX DBT there is no way I could properly do a valid comparisons with Harbeth and a X brand speakers. Scientifically, it is required that I do an instantaneous switchover to compare between two sets of speakers. But how is that possible? Firstly, how are we going to place two sets of speakers in an identical position in succession and instantaneously? Physically that is an impossibility unless I have some magical powers that I could replace the speakers in the exact position with a flick of the fingers. Can then someone argue that I am disillusined for saying Harbeth is one of the finest speakers that money can buy? Am I wrong to say they are natural and detailed speakers just because I could not do a scientifically accepted ABX test to prove that Harbeth was the better speaker?

Remember under the ABX rules everything must be equal before the test starts....

First, I don't know from where you quote A - B -X (meaning: A and B are two pieces of different equipment and X is one of them reintroduced in disguise as a control). I've never said I used that technique have I. I use A-B testing, that is, a direct flip-flop between the two. There is absolutely no need for me to try introduce X, the catch-you-out with all the extra complexity of designing the test, having a second person present to monitor your results, they alone knowing what X is. The differences are either so large that the A-B process swamps any residual bias or so small that they are inaudible. And as Harwood said ".... inaudible means just that .... inaudible." Whether A-B, A-B-X, leisurely hooking-up, reassembling, changing-over with no urgency over minutes or hours, the most essential factor is to try and get a grip on your emotions. The whole purpose of any quasi-scientific comparison is to remove emotions because they will cloud your judgement. Close your eyes, cover the equipment, turn your back and face the other way and listen to the reflected room-sound: being honest with yourself and don't prejudge - those skills are all you need. The A-B test merely sharpens the contrast (because our memory is so poor as I've illustrated above). The value is not in A-B process, it's in the heightening of the edge between black and white by juxtaposition.

Once you've made the best objective decision as to which is factually superior (against whatever objectivity scale you have) then stick to that decision. Remove any self-doubt. Resist the temptation to constantly trade-up (or trade sideways): get on with extracting your entire emotional fulfilment from the music not the gear.

I'm well aware that comparing different speakers in the room on an A-B basis is challenging. But if you'd let me show you in due course, with audio examples in this thread here I think you'll find that the differences in sound are sufficiently great that even though the comparison is far from perfect (because the speakers do not occupy exactly the same physical points in the room), there are work-arounds and the result is still useful and valid. You would expect me to have, after 30+ years of doing speaker A-B comparisons, a fairly well proven system wouldn't you considering what I've designed.

From the commercially aware audio equipment designer's perspective (i.e. mine) assuming that I have created some audio product the process works like this ....

1) Have I created the product for my own entertainment or amusement or do I intend to sell it commercially .... ?
2) .... in which case, will anyone buy it .... is it marketable?
3) How can I assess whether it stands a chance of finding a market niche before I commit to production without expensive and potentially unreliable market research?
4) The first step surely is to compare my product's sonic performance with existing, on-the-market products since I would be in direct competition with those existing products, their fans and supporters, dealers, distributors and media exponents so ....
5) ... beg, buy or borrow competitive products already on the market and selling and respected and set-up an A-B instantaneous comparative evaluation and that means ....
6) remove all emotions from the comparative process because ....
7) if I cannot be truly objective and be truthful about the sonic qualities of my own product v. competition I will run the serious risk that ....
8) .... when I bring it to market, others, detached from the emotional process of product conception, creation and marketing will readily determine my product's weaknesses and ....
9) my product will flop and ....
10) I'll lose my business, house, family, marbles .....

Simple stuff. The longer you listen, the more emotionally bonded you become. Removing that emotion allows you to be honest with yourself. I am the one who demonstrated a pair of prototype speakers at a public hi-fi show years ago with the tweeters inadvertently wired (by my own hands) out of phase. I had tears in my eyes at the beauty of my newborn creations because what I heard touched me on an emotional level - yet there must have been a huge hole in the frequency response. Nobody noticed - we were all carried along with the beautiful music: but had that pair been loaned to a journalist with test equipment and disconnected from the emotions of the event I would have been humiliated: and rightly so.

Upon discovery and to my shame, I promised myself never to place such lunatic over-confidence in my hearing alone and never work far from test equipment.

[EricW]

"Quality" - a fascinating topic, bigger than the entire HUG archive itself. Have you by chance read Zen and the art of motorbike maintenance by Robert Pirsig? This book is about trying to get to the heart of the term 'quality'. It's both impossible to define and yet universally understood.

Yes, absolutely, I read Pirsig's book when I was quite young and have re-read it a couple of times since. Even though it was published as a popular book, I think it's actually quite a profound philosophical work and may well be remembered as one of the significant books of the 20th century.

What I was trying to get at, I suppose, relates to a vague idea about the perception and discrimination of quality (as of, for example, a loudspeaker) as opposed to, say, the ability to perceive specific physical differences by means of sense perception. You've demonstrated quite convincingly that unless you perform an instantaneous A-B comparison of two sensory inputs that are relatively close to each other, the brain won't likely be able to distinguish between them because it's not good at assessing a specific physical property - exactly colour, say - by reference to an absolute scale of measurement.

But ... if the question is not "is it different?" but "which is better?", does that change the assessment? Can our brains generate a reliable impression of better/worse even if they are not able to analytically measure and specify specific physical properties of the sensory input?

Also, the A-B comparisons you have shown raise another question. Any gap between A and B samples reduces the ability to compare accurately, true. But at some point, if the differences are sufficiently large, the gap is going to cease to matter. Two similar but not identical shades of orange may be difficult to pick apart, but orange and purple are going to be easy no matter how large the gap is. So is there any way to quantify or explain what degree of similarity must already exist before an instantaneous comparison becomes necessary in order to detect any remaining differences?

[A.S.]

... (1) But ... if the question is not "is it different?" but "which is better?", does that change the assessment? Can our brains generate a reliable impression of better/worse even if they are not able to analytically measure and specify specific physical properties of the sensory input?

(2) Also, the A-B comparisons you have shown raise another question. Any gap between A and B samples reduces the ability to compare accurately, true. But at some point, if the differences are sufficiently large, the gap is going to cease to matter. Two similar but not identical shades of orange may be difficult to pick apart, but orange and purple are going to be easy no matter how large the gap is. So is there any way to quantify or explain what degree of similarity must already exist before an instantaneous comparison becomes necessary in order to detect any remaining differences?

Before I answer that, I've now cut the supermarket shelves exposure to one second - Supermarket 1 sec. exposure
Any easier to make a comparison now?

(1 >)How do we know 'which one is better'? We'll, to work towards that we have to introduce a known reference into the comparison. Since all loudspeakers merely generate an approximation of the original live sound, it's not as if we can make A or B some super high-precision loudspeaker. Even presenting the original musicians in front of us playing the same notes on the same instruments at the same tempo won't help: the loudspeaker reproduction experience is significantly different (in the way it fills the room) to the live instruments that we'd have created layer upon layer of experiential difference which would be very difficult to break down into digestible, resolvable components. We weould be overwhelmed with observational data, as useless as having too little data. We'd be best to use sounds for which we innately have an internal comparative reference: human speech would be ideal for that. That leads to (2) ....

(2 >) if we play speech on our A and B loudspeakers and flip-flop between them without a gap, we sharpen the contrast between the sonic events and this makes it much easier to at least hear differences. How important they are, how we rank them, even how a suckout in one compares with a mirrored peak in the other - these need careful thought and the ability to feel how one's brain is pulled this way and that as we interpret what we hear. We need to stand outside ourselves as best we can and just watch how we are influenced, own dispassionate observer. It makes me think of sitting outside a cafe on lovely warm day as a time traveller, watching oneself making a complete hash of parking a large car in a rather small space right in front of us.

We can sit around all day theorising about the outcome of an A-B test - as over 32,000 hits have demonstrated on just one of several other forums discussing my challenge this past two weeks. But until someone of the 32,000 actually rolls up their sleeves and puts themselves in the A-B hot seat they are just talking hot air. You can pontificate about bungee jumping 'till the cows come home but learning comes from actually taking that step into the unknown and feeling for yourself the pull of gravity. And then recognising that sensation for the rest of your live.

[A.S.]

The supermarket shelves exposure has no gap between A and B now. Now we have the exact equivalent of the operation of my A-B audio switchover box. Now that the gap between A and B is eliminated, how easily are you able to detect the differences between image X and Y? Supermarket A-B no-gap.

The point to stress here is that as we will surely have seen, the further separated in time exposure to X and Y are, the less precise our analysis of the events becomes. This is due to the electro-chemical nature of our brain: we do not store data as a hard disk does in certain, permanent bits and bytes. Our memory reduced the experience to some 'essential components' and stores that. How and why it selects certain features I do not know: we may even live with a partner our whole lives and not be able to pick-out the colour of their eyes or lips yet we recognise them immediately.

Because at the point of committing the experience to memory we 'data reduce' the experience (similar to lossy zip compression or lossy MP3) unless the contrast between the current experience and the ghost-memory of the previous experience is sufficiently dramatic, we are not able to detect a difference when separated by more than a second or so - as we've seen.

In my opinion, the proper evaluation of sound or image, taste or smell must be conducted under instantaneous A-B conditions, with as short a gap as possible.

[weaver]

Today was the first time I looked at these images on my work monitor as opposed to the laptop at home, also the first time I viewed 'full screen'.

Are the differences obvious when switched instantly? - yes

But a qualified yes; the obvious differences were obvious, but how about the fact that the two images have been (I assume) saved successively and that the (again, I assume) jpeg compression artifacts are actually different in both?

So go back to the original link in post #70 and view a (I assume) much higher resolution version of the same thing - now that looks different to either of Alan's images.

Well I've made three assumptions so far and here's another: I assume that like me, other users read Alan's post and looked for clear differences, the kind of differences that we actually see once we have seen them.

But go back to the original (post #70) - view full screen, then look at one of Alan's images full screen: is there a difference in quality? Even with a gap of several minutes would we say 'I don't remember it looking fuzzy like that' ?

This is an issue I have with looking - or listening - for differences, and I think it is what EricW was driving at a couple of posts back.

As an aside, on the day I saw Alan's first 'orange shirt' post I had indeed spent quite some time looking for a fabric sample (that wasn't where it should have been) as I had just started on a new production of an existing design and I wanted to check the colour match, according to my notes what I was weaving was correct but I still wanted to see the two samples side by side. On the other hand I would have known if the quality wasn't right immediately.

{Moderator's comment: if there is a quality difference (I'll ask him to check) that was unintentional. What he wanted you to spot was the packets of food he'd added to the shelves in the middle of the picture, a significant difference in the image regardless of the pixels etc.}

[A.S.]

We have today been asked by a prospective volunteer if we would redesign the test, deleting the A-B switch (introduced in post #6, purpose explained #28, construction and parts list in post #44) and permit him to listen for ten minutes or so to a track, change the amplifier and listen again.

Since this entire thread, and the over 35,000 internet viewings on sites that have reported this thread, is focused on the instantaneous nature of the relay-operated A-B switch, we have declined and asked the poster for his permission to post his approach here. If we receive permission, we will post it and our response. In our view, that suggestion is a wholesale re-write of the terms we laid out so clearly at the beginning. We remain significantly flexible about the precise details of the procedure, have a completely open mind to any equipment/brands/models the participant may wish to bring providing that they are of broadly comparable technical performance (i.e. are checked here that they measure well into a real-world speaker load) but the A-B relay switch is an absolute must-use.

The whole purpose of this thread, as explained in the above posts is to illustrate that under (blind, level matched) instantaneous A-B listening comparison the listener may arrive at a different opinion about the two items under comparison compared with his sighted, non-level matched opinion. There are frequent postings here on HUG asking the question 'what amp should I upgrade to'. As I have stated before, a reliable answer - the best answer I'd say - is to factor in styling, cosmetics, features, after-care, brand image and the like as primary purchase motivators, and give the vague 'sound quality' the correct weight in the purchase decision. A good dealer, working with a brand from the inside, should very well know about their after care support and be able to advise on this. As I said in post #6:

Amplifiers should be selected not for "sound quality" (whatever that is) but for facilities, design integrity, durability, after care and likelihood of being able to source service parts in 5 or 10 years. Also, if you're really serious, a quick peek at the financial standing of the brand might tell you if they are likely to have the financial resources to weather the economic downturn of the next few years which is effecting the whole consumer electronics sector. Some of the most likely surviving (UK) brands rarely advertise, rarely exhibit, have relatively old-fashioned designs, use standard parts, have been around for a generation and are rock solid businesses. They deserve to be respected and supported because they will be there to look after you in 10+ years.

I am certain that there are many alternative, equally valid, comparative tests that can be conducted on any manufactured engineering-based product such as audio equipment. Some would use measurement equipment solely, some listening solely: it's not for us to claim exclusivity over test methodology. What we are saying, and I repeat this for the very last time is this from post #28:

1) We definitely, 100% do believe that people (including me) can, do and will hear differences between amplifiers randomly put in front of us, hooked up, switched on and turned up!

2) We have observed that for psychoacoustic reasons that we do not understand or pretend to understand, when those very same amps are level matched (with test equipment) and then connected to a good quality relay change-over driving one pair of speaker, that when the change-over is operated entirely at random, those big initial differences heard in 1) diminish to little or nothing. So much so that it's actually not possible (in many instances) to hear any discontinuity or change in the music at all, except the minute gap as the relays flick over - perhaps one thousandth of a second. Even that may not be audible. Surprising, yes it is to me as well. This is, I agree, counter-intuitive.

Comparisons which remove the instantaneous relay switch-over arrangement may or may not be valid - we cannot comment on that - but they are a different sort of test to the one we propose, and have proposed from the start. They are not what this thread is about. The absolute minimum elements for our test are the use of precision instrumentation to be certain about the loudness of A and B at the ear and the instantaneous way of selecting A or B. Other tests, approaching the matter from alternative directions would not mandate these elements. But we do. They are absolutely non-negotiable.

As at today, we have not received one proposal to run the comparison as we defined with the switch-over box. There is one week remaining.

[STHLS5]

The whole purpose of this thread, as explained in the above posts is to illustrate that under (blind, level matched) instantaneous A-B listening comparison the listener may arrive at a different opinion about the two items under comparison compared with his sighted, non-level matched opinion.

Ok, back to the supermarket shelves. It's really difficult isn't it. Yes, unlike music, the image is flat, two dimensional. You could look at it for an hour or a year and the contents of these shelves will not change. Even though the image is static, we're still having great difficulty identifying the difference. Consider how much more complex music is. In effect, it's 5D: it has loudness, it has pitch, it has rhythm, it has tempo and it has a frequency spectrum: a huge amount of asynchronous data for our brain to absorb and interpret.

I am not suggesting you to rewrite the rules but am curious as to the reliability of the AB test to determine the absolutes in different amplifiers.

Just as you mentioned the 5D elements in music, I was rethinking the concept of open sound and speed that we discussed in the other threads. I am starring at two samples that appears to me identical yet distinctly different which is not visible under instantaneous AB tests. These visual samples now introduces another element that our brain introduce a third concept that is beyond the measurements of the instruments. And it is 100% repeatable.

[Attachments not working]. To those interested for the attachments pls PM me.


ST

NOTE: A.S. MANAGED TO ATTACH A TEST FILE TO THIS POST. WHAT ACTUALLY HAPPENED S.T. WHEN YOU ATTEMPTED TO ATTACH?

[A.S.]

I am not suggesting you to rewrite the rules but am curious as to the reliability of the AB test to determine the absolutes in different amplifiers.

Just as you mentioned the 5D elements in music, I was rethinking the concept of open sound and speed that we discussed in the other threads. I am starring at two samples that appears to me identical yet distinctly different which is not visible under instantaneous AB tests. These visual samples now introduces another element that our brain introduce a third concept that is beyond the measurements of the instruments. And it is 100% repeatable....

I'd like to see that.

Are we not going around in circles here dodging the central issues? The issue is this: for 25+ years since I took over, every model I have designed and released - and subsequently has sold very well to wide customer acclaim - has been designed using an A-B relay switch-over, starting with my first design, the original HL Compact of 1987. In those circumstances, would it not be reasonable to say that the A-B switch concept, at least for me as a designer with my livelihood on the line model after model at least works for me? That is not to say that the unskilled, inexperienced consumer, totally disconnected from the design experience would automatically be able to use such an analytical tool, but it would be a denial of fact to say that conceptually it is flawed. It isn't. It is super-analytical. It is the very best technique because by heightening the contrast in a millisecond between sonic event A and event B it gives our ear a fighting chance of hearing extreme subtlety.

The issues as revealed by a casual inspection of around 40,000 hits on internet forums of 'Harbeth's challenge' seem to me to break down into these categories ....

1) Outright rage and fury that someone dares to suggest that such a simple A-B switchover could even be conceivably used to expose sonic differences between two amplifiers, therefore immediate denunciation of the concept and me as a proponent
2) Natural suspicion, general disbelief but a willingness to prove the A-B can work to the participants advantage and expose big sonic differences, hence easily win the prize
3) Quiet curiosity, no firm opinions
4) General agreement that A-B tests have been previously shown to reveal the outcome Harbeth expects: that the differences under A-B are very much smaller than sighted, non-level matched
5) Compete agreement that A-B is the only scientific method and works with the limitations of the human ear, not against it.

Our overview is that slowly but surely the general view has worked from 1 to 4-5. The updated participants situation as reported here in post #58 is updated.

I believe that the real anxiety here comes from complete (and to be expected) inability of the consumer to imagine what it is actually like to be in the hot seat as the participant performing the test. It's not like being in the dentist under duress. It's not at all like an invasive medical examination. It's not frightening. You are not sweating. Your heart is not racing. There is no-one in a white coat with an electric prod ready to beat a confession out of you. The participant is completely in control. He can switch-over whenever he wants. He can get up, walk around, take a phone call, brew a cup of tea, eat his sandwiches, do some typing, go to the toilet, do some simple exercises ... he is not incarcerated in the listening room, glued to the hot spot. He can switch-over as and when the mood takes him. Maybe at first he'll switch over every 10 seconds or so, then every minute of so, then perhaps leave it in A or B for five minutes, then back to rapid switching .... there is no fixed way. So, if the participant was intent in completing the "100 switch-overs" I have mandated in the fastest time, this could be done in 100 x 10 seconds = 1000 seconds = 16 minutes. Has this been appreciated?

There is absolutely no reason to fear this A-B process. There is no right or wrong answer. It is not a group sport. The desired outcome is that the participant - and the participant alone - reaches his personal conclusion about the relative sonic merits of A and B. He can keep them secret. He can even lie: he may say in his heart that he couldn't hear a difference between A and B at the point of switch over, but he may put a public face on which says he clearly heard a night-and-day experience. I don't care! All I can say is that having no reason to deceive myself, having solely scientific curiosity as my motivator, under the conditions I have described, with the simple equipment I have built (the relay switcher), I have reached certain conclusions model by model, circuit alternative by circuit alternative. And those conclusions have generated tens of millions of pounds of revenue and a lot of satisfied customers around the world.

In growing anticipation that that the unfamiliarity of the A-B test is fuelling anxiety, I am now obliged to go one step further and actually show you, by means of a video with an accompanying high quality audio, exactly how it works in practice. The video has been made and will be edited and uploaded. Perhaps I should have done this at the very beginning. I had no idea that the A-B concept that has kept a roof over my head, educated my children and positioned Harbeth where it is, could be so feared. It is a superb tool for objective audio comparison. Because the A-B test is instantaneous comparative, unlike any other method which relies on human hearing and memory in peak condition, A-B testing is an insurance policy. It ensures that even in old age when my hearing is far past its best I can still design and work providing I can switch between a speaker that we know sound right - such as one of today's Harbeths - and something under development.

I don't fear old age and the natural degeneration of my hearing because of using the A-B switcher.

[Macjager]

I decided to do a pseudo experiment at home, not using the technique as described here, but to confirm what I feared. Based on everything that I have read here and the counter "arguments" spouted on other sites, I thought I should change my amplifiers around.

So, I disconnected my two mono blocks, worth a stupid huge amount of money, and replaced them with a Class "D" amp, "Big Joe Power Amp"...same set up, same position, same pre-amp, DAC, etc....I certainly can't hear any difference while listening, the so called theory of "you must listen for a long time to really "understand" your amp's sound" certainly does not seem to work in my cranium....

Needless to say, I am beginning to think that I have a hell of a lot of money tied up in stuff, for no valid reason...at least in stuff that is not affected by gravity. Mechanically designed items, which must fight gravity, ie turntables (downward pull of gravity thus arm counter weights and anti-skating [I like listening to records as well as digital]) and speakers (ie cones and air) need to have more quality engineering to overcome Mr Newton's Laws...as long as they use science and not marketing as the first step in design.

Without being too cynical, I guess the higher the price, the higher the mark up, the higher the profit...

Funny thing about the challenge is the number of words spilt to try and prove Alan wrong, and not one critic has stepped forward to take the challenge...sound like The House of Commons...


Cheers

George

[Kumar Kane]

I imagine that this test can also be used to determine the truth about:

1. Sonic differences caused by cables
2. Sonic differences caused by the so called burning in of non mechanical components like amplifiers and cables
3. Sonic impact of power supply conditioners, isolation platforms and other such related improvement causing exotica
4. Sonic difference between different upstream CD players - or between SACD and CD
5. Sonic differences between digital sources - as in wirelessly streamed lossless bitstreams and CD, as well as between lossless and lossy files
6. Sonic impact of adding an external DAC
and many others.

If the outcome of any of the tests is that there aren't any sonic differences in any of the categories, the next conclusion will be that because the test does not reveal the sonic differences, the testing process or protocol is defective.

Human nature will never change, people will always believe what they want to believe.

[A.S.]

I imagine that this test can also be used to determine the truth about:

1. Sonic differences caused by cables
2. Sonic differences caused by the so called burning in of non mechanical components like amplifiers and cables
3. Sonic impact of power supply conditioners, isolation platforms and other such related improvement causing exotica...

Absolutely. Now I think about it, I built the Mark 0 switch box in the mid 80s before I took over at Harbeth, in my audio hobby days, to instantaneously compare two loudspeaker cables driving the same pair of speakers. Click, click .... click, click ...... click, (listen longer) click ........ That told me all I needed to know.

Attached are pictures of the Mark 3 switcher. You'll see this in use in the video. For a product that will be launched in Munich at the beginning of May I have spent months making high-quality, believable acoustic measurements and then feeding those drive unit measurements into the simulator, predicting how the speaker would behave with different crossover topography, finally having to build both 'best candidate' crossovers, and then sit and carefully listen to them. The Mk3 box has the ability to switch two entirely different, physically separate crossovers into one pair of crossover-less speakers. The differences, even on instantaneous switch over are exceedingly small. They are so small that if I coughed, clapped my hands together, got up and immediately sat down, tied my shoelaces or in any other way just lost concentration for less than a second, I wouldn't be able to tell them apart. But there had to be one better than the other. The only way to tease out such tiny differences is by instantaneous comparison, as you'll see/hear from the video. So yes, this type of box can be used to compare two anythings.

Pictures of Mk3 box attached with two foot switches: red) latched change-over where the box is locked into A only or B only for longer term listening to one only, and black) instantaneous A-B switch and can also be operated by hand. Simple, yes but deadly effective.

The essence of this is not to convert Druids to Christians (or vice versa) but to empower the listener to make objective, repeatable decisions that separate emotions from facts. Whether the participant then ignores the experience and makes the opposite purchasing decision he is perfectly entitled to do. Even encouraged to do if the product he selects is more expensive or profitable for the seller! The specialist audio industry needs consumers to spend. Dealers need customers upgrading. My point is solely to prioritise 'amplifier sound quality' somewhere in a ranking of really important features and considerations, which for me would put durability and serviceability/service backup right at the top of the tree.

When my amplifiers need servicing I can pick-up the phone to the UK service department, talk to someone friendly and competent who know the amps inside out, and then day later, a brand new empty shipping carton will arrive for me to place my amp in and send off for service. Why the new carton? 'Well, sir, we want to be 100% sure that your precious amplifier reaches us in perfect condition ....'. Now that's what I call a real product benefit not some airy-fairy fantasy performance claim. When it fails I want someone to take responsibility for it, sort it now and return it to original spec and then to me at the lowest possible cost. It is an immense help if the design is modular, uses bog-standard industrial parts not esoterica and is easy to disassemble. Those benefits are what should be promoted at the point of sale. They truly are worth paying for.

This A-B method works. End of story.

>

[STHLS5]

I'd like to see that.

.... Because the A-B test is instantaneous comparative, unlike any other method which relies on human hearing and memory in peak condition, A-B testing is an insurance policy. ....

It is really confusing. I thought I saw a line about AB test and prepared a reply to that but I couldnt see it anymore. Maybe, I am hallucinating.

Here are two links to videos that look identical. I would appreciate if anyone can find some flaws in the clips that can tell them apart. Ignoring the background music, I find it that under instaneous switching of few seconds I couldnt tell which is which but if the exposure time is more then I could easily identify the difference. 100% score. I must say that I thought a bit out of the box because audio as known to us, especially in stereo, is more of tricks fooling our mind about depth, image, saoundstage and etc etc.

I am not doubting AB and have used it many times. However, two events (about speed and another one about the missing bass in one of the in soundtrack) recently I realized I may be hearing things differently and therefore asking all of us to rethink the AB approach. Under Alan's challenge I doubt anyone could spot the difference.

Would have been easier if I could insert the video here.

See http://youtu.be/ZxR7NpH55TQ

and here http://youtu.be/yFXgOiarchE

I left the background music on so that you know you are viewing a different clip. Mute the video and try again. Some may find it difficult. More details later.

ST

[EricW]

.. under instaneous switching of few seconds ...

ST

A contradiction in terms. If there's a gap of a few seconds, it's not instantaneous. Alan's examples have satisfied me that any gap, even a very small one, dramatically reduces the ability to compare reliably.

[A.S.]

A contradiction in terms. If there's a gap of a few seconds, it's not instantaneous. Alan's examples have satisfied me that any gap, even a very small one, dramatically reduces the ability to compare reliably.

Undoubtedly true. But there are two other issues ...

1) Both clips sound distorted under peak drive on my PC speakers which is distracting although probably not relevant to whatever point you are making (I'm guessing) and
2) The music is not continuous, and the tonality, content and structure of the music is different between the two clip. In short, the music has moved along and now we have the huge extra complication of trying to separate out in our mind what is a difference due to performance, may even be different recording 'takes', different venues, different performers, different microphones/positions, fresh/tired performer - the list of uncontrolled variables is endless. Surely we need to control and eliminate all other variables and just focus attention on the one factor change that we have deliberately introduced. How else can we get to the nugget of truth?

A simple example: we recently looked in great detail at the operation of the pianos pedals, with sonic examples. We observed that the sustain pedal is all-critical in livening up the sound, extending the decay and generally brightening the tone. Because the two excerpts to my ears have a different musical structure, how do I know if I'm hearing the different use of sustain as opposed to some feature you are expecting me to hear? I quote my previous comment:

To start with, western music usually has a structure: a beginning, a middle and an end. During that journey there is usually a change in pace, pitch and intensity. The listen-to-the-whole-track concept miserably fails as an objective comparison because that emotional journey and conclusion of the first song is then compared with the opening statement of the song played again from the beginning. That is highly confusing to our brain: to be led to an emotional climax after which our brain is awash with endorphins and then, after a pause, to start-out again.

The clips are sufficiently different in their musical structure that I can't even make a reliable spectral analysis of them: they are not directly comparable as I see it. Would you like to have another go?

[A.S.]

OK, let's remove as many uncontrolled variables as we can and get down the the heart of this. Let's use some old fashioned, easy to control and understand, sine wave tones. Actually, in preparing this post and these clips I've discovered an unexpected and highly beneficial aspect of using the mechanical relay change-overs. More on that later.

As I understand it, orchestras tune-up with one performer, sure of his pitch, generating a note at a certain frequency (or pitch) on his instrument against which all other instruments adjust themselves. To quote from Wikipedia here:

The A above middle C is usually set at 440 Hz (often written as "A = 440 Hz" or sometimes "A440"), although other frequencies are also often used, such as 442 Hz

But be aware! This concert pitch A440 is not written in stone by god: it is an entirely artificial creation, and perhaps surprisingly, is not consistent between continents or orchestras. Can you believe that there is a website devoted to advising what individual orchestras consider to be their reference pitch? It's true: here. Just to pick out some examples at random:

• Royal Scottish National Orch. Glasgow 443Hz
• London Classical Players 430
• London Philharmonic Orchestra 440
• Leningrader Philharmoniker 444
• Sydney Symphony Orchestra 441
• Wiener Philharmoniker 443
• Orquesta Sinfonica de Tenerife 443

So, small differences in tuning pitch. Now, supposing that you are a professional musician and you complete a performance today in London and fly to Glasgow tonight. You are expected for rehearsals at 10am sharp tomorrow, ready to go and you have been warned in advance about the pitch difference. In your hotel room tonight you have to adjust your instrument so that as you arrive tomorrow morning you are not going to hold things up fiddling around with tuning. Can you do it? Could you detect the difference in pitch? Ten rooms down the hotel corridor is another musician. He's using a digital tuning aid and has just tensioned his strings until the green 'in tune' lamp illuminated. You forgot yours aid and he's reluctant to lend you his. He wants to practice. The best he's willing to do is bow the reference pitch on his instrument, let you listen and send you scuttling back down the corridor, find your door card, let yourself into your own room, sit down, get yourself ready and play the note on your instrument adjusting tension as necessary.

What do you think of that process? How much confidence would you have in your audio memory to tune like that; how risky would it be that you arrive out-of-tune at the rehearsals? Who'd want to book you again? You're supposed to be a professional for goodness sake.

It's very easy for us to generate these sine waves. Would you like to hear the difficulties the audio memory has with audio minutia?

[Euler]

It is super-analytical. It is the very best technique because by heightening the contrast in a millisecond between sonic event A and event B it gives our ear a fighting chance of hearing extreme subtlety. [snip]

Because the A-B test is instantaneous comparative, unlike any other method which relies on human hearing and memory in peak condition, A-B testing is an insurance policy.

A healthy regard for the unreliability of our senses when it comes to making objective comparisons leads me to think Alan must be right about the A-B test -- at least when it comes to what one might call LOCAL sonic attributes -- like naturalness, resolution, and transparency -- sonic properties (of speakers, amplifiers, etc) that reveal themselves in the minute (small time interval) structure of the music. But could it be that the A-B test might not be the best way to evaluate more GLOBAL sonic attributes -- like the soundstage, the central phantom image, speaker disappearance, the curtain of sound -- that reveal themselves more in musical passages than in musical milliseconds?

Of course, in the design of speakers it seems it would make sense to focus on the local sonic properties, since they (together with speaker placement and the room acoustics) probably determine or help to determine the global sonic properties.

Bruce

[STHLS5]

A contradiction in terms. If there's a gap of a few seconds, it's not instantaneous. Alan's examples have satisfied me that any gap, even a very small one, dramatically reduces the ability to compare reliably.

I could have joined both clips in one long video and asked which part is which. The point is despite the large gap you cannot tell the difference. The audio is irrelevant in those clips. Mute it and concentrate on the snowy video.

...The clips are sufficiently different in their musical structure that I can't even make a reliable spectral analysis of them: they are not directly comparable as I see it. Would you like to have another go?

Ignore the audio. The clips are about the video. Re-lable the clips or rearrange them in anyway you want. Our brain is capable of distinguishing the random dots in to a meaningful visual clue.


Used IE9 W7 but may have used the compatibility mode accidently.

ST

[kittykat]

how are we going to place two sets of speakers in an identical position in succession and instantaneously? Physically that is an impossibility...
ST

think ive read somewhere that a major electronics organisation uses a turntable platform/ moving platform which can swap different speaker setups, albeit not instantaneously.

{Moderator's comment: Yes, JBL}

[STHLS5]

A contradiction in terms. If there's a gap of a few seconds, it's not instantaneous. Alan's examples have satisfied me that any gap, even a very small one, dramatically reduces the ability to compare reliably.

This message is from IPad.

The Purpose of this thread is to show that our human memory for sound and sight is unreliable that without instantaneous switching you can't possible correctly judge the sound or image. Looking at the video clips (IGNORE THE AUDIO in the CLIPS) under instantaneous switching it makes no sense to me. If I were to look at the random moving black and white dots I can't and most likely anyone could tell which of the clips it represent.

Unlike, the earlier supermarket picture, the shorter the exposure time the more difficult and it will be impossible for our brains to resolve and interpret the meanings in the clips. The fuzzy snowy clips indeed contain an image with different posture but it requires patients and the right exposure and a bit of a practice. My point is if I were to use this samples under AB without the subject is made aware what it would represent then there would be a high probably that no one could tell the difference between the clips.

Could you tell me without referring to the earlier clips whether the the X clip is the same as A or B? You can take either of the clip and ask me the same.

ST