Topic: Richard Clark Amplifier Pick-out Challenge

[Nelson de Leon]

Taken from:

http://tom-morrow-land.com/tests/ampchall/

Richard Clark $10,000 Amplifier Challenge FAQ

by Tom Morrow

Written 6/2006

The Richard Clark Amp Challenge is a listening test intended to show that as long as a modern audio amplifier is operated within its linear range (below clipping), the differences between amps are inaudible to the human ear. Because thousands of people have taken the test, the test is significant to the audiophile debate over audibility of amplifier differences. This document was written to summarize what the test is, and answer common questions about the test. Richard Clark was not involved in writing this document.

The challenge

Richard Clark is an audio professional. Like many audiophiles, he originally believed the magazines and marketing materials that different amplifier topologies and components colored the sound in unique, clearly audible ways. He later did experiments to quantify and qualify these effects, and was surprised to find them inaudible when volume and other factors were matched.

His challenge is an offer of $10,000 of his own money to anyone who could identify which of two amplifiers was which, by listening only, under a set of rules that he conceived to make sure they both measure “good enough” and are set up the same. Reports are that thousands of people have taken the test, and none has passed the test. Nobody has been able to show an audible difference between two amps under the test rules.

This article will attempt to summarize the important rules and ramifications of the test, but for clarity and brevity some uncontroversial, obvious, or inconsequential rules are left out of this article. The full rules, from which much of this article was derived, are available here and a collection of Richard's comments are available here.

Testing procedure

The testing uses an ABX test device where the listener can switch between hearing amplifier A, amplifier B, and a randomly generated amplifier X which is either A or B. The listener's job is to decide whether source X sounds like A or B. The listener inputs their guess into a computerized scoring system, and they go on to the next identification. The listener can control the volume, within the linear (non-clipped) range of the amps. The listener has full control over the CD player as well. The listener can take as long as they want to switch back and forth between A, B, and X at will.

Passing the test requires two sets of 12 correct identifications, for a total of 24 correct identifications. To speed things up, a preliminary round of 8 identifications, sometimes done without levels or other parameters perfectly matched, is a prerequisite.

Richard Clark normally has CD source, amplifiers, high quality home audio speakers, and listening environment set up in advance. But if the listener requests, they can substitute whatever source, source material, amplifiers, speakers (even headphones), and listening environment they prefer, within stipulated practical limits. The source material must be commercially available music, not test signals. Richard Clark stipulates that the amplifiers must be brand name, standard production, linear voltage amplifiers, and they must not fail (e.g. thermal shutdown) during the test.

Amplifier requirements

The amplifiers in the test must be operated within their linear power capacity. Power capacity is defined as clipping or 2% THD 20Hz to 10kHz, whichever is less. This means that if one amplifier has more power (Watts) than the other, the amplifiers will be judged within the power range of the least powerful amplifier .

The levels of both left and right channels will be adjusted to match to within .05 dB. Polarity of connections must be maintained so that the signal is not inverted. Left and Right cannot be reversed. Neither amplifier can exhibit excessive noise. Channel separation of the amps must be at least 30 dB from 20Hz to 20kHz.

All signal processing circuitry (e.g. bass boost, filters) must be turned off, and if the amplifier still exhibits nonlinear frequency response, an equalizer will be set by Richard Clark and inserted inline with one of the amps so that they both exhibit identical frequency response. The listener can choose which amplifier gets the equalizer .

FAQs:

How many people have taken the challenge?

Richard Clark says over a couple thousand people have taken the test, and nobody has passed. He used to do the test for large groups of people at various audio seminars, and didn't charge individuals to do the test, which accounted for the vast majority of the people who did the test. Around 1996 was the last of the big tests, and since then he has done the test for small numbers of people on request, for a charge ($200 for unaffiliated individuals, $500 for people representing companies).

When did the challenge start?

Sometime around the year 1990. Richard Clark says in a post on 7/2004 that the test with the $10,000 prize started about 15 years ago.

What were the results of the test?

Nobody has ever successfully passed the test. Richard Clark says that generally the number of correct responses was about the same as the number of incorrect responses, which would be consistent with random guessing. He says in large groups he never observed variation more than 51/49%, but for smaller groups it might vary as much as 60/40%. He doesn't keep detailed logs of the responses because he said they always show random responses.

Is two sets of 12 correct responses a stringent requirement?

Yes. Richard Clark intentionally made the requirements strict because with thousands of people taking the test, even random guessing would eventually cause someone to pass the test if the bar was set low. Since he is offering his own $10,000 to anyone who will pass the test, he wants to protect against the possibility of losing it to random guessing.

However, if the listener is willing to put up their own money for the test as a bet, he will lower the requirements from 12 correct down to as low as 6 correct.

Richard Clark has said “22 out of 24 would be statistically significant. In fact it would prove that the results were audible. Any AVERAGE score more than 65% would do so. But no one has even done that”.”

Do most commercially available amplifiers qualify for this test, even tube amplifiers and class D amplifiers?

Yes. Nearly all currently available amplifiers have specs better than what are required for the test. Tube amplifiers generally qualify, as do full range class D amplifiers. It is not clear whether Richard Clark would allow sub amplifiers with a limited frequency response.

Besides taking Richard Clark's word, how can the results of the test be verified?

Many car audio professionals have taken the test and/or witnessed the test being taken in audio seminars, so there isn't much doubt that the test actually existed and was taken by many people. One respected professional who has taken and witnessed the test is Mark Eldridge. Because the test has been discussed widely on audio internet forums, if there were people who passed the test it seems likely that we would have heard about it. Sometimes there are reports of people who believe they passed the test, but upon further examination it turns out that they only passed the preliminary round of 8 tests, where levels were not matched as closely as for the final test.

How can audio consumers use the results of this test?

When purchasing an amplifier, they can ignore the subjective sound quality claims of marketers. Many amplifier marketers will claim or imply that their amplifiers have some special topology, materials, or magic that makes the sound clearly superior to other amps at all volume levels. Many consumers pay several times more than they otherwise would for that intangible sound quality they think they are getting. This test indicates that the main determinant of sound quality is the amount of power the amplifier can deliver. When played at 150W, an expensive 100W measured amplifier will clip and sound worse than a cheap 200W measured amp.

Does this mean all amps sound the same in a normal install?

No. Richard Clark is very careful to say that amps usually do not sound the same in the real world. The gain setting of an amplifier can make huge differences in how an amplifier sounds, as can details like how crossovers or other filters are set. When played very loud (into clipping), the amplifier with more power will generally sound better than a lower powered amp.

Most people perceive slight differences in amplitude as quality differences rather than loudness. The louder component sounds “faster, more detailed, more full”, not just louder. This perceptual phenomenon is responsible for many people thinking they liked the sound of a component when really they just liked the way it was set up.

I changed amps in my system to another one with the same measured power and I hear a sound quality difference. Does this show that the test results are invalid?

No. Installing a new amplifier involves setting the gains and crossovers, and any slight change you make to those settings is going to affect how things sound.

[Nelson de Leon]

Part 2:

As of early 2006 , there have not been any recent reports of people taking the test, but it appears to still be open to people who take the initiative to get tested.

Do the results prove inaudibility of amplifier differences below clipping?

It's impossible to scientifically prove the lack of something. You cannot prove that there is no Bigfoot monster, because no matter how hard you look, it is always possible that Bigfoot is in the place you didn't look. Similarly, there could always be a amplifier combination or listener for which the test would show an audible difference. So from a scientific point of view, the word “prove” should not be used in reference to the results of this test.

What the test does do is give a degree of certainty that such an audible difference does not exist.

What do people who disagree with the test say?

Some objections that have been raised about the test:

Richard Clark has a strong opinion on this issue and therefore might bias his reports.
In the real world people use amps in the clipping zone, and the test does not cover that situation.
Some audible artifacts are undetectable individually, but when combined with other artifacts they may become audible as a whole. For instance cutting a single graphic EQ level by one db may not be audible, but cutting lots of different EQ levels by the same amount may be audible. Maybe the amps have defects that are only audible when combined with the defects from a particular source, speaker, or system.
Some listeners feel that they can't relax enough to notice subtle differences when they have to make a large number of choices such as in this test.
There is a lack of organized results. Richard Clark only reports his general impressions of the results, but did not keep track of all the scores. He does not know exactly how many people have taken the test, or how many of the people scored “better than average”.
If someone scored significantly better than average, which might mean that they heard audible differences, it is not clear whether Richard Clark followed up and repeated the test enough times with them to verify that the score was not statistically significant.
Is there one sentence that can describe what the test is designed to show?

When compared evenly, the sonic differences between amplifiers operated below clipping are below the audible threshold of human hearing.

 

Links

Full Rules of the Challenge dated May 25, 2005
Richard Clark's comments on the challenge
A carsound forum thread about the challenge, containing more comments from Richard Clark.
 

Note from the author

I wrote this Summary/FAQ because I found that many of the people who disagreed with Richard Clark about the challenge simply didn't have the whole story on the challenge. I originally thought the challenge was flawed even after I read the rules a few times, but after reading lots of comments from Richard Clark, my objections were answered and now I believe that understanding the challenge is a very useful tool for learning what is audible and what isn't. I have no relationship with Richard Clark and have never communicated with him except that I've read his public postings about the challenge. If anyone finds typos or factual errors in this document please contact me

Mahirap itong challenge na ito because hindi lang isang beses mo kailangan ma-distinguish ang difference! Passing score ata is 22/24. I for one, believe that amplifiers of different brands and models sound different. Some are noticeably audible, some are not. Ang maganda lang sa test na ito, you get to choose the brand & models of gears.

Mods, i think we already have a topic re amplifiers sounding the same pero hindi ko talaga mahalukay sa baul ng Pdvd.  Kindly merga na lang if there is.

[tony]

you opened up a can of warms......

my take, different amps will sound differently
on different speakers using different sources,
i do not think this can be debated upon....

what the challenge poses i think is that it will be very hard
to pick a particular amp from among several amps that
are feeding the same set of speakers, using the same gain,
run well below clipping and using the same input...
it is this context that the statement, "all amplifiers sound the same"
was based upon...

i do not think that audiophiles will accept this, the testing methodology
alone is subject to heated debates by either camps...

[synchro_01]

Is he the same Richard Clark who won IASCA once upon a time? If yes then he pioneered the use of:

- Horns and horn loaded compression drivers in a car environment
- aperiodic chambers for subwoofers

The guy is a maverick audio genius.

[Nelson de Leon]

you opened up a can of warms......

my take, different amps will sound differently
on different speakers using different sources,
i do not think this can be debated upon....

what the challenge poses i think is that it will be very hard
to pick a particular amp from among several amps that
are feeding the same set of speakers, using the same gain,
run well below clipping and using the same input...

i do not think that audiophiles will accept this, the testing methodology
alone is subject to heated debates of either camps...

I kinda was expecting you to reply first.  Pero in fairness maganda naman talaga ang parameters niya di ba? Pwede ka naman mamili ng amp of your choice. 

[tony]

naku, ewan ko naman kung may papayag na mag shoot out
na hindi nakikita kung ano ang mga tinetesting....
part of the appreciation of the system is in the looking....
kung walang nakikita at tenga lang, ewan ko kung ano ang kalalabasan..
parang "the Voice" blind auditions lang.....

pero magandang malaman kung ano ang pipiliin ng madlang people,
base lang sa tunog.....

yung tipong from the start eh hindi alam ng mga judges kung anong speaker
ang nakasalang, basta, a, b, c, or d lang and then take a pick....

[Hammerheart]

They don't master. Some amps really sound better than others. There are less expensive amps that sound better than more expensive amps too. Amps have their own character, kaya minsan palit tayo ng palit ng amps hangga't di natin nakukuha yung gusto nating tunog.

[tony]

They don't master. Some amps really sound better than others. There are less expensive amps that sound better than more expensive amps too. Amps have their own character, kaya minsan palit tayo ng palit ng amps hangga't di natin nakukuha yung gusto nating tunog.

we are not even talking cost here, just the sounds...
but yes, they are those who think that low cost amps must be poor quality...
that is why they sell those amps that took them 1k to make at say 70k,
after all this is free enterprise at work here...

[tony]

I kinda was expecting you to reply first.  Pero in fairness maganda naman talaga ang parameters niya di ba? Pwede ka naman mamili ng amp of your choice. 

all the time....

[Hammerheart]

we are not even talking cost here, just the sounds...
but yes, they are those who think that low cost amps must be poor quality...
that is why they sell those amps that took them 1k to make at say 70k,
after all this is free enterprise at work here...

Yes i agree. But really, for me amplifiers doesn't sound the same.

[tony]

Yes i agree. But really, for me amplifiers doesn't sound the same.

you are exactly within your right to believe that way.....

[Arulco]

I've always wanted to try comparing high to low priced good quality amps at matched levels in a controlled ABX blind test na walang bias. Sa tingin to doon talaga magkakaalaman. Problema sobrang hassle mag setup at organize ng ganun. Kung meron sana tayong katulad ng NRC Acoustic Labs sa Canada na ginagamit ng PSB dito sa Pinas.

So far sa sighted AB un-matched level testing na madaling gawin (like when shopping for amps here), iba talaga yung sound signature ng mga amps.

[Nelson de Leon]

They don't master. Some amps really sound better than others. There are less expensive amps that sound better than more expensive amps too. Amps have their own character, kaya minsan palit tayo ng palit ng amps hangga't di natin nakukuha yung gusto nating tunog.

Yes. I think everyone who posted above including ako, and the author agrees. But the challenge actually is to prove that there is audible difference after 24 tests. The trick i think is, to put 2 amps of opposite character, one dark sounding and the other amp, a very bright amp, both having at least 100w and match it sa isang speaker having 96 db up na sensitivity. At 60-80 db spl, baka audible na ang difference. Even then, there is also the "burn out" factor ng pandinig natin sa testing. Based on my personal experience, after mga 20 tests, nag-start na ang burn out ng pandinig ko. Sometimes, after continuous exposure din sa isang sonic characteristic, it would sometimes sound neutral to you even though it's bright or dark.

[audiojunkie]

IMHO, amp's sound depends on the preamp, source and speakers ... Cables also is another factor to consider...

[tony]

IMHO, amp's sound depends on the preamp, source and speakers ... Cables also is another factor to consider...

whatever your heart desires and makes you happy....
and everyone after spending time and money can only be proud of their system...
i think the purpose of this thread is how to pick up an amp for its sound based on the challenge...

[tony]

Yes. I think everyone who posted above including ako, and the author agrees. But the challenge actually is to prove that there is audible difference after 24 tests. The trick i think is, to put 2 amps of opposite character, one dark sounding and the other amp, a very bright amp, both having at least 100w and match it sa isang speaker having 96 db up na sensitivity. At 60-80 db spl, baka audible na ang difference. Even then, there is also the "burn out" factor ng pandinig natin sa testing. Based on my personal experience, after mga 20 tests, nag-start na ang burn out ng pandinig ko. Sometimes, after continuous exposure din sa isang sonic characteristic, it would sometimes sound neutral to you even though it's bright or dark.

true, once nag judge ako sa malabon, and after a few hours
they all sounded the same to me... napagod na....
i think that playing music is for enjoyment,
if you have to strain to listen for hours, work na yan, hindi na enjoyment....

[jjohnc]

Sakin mga Sir B.S...I have been testing/owning amps for years, problem with this test is may Amplifier requirements ,...iba iba characteristics ng amps yun lang po 

[Nelson de Leon]

true, once nag judge ako sa malabon, and after a few hours
they all sounded the same to me... napagod na....
i think that playing music is for enjoyment,
if you have to strain to listen for hours, work na yan, hindi na enjoyment....

Yup. Pero we cannot also deny that there are also people who find enjoyment in critical listening because para sa kanila, only the best musical reproduction can they enjoy the music.

Kaya ako dati, whenever i fire up my system, mga 45 minutes to 1&1/2 hours of critical listening, then after that, i play mga favorite music ko.

[tony]

you are lucky to have the luxury of time...
for me naman kasi, i get more kicks out of making
something that works.....
gives me that natural high.....

[Nelson de Leon]

Another point ata ni Richard Clark is, when buying an amplifier, though quantifiable naman ang difference, is it worthwhile to buy the expensive amp? For me, yes it is definitely worthwhile kung afford ko.

Dapat ata palitan ang title. Any suggestions?

[Nelson de Leon]

you are lucky to have the luxury of time...
for me naman kasi, i get more kicks out of making
something that works.....
gives me that natural high.....

And that's your definition naman of enjoyment. 

[jjohnc]

And that's your definition naman of enjoyment. 

+++1000000000000

[tony]

Another point ata ni Richard Clark is, when buying an amplifier, though quantifiable naman ang difference, is it worthwhile to buy the expensive amp? For me, yes it is definitely worthwhile kung afford ko.

Dapat ata palitan ang title. Any suggestions?

expensive is a relative word
so for me< buy the amp that pleases your ears and your wallet in that order

[Nelson de Leon]

expensive is a relative word
so for me< buy the amp that pleases your ears and your wallet in that order

Oo nga naman. What may be expensive for me may be el cheapo for you. 

[Hammerheart]

Sa akin naman masters nelson and tony, i listen to music not my gears but, dapat pleasing to my ears and nandun yung factors na hanap ko sa system.

[jjohnc]

Oo nga naman. What may be expensive for me may be el cheapo for you. 

humble naman pre kaw pa,...btw i am all tears just auditioning these amps this moment using Transparent PoweLink PLMM2X feel na feel ko Pre,..sobrang ganda lahat ...ang hirap e explain ngayon basta kakaiba,...(Momentum, Pass and Plinius)

[dts-HD 3D]

Do all Amplifiers Sound The Same?
No, but according to the article they may sound the same using an equalizer.

[jh@meeh]

Sabi ni Alan Shaw ng Harbeth, ang amplifier sound the same in controlled environment. Sa forum ng Harbeth, minority sya sa case na ito.
willing pa syang magbigay ng harbeth speaker kung mali daw ang kanyang pinaglalaban...hehehe

[ninjababez®]

Do all Amplifiers Sound The Same?
No, but according to the article they may sound the same using an equalizer.

yan din intindi ko sa article eh. 
mas ok sakin ang double blind test (+spl meter para same db) yung may pinagpipilian ka na gears. 
yun ginawa nila masyado matrabaho, at may tendency malito yung nag test

[Nelson de Leon]

Do all Amplifiers Sound The Same?
No, but according to the article they may sound the same using an equalizer.

Binasa ko ulit.

All signal processing circuitry (e.g. bass boost, filters) must be turned off, and if the amplifier still exhibits nonlinear frequency response, an equalizer will be set by Richard Clark and inserted inline with one of the amps so that they both exhibit identical frequency response. The listener can choose which amplifier gets the equalizer.

Napakahirap nga kung nearly the same ang frequency response ng both amplifiers. Though still may pagkaka-iba pa din naman like ringing etc which makikita sa waterall plot, ang hirap talagang ma-identify.

Sabi ni Alan Shaw ng Harbeth, ang amplifier sound the same in controlled environment. Sa forum ng Harbeth, minority sya sa case na ito.
willing pa syang magbigay ng harbeth speaker kung mali daw ang kanyang pinaglalaban...hehehe

Baka siya ang may control ng environment.