Jean-Louis Gassée on HomePod Reviews

Jean-Louis Gassée, former Apple executive and created of BeOS, doesn’t like the way HomePod reviews have been done.

With its HomePod speaker, Apple has once again reshuffled existing genres. As an almost singular representative of the new consumer computational audio devices, HomePod’s slippery algorithms defeat quick and easy reviews.

He criticizes most tests, as not being scientific, and highlights David Pogue’s “blind” test of four speakers with give people.

He discusses the “computational audio” used by the HomePod, and notes:

This is where we find a new type of difficulty when evaluating this new breed of smart speakers, and why we must be kind to the early HomePod reviewers: The technical complexity and environmental subjectivity leads to contradictory statements and inconsistent results.

I think he’s missing the point. When one reviews something subjectively, the goal is to find out how it sounds to each listener. You can double-blind all you want, but that’s not how people perceive music. There is certainly room for measurements – but not when they’re done wrong – but the true test of a device like this, especially one where the surroundings change the sound, is to have listeners judge it.

Yes, when you have four speakers, and their volume isn’t perfectly balanced, that is an issue, but the main takeaway in Pogue’s review was that a) no one liked the Amazon Echo, because it’s a cheap, tinny speaker, and b) the HomePod may not be the best. It is notably very bass heavy, which means that some music will sound good, and some won’t sound very good at all. Compared to the other speakers – which have a flatter sound signature – the HomePod makes the mistake of imposing a tone on all the music it plays, and not allowing for individual user adjustments. (I’m not sure if all the better speakers that David Pogue tested allow for EQ tweaking; the Amazon Echo probably doesn’t, because it’s not that much of a speaker; the Sonos One definitely does, via the Sonos app.)

Finally, I find it almost risible to see the graphic that Mr. Gassée has included in has article as proof that the test was rigged. He points out that a louder speaker generally sounds better – which is well known – so the people who preferred one speaker must have been closer to that speaker.

Gassee

This is a clear example of bias. Persons one and five were certainly closer to the speakers on the end, but persons two, three, and four were closer to speakers B and C. But none of them like it. Mr Gassée’s lines are ludicrous; he’s talking about the distance, yet ignoring the fact that, for example, person three is notably further from speakers A and D, and much closer to speakers B and C.

This is a glaring error in logic, and it’s a shame to see it included in an article that gets so technical about computational audio, electro-acoustc music at IRCAM, and so one.

13 thoughts on “Jean-Louis Gassée on HomePod Reviews

  1. But the thing is that the HomePod is a chameleon. It changes to fit its surroundings.

    As Peter Aczel (of Audio Critic) says on the determinants of sound quality:

    “It would probably be more accurate to say that the speakers, the room, and the placement of the speakers within the room constitute a single system second in importance only to the program material.”

    HomePod utterly changes that equation since it actively listens to the sound returning to it and adjusts the sound it is outputting to try to achieve the most pleasing sound possible.

    This is precisely why HomePod performs poorly in a room with extensive sound dampening material. (Which Pogue’s room may have approximated). It can’t hear itself and therefore can’t do the magic DSP which is its fundamental feature.

    With HomePod, the sound the listener hears depends on HomePod’s algorithms. It seems clear that HomePod modifies its output extensively based on the acoustic map of the room. (Ie very different if placed in a corner v. an open shelf v. several feet from any walls.) Whether any or all those modifications are pleasing to any one listener is really the question.

    Also note that HomePod tries to make any room sound good. Automatically compensating for particular frequency ranges that may be absorbed or reflected. Etc.

    So it really turns Aczel’s rule on its head. HomePod tries to take the room out of the sound quality equation. And achieve pleasing, room-filling sound from a tiny single-point-source enclosure. This is a kick-ass innovation that Apple has gotten almost no credit for, so far.

    And presumably HomePod’s algorithms can be tweaked in a future software update. To provide pleasing sound in even in an even greater range of suboptimal rooms.

    I think it would be really interesting to read comparisons of HomePod v. the others in a variety of conditions. Rooms with very different sound characteristics including a bunch that break all the rules of an audiophile listening space. Various placements within these rooms. Maybe even an all-tile subway bathroom! My suspicion is that HomePod can do far better at making rooms with poor-to-average acoustics sound damn good.

    But then, nobody seems to want to find out what HomePod’s strength’s are. All are too busy complaining that Siri can’t answer bar-trivia.

    • Oh, my. No.

      “This is precisely why HomePod performs poorly in a room with extensive sound dampening material. (Which Pogue’s room may have approximated). It can’t hear itself and therefore can’t do the magic DSP which is its fundamental feature.”

      The whole point of the HomePod is that it should sound the same – more or less – in any room.

      Given how thick Pogue’s curtain was, I don’t think it made much of a difference; it wasn’t much thicker than the cloth that’s on the HomePod. At its worst, it would muffle the treble a bit.

      But the broader point is that many people have reviewed the HomePod – including myself – comparing it to other speakers, or to stereos. It’s pretty unanimous, from those who care about sound, that it is overly bassy. That’s a sound signature that Apple is unlikely to change; it is likely close to the sound of Beats headphones.

      The main problem I see with the HomePod is the lack of EQ; this could be changed in a software update. But also the fact that it tries to tweak the sound without knowing where the listener is. The sound does change slightly as you move around a room, but I don’t think it can ever be as good as a pair of speakers pointing toward a listener.

      • I suspect Pogue’s curtain may have had a drastic effect. First, I believe HomePod listens and adapts all the time[1]. With the curtain introduced, it may have decided that it was suddenly in a much smaller space with teble-deadening material all along one side. And extensively compensated for that. Using eye masks or hiding the speakers in shadows would be a simple way to eliminate this potential distortion.

        BTW, didn’t your review suggest the bass was too heavy in classical music and certain jazz? Algorithm tweaks could well eliminate those effects. Although I doubt Apple is likely to optimize for those genres.

        [1] All reviewers have said HomePod is excellent at hearing “Hey Siri” even over max volume sound. It knows what sound it has put out and expects to hear back. It uses DSP to cancel out everything but your hail. It has to have an up-to-date acoustic map to do that effectively. And seems to do it extremely effectively. (Which is another HomePod strength that has gotten very little mention.)

        Craig

        • Many people have suggested that the reviewers should have been blindfolded. But then they wouldn’t have been able to take notes. If the curtain did dampen the sound, then it would increase the treble, most likely cancelling out the effect of the curtain.

          The bass is too heavy for lots of music, but it’s especially muddy with classical music and jazz, though not all.

          And, yes, the Hey Siri thing is truly impressive. If you don’t have another device nearby that has Hey Siri turned on…

          • But HomePod likely optimized for its side of the curtain; not the side the listeners were on.

            AIUI, if you raise either your Apple Watch or iPhone, it will take your Hey Siri request. If not, HomePod will. Not true?

            (HomePod isn’t available in Canada yet. I may yet spend the extra money to have it transhipped.)

            • No, my watch caught my Hey Siris several times when my arms were just in my lap. And other people have had the same problem with the iPhone trapping their requests.

  2. Pardon my ignorance, but what, exactly is the purpose of the HomePod? It seems one of its functions is to deliver some sort of “pleasing” sound to listeners whose perception of accurate/neutral/natural reproduction becomes ever more polluted with the rise in “convenience”. The “New York Times” just published an essay on the social, intellectual, and emotional destructiveness of having everything done “conveniently” for us.

    https://www.nytimes.com/2018/02/16/opinion/sunday/tyranny-convenience.html?emc=edit_th_180218&nl=todaysheadlines&nlid=510164720218

    My desktop speakers are the Monsoon 2000 system, whose planar-magnetic satellites (based on Bruce Thigpen’s design) are pretty darn good — good enough to give a fairly accurate representation of a classical orchestra. The separate woofer has controls that allow adjusting the bass for an exciting “Whoompf!”, without getting muddy. The HomePod might have worthwhile uses, but for this serious/critical listener, listening to music isn’t one of them.

    • It’s certainly not for you. :-)

      It is, however, for people who just want music to sound okay. It’s much better than most Bluetooth speakers, and it is quite loud and can fill a room. The fact that it is truly omnidirectional is interesting, as most standalone speakers are not.

  3. The graphic of Mr. Gassée about the distance between speaker and listener is very misleading.
    As I see in the video the distance between listeners and speaker is far greater than the distance between speaker A and D (or listener 1 and 5).
    So the difference of the distances from listener to speaker A, B, C or D is nearly the same and ultimately irrelevant.

  4. If you want to get a good understanding of HOW Consumer Reports came out with such a negative assessment of the HomePod, in contradiction to the almost unanimous superlative reviews and testing results done by audio engineers and technology professionals, read this excerpt from CR’s “review”:

    “Consumer Reports evaluates sound quality for speakers, smart or otherwise, in a dedicated listening room in which our experienced testers compare each model with high-quality reference speakers. Each test unit that allows for user controls is tuned for optimum sound quality—we want the speakers to sound their best.”

    Then, take a look at the photo of CR’s testing “rig”. It has all speakers crammed together on a multi-shelf stand with many speakers in front of, behind, above, and below each other. It is a very cumbersome setup, in a very unnatural test environment (unlike ANY room and speaker placement that a real person would ever experience).

    In the photo of the Consumer Reports speaker setup in the listening room, notice that the HomePod speaker has no space or flat wall surfaces on the left and right sides, and on the back. The HomePod uses that free space around it, and the rear and side walls to reflect the left and right channels, using audio beamforming, while the center channel (audio that is centered in the sound space) is beamed forward.

    The HomePod is unlike other speakers that just have their internal speakers facing forward, and that DO NOT rely on reflected audio beamforming.

    In Consumer Report’s setup, instead of clear, reflected left and right audio channels, the sound would have been muffled. It is unknown whether this debilitating positioning of the HomePod was done purposely, or if it was done due to a lack of understanding by the CR testers of the HomePod’s unique audio technologies.

    In other words, it was a very poorly setup test environment, and the review consisted of “listeners” giving their own subjective feelings about which speaker they liked best. Aside from any unstated motives or biases of the “listeners” at Consumer Reports, there was no methodical or scientific testing done of the speakers, in various “natural” room environments. In contrast, this type of analytical and realistic testing WAS done by other reputable audio engineers and technology professionals.

  5. Kirk as I have stated elsewhere I see this Home Pod as being a FIRST STEP to having good (yes I said good) sound for music and possibly video soundtracks sans separate amps/digital/analogue sources while at the same time attempting to compensate for different room configurations. It will never satisfy those who fancy themselves audiophiles . Nor will it (most likely) ever satisfy those who do not like Apple products. But given what I BELIEVE they are trying to do it will provide (with further improvements/modifications and size offerings) excellent sound with minimal footprint and ease of use for those who are in the Apple “ecosystem”. What say you Kirk

Leave a Comment