![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://lemmy.ml/pictrs/image/8d1bf042-81bb-47fd-aef0-3a33b417c27b.jpeg)
I just want to jump in here as the whole thing about the tonnes of factual errors stuff…
A lot of the allegations about the accuracy of their data basically came down to arguments about the validity of statistics garnered from testing methodology; and how Labs guy claimed their methods were super good, vs other content creators claiming their methods were better.
My opinion is that all of these benchmarking content creators who base their content on rigorous “testing” are full of their own hot air.
None of them are doing sampling and testing in volume enough to be able to point to any given number and say that it is the metric for a given model of hardware. So the value reduces to this particular device performed better or worse than these other devices at this point in time doing a comparable test on our specific hardware, with our specific software installation, using the electricity supply we have at the ambient temperatures we tested at.
Its marginally useful for a product buying general comparison - in my opinion to only a limited degree; because they just aren’t testing in enough volume to get past the lottery of tolerances this gear is released under. Anyone claiming that its the performance number to expect is just full of it. Benchmarking presents like it has scientific objectivity but there are way too many variables between any given test run that none of these folks isolate before putting their videos up.
Should LTT have been better at not putting up numbers they could have known were wrong? Sure! Should they have corrected sooner & clearer when they knew they were wrong? Absolutely! Does anybody have a perfect testing methodology that produces reliable metrics - ahhhh, im not so sure. Was it a really bitchy beat up at the time from someone with an axe to grind? In my opinion, hell yes.
Exactly; the statement will have been reviewed by lawyers who would have suggested rewording it.