Site hosted by Angelfire.com: Build your free website today!

Bogus studies and tests


ZDNet: Foley:Microsoft's questionable research papers
The Independent Institute, says its tag line is a "nonprofit, nonpoliticized, scholarly public policy research and education organization." But, wait: there's more. According to Liebowitz, professor of managerial economics for the University of Texas at Dallas, The Independent Institute's public relations agency is Edelman PR Worldwide. One of Edelman's biggest public affairs clients is Microsoft.

Earlier this year, Edelman was discovered to be at the heart of a public-image makeover campaign designed to improve Microsoft's reputation in the face of growing federal, state and private legal battles. According to The Los Angeles Times, which originally broke the story on the Edelman plan, Edelman had proposed making available to reporters "unbiased" users and industry experts, without identifying their connections to Microsoft or the agency. Microsoft officials claimed that these kind of misrepresentations were not part of the plan.


Slashdot on Mindcraft
1. The benchmark had been paid for by Microsoft. The Mindcraft press release failed to mention this fact.

2. Mindcraft did in fact get a useful answer to its request for help tuning the Linux system. But they did not answer the request for more information, neither did they follow the tuning suggestions given Also, they forged the reply email address to conceal themselves -- the connection was made after the fact by a Usenetter who noticed that the unusual machine configuration described in the request exactly matched that of the test system in the Mindcraft results.

3. Red Hat, the Linux distributor Mindcraft says it asked for help, reports that it got one phone call from them on the installation-help line, which isn't supposed to answer post-installation questions about things like advanced server tuning. Evidently Mindcraft's efforts to get help tuning the system were feeble -- at best incompetent, at worst cynical gestures.

4. An entertainingly-written article by the head of the development team for Samba (one of the key pieces of Linux software involved in the benchmark) described how Mindcraft could have done a better job of tuning. The article revealed that one of Mindcraft's Samba tweaks had the effect of slowing their Linux down quite drastically.

Another Usenet article independently pointed out that Mindcraft had deliberately chosen a logging format that imposed a lot of overhead on Apache (the web sever used for the Linux tests).


ABC on Mindcraft
Last month, Microsoft decided to do a benchmark test of its Windows NT operating system vs. its newest, boldest competitor, the open-source Linux operating system. Microsoft contracted with Mindcraft, a Los Gatos, Calif., testing firm, to run the comparison, and when the results were announced, Windows NT was the clear winner.

Normally, that would be the end of it. But the community of hackers and developers that built Linux was furious. While Microsoft engineers spent a great deal of time "tuning" the Windows NT test computer - making sure it would run the test software at its highest efficiency level - Mindcraft installed the Red Hat Linux Operating System practically out of the box, with no tuning whatsoever.

Microsoft paid for the test, hosted the testing machines in its own labs, tuned and tweaked its machine - and won.


Salon on Mindcraft
Sure, Mindcraft -- either itself or through intermediaries -- did ask for help in an Internet-based discussion forum. But it did so under false pretences: It didn't tell its would-be helpers that it needed the information because it was comparing the performance of Windows NT and Linux at the commercial request of Microsoft. And its request for advice originated from a computer with a Microsoft address. It refused to answer any requests for more information about its test. Finally, it apparently promised to redo the test with the input of Linus Torvalds himself -- but isn't giving him the info he needs to do the job right.

The Register: MS-Gartner in tangle over Linux-knocking reports
From the look of events over the weekend Microsoft would appear to have appointed Stan Laurel as VP i/c propaganda. What else can you say when Microsoft's famous Linux Myths Web page references a stack of "Gartner" reports questioning Linux's viability, and then the reports turn out to have been written by er, Microsoft? Actually they're only maybe written by Microsoft, because although Gartner claims they are, Gartner also, er, claims they're not. Well, not exactly anyway.

99-Oct CNN: Joe Barr on Gartner
While Gartner Group has displayed only the shallowest level of understanding of Linux and the PC software market, these reports are not the result of purposeful bashing on behalf of Microsoft or anybody else. They are simply the result of analysts writing about things they don't understand.

32bitsonline parodies Gartner Group
While we do not view Gartner as a serious competitor for The Enquirer at the desktop, Gartner will not disappear from the computing landscape through 2004.

OS Opinion: The ZDNet cracker challenge
First and foremost, I have subsequently checked, and discovered that yes, PCWeek dropped the ball by omitting the crucial RPM patch (among others), and that it was intentional. For that bit of unintentional error, I truly apologize.

My misinformed assertion came from a huge flood of excuses and misleading statements from PCWeek, which tended to create a cloud of confusion. Anyone who has seen something go seriously wrong at work knows exactly how this cloud is created, and why...it usually involves something better known as the Emergency CYA maneuver. (This is not to be confused with the "FUD" phenomenon, since FUD is a deliberate tactic, planned well in advance, to meet a specific goal. Emergency CYA on the other hand is a survival tactic commonly employed by those who screw up and immediately feel the need to rescue their credibility from the proverbial toilet.)


00-Feb ZDNet: Keylabs NDS vs AD tests
...The poll ...the perpetrators
In an odd coincidence, KeyLabs Inc., a technical performance benchmarking firm based in Lindon, Utah, touts the performance of both Active Directory and Novell Inc.'s competing NDS eDirectory as superior to each other.

00-Jul spec.org: Spec.org: Second Quarter 2000 SPECweb99 Results [Linux beats Windows
Here is the interesting part: The W2K machine received a score of SPECweb99 = 1598, which means that it was able to handle a median of 1598 Conforming Simultaneous Connections. The Red Hat Tux 1.0 machine received a score of SPECweb99 = 4200, which means that is was able to handle a median of 4200 Conforming Simultaneous Connections.

FUD wars