Tuesday, October 9, 2007

Second Grade Sense

No sooner does Ginsu belittle a study as "not in touch with reality", than he apologises for overlooking the fact that the Lab itself is about as accurate as a blind archer.

Even this new post is confusing in tone. As Malachi Petunia (comment # 36) says, is this post an apology for blasting the previous study? Is it a boast about how honest and transparent they are about Second Life? Look at us, we're apologising for blasting someone else blasting second life, that's how you know we're a-okay and transparent?

Leaving aside the fact that the "Second Grade Math" post is riddled with errors and out-right falsehoods (which I will get into later in this post), it's a joy to notice that after a post admitting they often have no clue what they're doing (or pretending they don't know), we see calls from across the SLscape with people praising them for their transparency.

What transparency? Where? Am I not looking somewhere? I check JIRA and the Blog and my emails, and yet every time the grid absurdly crashes, usually the only reason given is "payment issues" or just plain "issues". What issues? It was never stated in the post itself, and I don't feel like digging through the 100+ comments that are laced with 'wah wah where is my land' to 'awesome!!!11!~ keep be awesome linden lab!!' as a few examples.

As some other commentators in the recent SL blog post noted, the Lindens are confusing sheer volumes of information with actual usable data. Posting post after post on grid restarts and updates on every little sneeze that occurs between a massive world failure and the issue being resolved is just belching out tons of meaningless information.

What we really need is a clear concise post on what the issue is, how long it may take (please highball the estimate, for example, if it might take you 3 hours then say it'll take 7), and one and only one update noting that the issue is resolved by whatever you did to resolve it. That's all the information I look for when the grid starts bucking like a rodeo clown impaled on a bull's horns. It would also be pleasant if they could give a least two weeks notice, and a month preferably, on any major grid upsets or major feature insertions. VAT, for example, was kinda just dropped in there, "Today we rolled restarts, Torley's giving a new tip of the week, and oh yeah, we're gonna add VAT to the Euros."

They should cut the fat from their official statements and stop trying to drop hints about future updates that are so far down the line as to be no better than mind experiments. Havok 4 is a good one, a hint dropped about two years ago which is just released as beta the last month or so. In that time span, people whined and moaned and asked when was Havok 4 coming, is it here yet is it here yet is it here yet? Or how about grid stability? How many times has that been alluded to this year already?

The Lindens treat their SL blog like I treat mine: Personal, prone to rants and ravings, and often inaccurate and misinformed. That's par for the course for me, but I'm not running a company now, am I? The Lindens are, and their blog should at least reflect that and do away with gossip and the thousands of meaningless updates that say things along the lines of "2:30 am: the tech team had a pizza while trying to find the fatal software bug in the viewer".

And more to the point, what was with Ginsu flaming out Yankee Group for giving their interpretation of the facts they have on hand? Let's examine Ginsu's calculations that are supposed to be the final word on this matter (and which he so thoughtfully provided on the "Second Grade Math" post as an example of the true figures).

The claim made was that the average user spent 12 minutes a month in Second Life. Ginsu's closest estimate to how they came up with this number is that (his own words):

"Just this past August, users of Second Life spent over 23 million hours in Second Life....

...Maybe they werecounting cumulative total registered accounts through August, 9.3 million users. Uh, but even that is still an average of 2.5 hours.

As near as we can tell, that might be the average time that users spent logged in on the Second Life website in a month."

First, let's dispense with the notion that Yankee counted people logged into the website. That's silly. I don't know many people who log into the website, they all usually just log right into the client itself. I suppose they might have taken the traffic that the site has into consideration, but I highly doubt they used any of that data to compute time spent in Second Life. Notice that. IN Second Life. The website is not Second Life.

Second, Ginsu here is taking an average. The total amount of hours divided by the total number of (in this case) registered accounts through August '07. Averages are great for many many things. But the problem with averages is that there is no weight to them. An example is in order: I could say that the average person drives 12,000 miles per year (totally made up). All well and good. But it doesn't tell me anything except that people in the USA drive a lot. Outside of a school presentation on the 21st century, this average is useless. Taking the total miles driven in the US and dividing by the population of the country is vague at best.

What would make it useful? Break it down by the environment into city, suburb, and rural so we can form an idea of the amount driven in each. Perhaps divide it up by the type of car (perhaps sedans have 7000 miles/year while SUVs go 31000 mi/yr). The key factor here is that we're making categories into our averages, making them specific enough to be useful without making it too detailed to be useful. In our case (Second Life), we might analyse a range of mileages and see how many fall into each category, and weight the average according to that.

So, 2.5 hours per registered account is rather broad. We need a weighted average. If nine million of those 9.3 million spend 3 minutes online and the remaining 300,000 spend five hours, then obviously 2.5 hours/account is flawed. And this is where I think Yankee is coming from. Be generous and assuming 974,000 users are regulars, while 8,326,000 are one day wonders. That is a 1:8 ratio between those who play dedicated and those who day trip. Only 12.5% of the population registered so far is spending any significant time in Second Life itself. The rest is spending at most three hours in Second Life, and usually less than an hour.

Do I think the 12 minutes a month figure is a tad ridiculous? Yes. But 2.5 hours per month is almost as insane. The figure (without any real calculations on my part) is probably in the range of 60-40 minutes per month. I think that balances out the SL diehards with the massive tsunamis of one-and-dones.

In any case, Linden Lab should either polish their figures or lay off third parties publishing their own figures. If they do either, I'll die from shock.

No comments: