Friday, January 18, 2008

Statistics and You!

The Lindens have released some metrics the other day of user hours, concurrency, Lindex activity, and a few other things. Let's go over the things they have posted straight off of their blog.

This is the LindeX volume in Million of USD against each quarter. They claim that quarter 2, 3, and 4 were 'slow months' due to stricter credit card protocols, shutting down casinos and charging VAT tax on Euros. Despite this, they say, the economy grew. It must have! Look at the cool red arrow pointing up!

But the problem is that the arrow is lying to you. It goes from Q3 2006 to Q4 2007, and is what we call a 'secant modulus'. A secant modulus is measuring the rate of change by drawing a direct line between two points on a curve. The problem with this kind of modulus is that depending on where you set the beginning and the end, you can get wildly different calculations of growth. I could just examine Q2, Q3, and Q4 2007 and say growth has stagnated. And so on. Anyone could shift the secant modulus any which way and say it fits their story.

Look! According to this graph, using the Lindens' methods, Philadelphia is growing faster than any other city in the world! Isn't this wonderful?

Let me offer my take. The economy was growing strong until halfway into Q1 in 2007. Then it leveled off and stagnated. It actually shrank until Q3. What does this mean for us? If this trend continues, LindeX volume will just remain the same. I agree with the Lindens that the fact this graph didn't plummet with the events they have listed is a good showing of the resiliency of the economy, but did each event carry over so long as to level off the exponential trend?

In June 6th, 2006, the Lindens opened up Second Life so that one did not need a credit card or even verify any information. Notice that before that, Q1 and Q2 of 2006 were growing linearly, meaning that it was growing at a constant speed. Once registration opened, the graph explodes into an exponential, which means that the rate of growth wasn't constant, it was twice as much as the rate before it. So week 1 of June would be +2, week 2 would be +4, week 3 would be +16, and so on. This goes until Q2 2006. Then it levels off and doesn't growth or shrink significantly.

I think you know where I am going with this. Before registration opened, growth was constant. People were coming in and spending and earning money but credit card restrictions throttled it. When that barrier was removed, you had this incredibly large influx of people. And they filtered into the world, made money of their own, added cash into SL, and boosted the economy in general. People were able to jump into SL, try it, and a few became members and started pouring money into SL through the Lindex. It was only a few, but even 10% of those millions was enough to add the massive flow that we see in this graph. So why the drop in Q2 2007? Because by then, SL's glow had faded. The focus shifted from 'try before you buy' into 'visit and tour'. People were no longer coming into SL to test the waters, they were coming in solely for checking out a cool build or as part of a tie-in (CSI being the most memorable). In Q2 we had the opening of several corporate builds which drew people who really didn't stick to SL. Since we weren't adding any new blood to the churner, the LindeX didn't grow.

They say gambling and VAT tax choked the grid, but I'm not so sure. I'd expect to see a spike in volume as gamblers and Euros try to pull out their L$ before those policies were enforced. I'd expect to see more trading going on. This isn't a graph of the US$/L$ rate, this is just the volume of these transactions. Perhaps people weren't cashing out as much as SL blogs would have you believe, or perhaps casinos went underground and Europeans got Swiss bank accounts. Despite the pounding SL took in November 2007 with downtime, rolling restarts, and constant mandatory viewer updates, the LindeX volume didn't die. Go figure.

A side note is that a real life friend of mine pointed out that this LindeX graph isn't broken down into per capita, i.e. divided by the number of people running around Second Life. I'm lazy, so I leave it up to you. However, if my intuitions are right, then doing so will destroy this 'growth', because SL has been growing population wise very very fast. That 'leveling off' could very well be a plummet. It's all in the ratios.

For User/hours and concurrency, that is a mixed bag. At which point are we seeing an end to growth or the system reaching its limit? 60,000 concurrent users may be all that the current hardware can hold. Who knows? User hours also topped off. Who knows? I sure don't.

There's much more of this. They listed a bunch of additional statistics, which can be found here. This breaks everything down into little itty bitty pieces. Don't take my word for it. Check it out for yourself. I'm not engaged or have any background in this kind of analysis, so again, I strongly encourage you to go out and do your own calculations.


Anonymous said...

I think that the most interesting thing about all of the information that the Lindens have published is the July/August 2007 numbers. Of course, those numbers show a lot about the effects of policy changes in SL.

Toward the end of July, LL banned gambling. There was no warning, and the changes went into effect, immediately. The reason you didn't see a spike in volume as gamblers and Euros try to pull out their L$ is because, in both cases, there was no warning, at all.

In August (after the gaming ban) we saw a huge drop in dollar amount of L$ sales, yet the # of sells was the second highest in history.

These spreadsheets are definitely not telling us the complete story. It's hard to tell what's missing, since it's not there, but I guarantee there's more to the story.

Of course, you're right... the LindeX didn't die, and in fact, stayed pretty stable. The fact that the value of the L$ is controlled by the company that buys and sells it, though, I try to stay away from comparing it with a "real" currency. I know for a fact that in late July, I personally "cashed out" with about L$450,000 (~$1,575 US). I also know that was pretty small time, since I didn't even own a casino.

Of course, you already know that there's a lot of other factors that push these numbers around. It's not just the money. By all logical thinking, hours/concurrency should be pretty well matched with LindenX volume. You bring up a good point about "per capita". Other things like crashes, downtime, and other technical problems are also slowing the numbers down, though.

Thank you, Anna

Anna J Tsiolkovsky said...

Very true, very good points.

I think an additional problem (besides the horrid graphs) is that in these spreadsheets, you get all kinds of information, and most people become paralyzed by it. I know I am a tad intimidated by seeing row after row of numbers. Comments on the blog, people know the graphs are wrong. But taking time to pour over the actual data (which you did and I thank you for your insights in that regard) is time intensive and people become enraged and move on and forget after a few weeks.

Regarding missing data, they do add a disclaimer on the front page:

"While we hope to provide accurate and useful information, please note that we do not guarantee the accuracy of any information; and we expressly disclaim all warranties and limit liabilities as more fully described in the Terms of Service posted at"

So in other words,

"We'll try as hard as we can, but not much more."

Anonymous said...

One small step at the time! Congratulations!!!If you are interested in balanced scorecard, KPI and metrics in business, check this web-site to learn more about Metrics and development metrics.