Originally posted by cutthecrap

Interesting. I like your approach towards the subject, very concrete and analytical and a lot better than what I see of a lot of other posters who inject a hell of a lot gut feeling, conjecture and wild guessing in them based on their subjective viewpoint.

About Xfire, I'd like to add some other points that people should keep in mind too, what I read about or found out in the course of time. Xfire apparently hasn't an equal spread worldwide, meaning that its use in the east, like China and Korea, is far more rare to maybe even to the point of nonexistent than its use in the West. Xfire also seems to suffer an overall declining trend, in that the total number of Xfire users seem to be on an ongoing continuing decrease, thus also for MMO's, that should be taken into account. I think it'd be best to use other activity measuring tools, like Raptr, Steam or WoW census in combination with Xfire, then you'll see how reliable trends are or how large the error margin seems to be.

But for the rest, nice post, liked it, good points in it.

Nobody, well almost nobody, expects exact precise results from Xfire, for all intents and purposes it has been proven decently reliable tool that gives results in general ballpark area.

When all signs point same direction - you can be pretty sure its going it that direction. Xfire is just one of those signs.

Originally posted by MosesZD

 

I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.

 

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.

 

I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

Originally posted by FrodoFragins

Originally posted by MosesZD

 

I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.

 

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.

 

I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

 

Being a not completely random sample and a very accurate measure of a game's users are not mutually exclusive. 

In fact, you can have samples that aren't random... but produce very, very, very accurate results.

 

 

 

 

Originally posted by fadis

Originally posted by FrodoFragins

Originally posted by MosesZD

 

I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.

 

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.

 

I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

 

Being a not completely random sample and a very accurate measure of a game's users are not mutually exclusive. 

In fact, you can have samples that aren't random... but produce very, very, very accurate results.

No statistician would accept them as accurate on their own.  They could only end up being deemed accurate if they happen to match up with results they get from more reliable samples.  That's the point.  XFire could be +- 1% or +- 50% for all we know.  It's foolish to assume they are accurate based purely on sample size.

Originally posted by FrodoFragins

 

No statistician would accept them as accurate on their own.  They could only end up being deemed accurate if they happen to match up with results they get from more reliable samples.  That's the point.  XFire could be +- 1% or +- 50% for all we know.  It's foolish to assume they are accurate based purely on sample size.

 

Statistics masters degree here. Xfire is more than just a valid sample - it's a fantastic sample.

Most samples for popular polling agencies (those who poll for american elections etc) go with N=1000 (often with poor sample selection), cosmetics companies and advertising agencies go as low as N=100 quite often.

Television ratings (specifically Nielsen ratings) are based on less than 0.1% of the total number of TV viewers in the US, with a massive N=25,000 - one of the biggest samples in existence, and even that pales compared to Xfire.

Xfire is in the millions, you just don't get that kind of survey size normally, self-selected or not.

As for sample bias, Xfire includes players of hundreds of different games, making it even better to judge both metrics within MMOs, and metrics for MMOs relative to other game types - and self-selection is not a statistical problem, since most surveys are done that way.

Xfire is a dream sample for a statistical study.

Originally posted by Greyhooff

Originally posted by FrodoFragins

 

No statistician would accept them as accurate on their own.  They could only end up being deemed accurate if they happen to match up with results they get from more reliable samples.  That's the point.  XFire could be +- 1% or +- 50% for all we know.  It's foolish to assume they are accurate based purely on sample size.

 

Statistics masters degree here. Xfire is more than just a valid sample - it's a fantastic sample.

Most samples for popular polling agencies (those who poll for american elections etc) go with N=1000 (often with poor sample selection), cosmetics companies and advertising agencies go as low as N=100 quite often.

Television ratings (specifically Nielsen ratings) are based on less than 0.1% of the total number of TV viewers in the US, with a massive N=25,000 - one of the biggest samples in existence, and even that pales compared to Xfire.

Xfire is in the millions, you just don't get that kind of survey size normally, self-selected or not.

As for sample bias, Xfire includes players of hundreds of different games, making it even better to judge both metrics within MMOs, and metrics for MMOs relative to other game types - and self-selection is not a statistical problem, since most surveys are done that way.

Xfire is a dream sample for a statistical study.

I'm pretty sure Nielsen is careful about the socio-economic, racial and other distributions of their households.  Their goal is to try to accurately cover all TV watchers.  Whether they are really accurate is another issue, but they've got all the advertisers believing their numbers.

 

XFire takes anyone that runs the app.  How many casual gamers know what XFire is, let alone download and run it on their PC?  That alone limits their accuracy.  XFire doesn't make any claims as to the statistical accuracy of their numbers.  I'm not sure why anyone else feels the need.

 

Sample size is only the first bit in getting results with some degree of accuracy. Xfire certainly has a large enough sample size. The sample size keeps changing each week though, which is affecting the results. There is also no way to categorize or filter the respondents. It would be a great tool with a lot of the additional info that is not provided.

What you are left with is basically a weathervane. It tells you direction, but has no ability to give you a true intensity.

I'm changing the chart to only show data from sundays.

It was getting too big and modifying it every day was a bit too much. I have exams till saturday, will have the modifications done next week.

Originally posted by Greyhooff

 

Statistics masters degree here. Xfire is more than just a valid sample - it's a fantastic sample.

Most samples for popular polling agencies (those who poll for american elections etc) go with N=1000 (often with poor sample selection), cosmetics companies and advertising agencies go as low as N=100 quite often.

Television ratings (specifically Nielsen ratings) are based on less than 0.1% of the total number of TV viewers in the US, with a massive N=25,000 - one of the biggest samples in existence, and even that pales compared to Xfire.

Xfire is in the millions, you just don't get that kind of survey size normally, self-selected or not.

As for sample bias, Xfire includes players of hundreds of different games, making it even better to judge both metrics within MMOs, and metrics for MMOs relative to other game types - and self-selection is not a statistical problem, since most surveys are done that way.

Xfire is a dream sample for a statistical study.

 

Yeah.  I know that.  You know that.   Anyone who managed to pass a college stats class knows that.   Yet you see the responses.   I don't respond because I'm tired of people who know nothing throwing what are pointless and meaningless criticisms against the wall to see what sticks.  

 

For example, for a 100 million gamer population at a 99% confidence level and 1% confidence interval requires a sample size of  just 16,638 gamers.   Yet these critics will assert, with a straight face, that a sample of a MILLION isn't valid for trend-spotting...

 

Sigh.   This is why we can't have good discussions about such things.  People who have no education or experience in something that requires at least a cursory education in these things just constantly derail any such efforts.

 

This clip, I think, is one of the funniest takes on how I feel about this:   http://www.youtube.com/watch?v=uDYba0m6ztE   1:00 in really hits it...

 

 

Originally posted by Metentso

I'm changing the chart to only show data from sundays.

It was getting too big and modifying it every day was a bit too much. I have exams till saturday, will have the modifications done next week.

 

Thank you.  

Originally posted by mikahr

[...] for all intents and purposes it [XFire] has been proven decently reliable tool that gives results in general ballpark area.

[Citatian needed]

(Especially given that in this very thread it has been shown that Xfire fails to correctly predict WoW. "Proven", indeed.)

[mod edit]

Originally posted by Myria

Originally posted by mikahr

[...] for all intents and purposes it [XFire] has been proven decently reliable tool that gives results in general ballpark area.

[Citatian needed]

(Especially given that in this very thread it has been shown that Xfire fails to correctly predict WoW. "Proven", indeed.)

[mod edit]

Your comment about WoW is ill-founded. In the very short period of time that WoW numbers were looked at a significant drop was observed that corresponded to a significant event namely the launch of D3. I am not aware of how many people took up the offer but probably 'a lot' as it was a decent offer. So a sharp drop and a likely explanation.

Thank-you for pointing out that XFire in big broadbrush weathervane type strokes does seem to be a reasonable tool at least for SWTOR.

[mod edit]

Originally posted by Myria

Originally posted by mikahr

[...] for all intents and purposes it [XFire] has been proven decently reliable tool that gives results in general ballpark area.

[Citatian needed]

(Especially given that in this very thread it has been shown that Xfire fails to correctly predict WoW. "Proven", indeed.)

[mod edit]

Just go through this thread (both parts of it) and youll find a lot of examples.

I know, i know, youll have to go through countless spam posts like yours, but hey, not my fault now.

Everything you want to know has been explained already....many times.

It's impossible to meaure XFires accuracy for WOW because Blizzard doesn't release numbers for just EU and America.  They include Asia and so any losses in NA and EU can be offset by gains in Asia and South America.

What the hell else kind of proof do you need that xfire has been right all along about swtor? The game is practically dead, they've merged from 214 servers down to 20 something, and they're talking about going F2P. How is this still a discussion?

Originally posted by FrodoFragins

Originally posted by MosesZD

 

I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.

 

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.

 

I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

Sample of gamers.    Don't want a random sample.  Don't care what Facebook gamers do.  Don't care what bridge players do.  Don't care what non-gamers do.   

 

If, in the real world, I wanted to sample black men for how they felt about some issue, I wouldn't ask white women what black men thought.   I'd just ask black men

 

Once again.  It's a sample of gamers.   They're tho only ones who matter.   Their trends matter.   Not my mother.  Not my non-gaming friends.    Not the Pope.   Also, it's so big that it's unlikely to be significantly different.   We're talking a million people here when you need sample sizes of under 17,000 to be 99% accurate.  Further, it's not relevant in any case.  Even if there is bias or error in the sample, it's still insignificant as a predictive property because the error will be there from beginning to end and,therefore, is not relevant for trend analysis.

Originally posted by kanezfan

What the hell else kind of proof do you need that xfire has been right all along about swtor? The game is practically dead, they've merged from 214 servers down to 20 something, and they're talking about going F2P. How is this still a discussion?

 

You know, I've pointed that out.   I've pointed out that every single trend, including the loss of 900,000 gamers in the first quarter, has show this game to be massively dropping off the face of the planet.

 

I honestly find it weird that people are still coming in here and objecting.   Even on the BioWare forums the 'The Server Population is Dropping..." discussion thread has died as it's an accepted fact the game has gone through a hard drop in its population.

 

 

Originally posted by MosesZD

Originally posted by FrodoFragins

Originally posted by MosesZD

 

I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.

 

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.

 

I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

Sample of gamers.    Don't want a random sample.  Don't care what Facebook gamers do.  Don't care what bridge players do.  Don't care what non-gamers do.   

 

If, in the real world, I wanted to sample black men for how they felt about some issue, I wouldn't ask white women what black men thought.   I'd just ask black men

 

Once again.  It's a sample of gamers.   They're tho only ones who matter.   Their trends matter.   Not my mother.  Not my non-gaming friends.    Not the Pope.   Also, it's so big that it's unlikely to be significantly different.   We're talking a million people here when you need sample sizes of under 17,000 to be 99% accurate.  Further, it's not relevant in any case.  Even if there is bias or error in the sample, it's still insignificant as a predictive property because the error will be there from beginning to end and,therefore, is not relevant for trend analysis.

same tired old arguments to try and vindicate xfire as a tool for judging game subs, it doesnt work never has. its only a gauge of what xfire users do, they don't represent MMO players, even as a fraction, or subset, results generated are relevent only to xfire and to no other criteria, sophistry in representing statistics in a way that appears to reinforce  perception of a predisposed outcome is just that, misleading, of course it is, how could it be otherwise. But.. that SW;TOR is in trouble is also without doubt, to what scale, remains to be seen. though i have to say, in the Euro servers at least, there are now just 5 servers with a population. Those are factors you can work with, rather than some dubious statistic from xfire.

Originally posted by MosesZD

Originally posted by kanezfan

What the hell else kind of proof do you need that xfire has been right all along about swtor? The game is practically dead, they've merged from 214 servers down to 20 something, and they're talking about going F2P. How is this still a discussion?

 You know, I've pointed that out.   I've pointed out that every single trend, including the loss of 900,000 gamers in the first quarter, has show this game to be massively dropping off the face of the planet.

 

 I honestly find it weird that people are still coming in here and objecting.   Even on the BioWare forums the 'The Server Population is Dropping..." discussion thread has died as it's an accepted fact the game has gone through a hard drop in its population.

 

---

Originally posted by Phry

---

Originally posted by MosesZD

---

Originally posted by FrodoFragins

---

Originally posted by MosesZD   I don't see you explaining it...     I know a lot about XFIRE.   First, they have 21+ million installs and a million daily users.   That means, as a sample size, they are WELL IN EXCESS OF a the sample size needed for a 99% certainty with a 1% error margin on every day's sample to a games' popularity and, therefore, underlying population.   So any data point is almost certainly representative, never mind the cumulative chance of being correct that becomes so large, so fast it's not even worth discussion.   Even if all you took in college was stats for liberal arts majors, this sample-population issue should be entirely clear.  

---

Sorry to break it to you but XFire is anything but a random sample, which is needed for your 99% certainty.   I think the XFire numbers are interesting.  But XFire caters to a certain type of player/person.  XFire numbers aren't worthless, but neither are they very accurate measures of a games users.

---

Sample of gamers.    Don't want a random sample.  Don't care what Facebook gamers do.  Don't care what bridge players do.  Don't care what non-gamers do.      If, in the real world, I wanted to sample black men for how they felt about some issue, I wouldn't ask white women what black men thought.   I'd just ask black men   Once again.  It's a sample of gamers.   They're tho only ones who matter.   Their trends matter.   Not my mother.  Not my non-gaming friends.    Not the Pope.   Also, it's so big that it's unlikely to be significantly different.   We're talking a million people here when you need sample sizes of under 17,000 to be 99% accurate.  Further, it's not relevant in any case.  Even if there is bias or error in the sample, it's still insignificant as a predictive property because the error will be there from beginning to end and,therefore, is not relevant for trend analysis.

---

same tired old arguments to try and vindicate xfire as a tool for judging game subs, it doesnt work never has. its only a gauge of what xfire users do, they don't represent MMO players, even as a fraction, or subset, results generated are relevent only to xfire and to no other criteria, sophistry in representing statistics in a way that appears to reinforce  perception of a predisposed outcome is just that, misleading, of course it is, how could it be otherwise. But.. that SW;TOR is in trouble is also without doubt, to what scale, remains to be seen. though i have to say, in the Euro servers at least, there are now just 5 servers with a population. Those are factors you can work with, rather than some dubious statistic from xfire.

---

You are right, but not in response to MosesZD. When all you want to see is a trend - which only gives you the information "More" or "Less", then XFire is not complete nonsense. Once you get into trying to get sub numbers or talking about a % drop or rise over a certain period of time, then you get into nonsense numbers.

** edit **
It's also only "More" or "Less" players, which might not be related to subs. WoW often has "Less" players, but the number of subs remains constant until an expansion or update, then WoW gets "More" players.