Fleen: The Awkward Christmas Dinner Of Our Obligation To Existence » Years Later, My Prob/Stats Professor Continues To Haunt Me

Years Later, My Prob/Stats Professor Continues To Haunt Me

Okay, so Ben Gordon has written a critique of the Halfpixel Business Model (as described in How To Make Webcomics) and come to the conclusion it doesn’t work. I wish I had time to dig into this the way it deserves, but there’s no way I’m going to be able to in the near future.

So let’s be clear that this is not a formal analysis of Gordon’s entire thesis, but specifically a response based on his numbers. I’m going to talk about this using casual terminology so as to make my thoughts as accessible as possible to everybody that doesn’t know (and, rightfully, doesn’t care) about the difference between skew and kurtosis. Onwards.

Gordon looked at a sample of webcomics, and sought to estimate how much money could be made from his reading of HTMW‘s “10% Rule” (5 – 10% of your readership will open their wallets and buy things). His calculations led him to conclude that the rule is fundamentally flawed, but pointed out:

I hope someone will find fault with my analysis, because if it is sound, it is a setback for webcomics.

I’m not sure if his conclusion can be proved or disproved (we are, after all, talking about applying mathematical rules to a creative endeavour), but if his conclusion’s based solely on the numbers, I think that I’ve found the fault he was looking for, from a purely statistical standpoint. Consider the following statements from his posting:

[the business model] cannot be verified by the majority of case studies

I’ve chosen comics in a range of sizes from a list in Wikipedia which reports comics that support their creator(s). … I removed the ones that don’t belong and analyzed the rest.

The formula for estimating each comic’s profit is: … We assume the average profit per sale is $5 — typical for a t-shirt

[five calcluations of estimated webcomic profits ranging from $975 to $24,000]

First off, we need to agree on some terminology — Gordon doesn’t have “a majority of case studies”, he’s got one study with five data points. Semantics? Nope — because the number of data points is a critical element of how much we can draw reliable conclusions from the numbers. We’ll come back to that in a moment.

Secondly, Gordon’s eliminated data that “don’t belong” (for example, Achewood was eliminated because Time magazine declared it the best graphic novel of 2007 — which may have artificially inflated its numbers, I guess), meaning that we’re not looking at a random sample. We’ll come back to that, too.

Thirdly, the assumption of profit per sale is entirely arbitrary — $5, which is described as the average profit on a t-shirt (I don’t sell shirts so I can’t say, but having ordered custom shirts from the same guy many webcomickers use, I think it’s probably a bit low). But the profit per shirt doesn’t matter anyway, because it assumes that any item the creator makes will produce the same profit. Unfortunately, this doesn’t hold up.

Case in point: I have purchased a number of originals from a number of webcomickers (some of whom describe themselves as entirely self-employed by their strips and others that do not); prices have ranged from $20 to $175. Profit on even the lowest priced of them is several times Gordon’s assumption, and on the high end it utterly destroys his model. Okay, many webcomickers sell shirts, and okay, the profit on a shirt probably occupies a fairly narrow range of values, but what do we do with all the other items? You’ve got books, prints, hoodies, skateboard decks, hot sauce, and an upsell (of $5 to $10, generally) to get the item signed/sketched. That’s an incredible variation.

That price range actually points to the real problem in Gordon’s analysis — the distribution curve of those “price per original” data would form a flat line. It’s not a set of consensus values with outliers because there’s too few points — this does not allow for meaningful statistical analyses. The same situation exists with the estimated profit figures he gives: 975, 2012, 8000, 17270, 24000 … that’s only five data points. The confidence that we can derive from any analysis over such a wide range, with a distribution curve that looks like a flat line, is vanishingly small.

Statistical analysis only works if any random datum that you select to calculate can be assumed to represent many, many, similar (to the point of being essentially identical) other data that you don’t bother to include in the analysis. The key thought here is Margin of Error. You know MoE — it’s what tells you that a political race between, say, the Harbinger of the New Golden Age and the Evil Throwback to All That’s Unholy is presently split 52% to 48%, plus or minus 4.3% (and since the MoE is greater than the difference between HNGA and ETATU, we essentially don’t know who’s ahead).

Also bear in mind that the MoE is probably only to the standard level of “95% confidence”, which means that there’s a 5% chance that the real split could be even more than 4.3%. I’m going to run one simple equation to drive this home. It’s a rule of thumb that if you want to calculate the margin of error to a 95% confidence level you can do so approximately with:

0.98/√n

where n is the number of samples. In this case, n equals 5, which gives us

0.98/2.236 = 0.438 = plus or minus 43.8%

So there’s a 95% chance that the five data points we have are representative of webcomics earnings potential, with the assumption that any number we come up with could conceivably be off by as much as 43.8% from the true value. That’s not a number that we can be very confident in. Add to that the fact that statistics in general is predicated on random samples (but Gordon selected his population), and we have numbers that can’t be relied upon to any degree, even if we take the problematic $5 assumption off the table.

Heck, even recalculating for every self-reported self-supporting webcomicker isn’t going to help, because the number is still too low to provide statistical significance (honestly, we’d want a population several thousand and a sample of at least 500 to have much confidence in the numbers). It’s still an anomaly to make a living this way, and there simply are not enough data to allow for any analysis beyond the anecdotal — which is precisely what HTMW affords. This is not to say that Gordon’s question shouldn’t be asked or that his conclusions are wrong — but it is pointless to try to draw any statistical meaning from these numbers.

Speaking of “pointless”, I strongly urge that you avoid the related thread at The Daily Cartoonist, as it quickly devolved (despite Alan Gardner’s specific request to stay on the damn topic) into truly astonishing levels of dickery re: webcomickers do not have careers/incomes/lives/redeeming qualities.

It never ceases to astonish me that individuals that I have met — and who are perfectly polite and rational in person — turn into such raging exemplars of John Gabriel’s best known theorem (minus the anonymity … weird) when discussing this particular topic. I stopped reading in disgust after about 20 comments and won’t go back there. Proceed at your own risk.

The discussion at the original post is, by contrast, civil, productive, and based on logic. Gordon has been polite in responding to questions and everybody is doing their best to treat the question as an intellectual exercise designed to figure out the truth. Bravo.

At the Webcomic panel at Word on the Street in Toronto last weekend, someone asked whether or not anyone on the panel “made a living” from what they were doing.

It was clear he asked this for two reasons: 1) For the most part, those sitting at the front of the tent were extremely young (and likely to have some support from parents). 2) It was something worth discussion for a panel on a relatively new medium.

Joey Comeau took the microphone and reacted with some hostility (I thought) to the question. It seemed like a defense mechanism… which was striking, given the circumstances, and his relative establishment (and “fame”) in the industry. A Softer World’s certainly been around for more than 3-5 years, anyway.

It seems like in order to be a financial success in webcomics you have to be doing something that no one else is doing… you have to be someone like Meredith Gran, or Chris Onstad, and constantly on the lookout for new ways to promote yourself and make money. It probably doesn’t hurt if you’re lucky.

I’ve been talking with Ben for months and I can safely state that his concept of the webcomic economic model is… well, it’s a fairly gloomy one in general, even though he seems to stake a lot more on his work than I do.

Even I know a little better than to assume more than 1% conversion on a given web reader, but that’s true for almost any site.

(And as for the Daily Cartoonist… what did you expect from Ted Rall? Seriously?)

The problem with this whole this is this: the book does not present a formula. Ben is trying to test something that doesn’t exist.

Both Kellett and Straub are on the record in threads on this issue – it is NOT a formula, it was not meant to be a formula – period.

So aside from his numbers not having statistical significance (which I don’t think you need to be an expert in stats to figure out) the whole point of testing some nonexistent formula is pointless.

I applaud Ben for calling people to the roundtable to discuss how we can improve on what has been written in the book – that was something that can help the community.

But IMO the article falls off the tracks completely on a false premise.

Yes, let’s discuss ways we can be a success with our comics. Let’s not try to shove everyone into a “success funnel” that doesn’t exist. The path to success is one that starts with hard work and is unique and different for every comic. All you have to do is look at the successful webcomics to see that.

[…] The Daily Cartoonist played host to a spirited debate started by an analysis of the business model for webcomic creators described in the Halfpixel book ‘How To Make Webcomics’ by Ben Gordon. Fleen has more. […]

Seriously? I’m surprised that no one has thought to consider that the 5-10% is an annual figure, and by that standard might be achievable (divide by 12 to get your monthly supporter response — which would be just over four-tenths of a percent every month, presuming no repeat buyers and only 5% support).

This does not address how much profit is made from each purchase, but I still think it is an overlooked point.

I ballparked my gross revenue at between $3 and $5 per reader per year, depending on a) how many readers I actually have, and b) how much I actually gross this year.

Those numbers are incredibly rough, and don’t begin to take into account COGS, postage, reinvesting, or other business expenses. A more useful measure is what The Tayler Corporation pays me every two weeks. That would be $2500.

So… my pre-tax income is $60,000/yr, which is feeding a family of six. I think I’m doing that with about 40,000 readers.

Long ago I guesstimated that each reader is worth $1.00 per year to me in the form of personal income. These days I’m getting $1.50.

I don’t know if you got into hot water because you missed a point or I left it out: I only printed five case studies, I have a much bigger supply. This should have been clear from the overall description of the process. All your assumptions flowing from my desire to provide anonymity for the subjects are void. I don’t think I left it out, but it’s bizarre to think anyone would be so stupid as to use such a small study.

You also flounder on the 5% figure, as do quite a few people. The problem is I found it to be the most reasonable figure based on the research I did. However, it is a dynamic model, and you can put in any number you want. As I have said, doubling it doubles the outcome, though that hardly makes it reassuring. A late letter came in today suggesting a higher figure is warranted, but the challenge is to make the whole model plausible, and if you pump any single figure too high it becomes unsupportable. Perhaps we will have several variations routinely presented to that people will not fall down on a particular variable, mistaking it for a non-variable.

To the extent being successful is random, my sample might be called a random sample, but it is not presented as anything other than an analysis of a particular list, and one which required some adjustments.

As long as we are talking about assumptions, your assumption that I did not check data on volunteers, known data situations and random selections to see if trends deviated away from what seems like a wise starting variable for any used is not simply naive, but it violates the fundamental rules of journalism: you didn’t inquire. I did that legwork and have the transcripts of my interviews with the people close to this model. You have not read them.
You have presented an article that is perhaps the most reckless piece of coverage I have received, assuming irresponsibility x audience size = dereliction of duty.

I would think as a webcomic journalist who has been coasting on press releases for some time, you would be pleased that people are working hard to advance the medium. You’d do a fast swoon and a faint if you new who else was working on this, but because of false authorities like you, they’re smart enough to keep their heads down.

Next time, think before you speak. And don’t be such a coward as to avoid confronting me in person. Geez, you’d think I was dealing with someone in eighth grade.

Whatever you may have addressed that I don’t have time for now I’ll get to in the future.

Please explain your problem to me. You have been unpleasant for as long as I can remember, and I have always been collegial, supportive and open. If there is something personal that offends you, you’re going to have to address it with me or stop acting out.
What an interesting character you are. I always thought you were smoldering about me for some reason, and the thing you choose to target, after throwing around some statistical terms for the rubes, is this, where you could have actually been of significant help. Not to mention, you may have influenced a lot of other people not to provide data, helping keep webcomics in the dark ages. I wish you had simply talked to me, then printed your conclusions. The lost opportunity is a shame, a real shame. I increasingly wonder why I publish all my research to help other comics people when people like you make it so much more difficult. Such a small number of people, so disproportionately difficult as to ruin it for everyone.

Thanks for turning this into Newsarama. Well done. Good thing I do an online graphic novel instead of a webcomic. That way I’m not associated with either side in this argument. Not just here, but everywhere, as it seems this cannot be discussed in a civil manner. I’m expecting too much of the internet, I guess.

Gotta remember, never read the comments.

If you want to make money, the trick is to make a comic a lot of people like.

[…] and The Floating Lightbulb exchange posts (1 & 2) over critical analysis of How To Make Webcomics. AmalgametedArtists.com has […]

The halfpixel analysis seems ridiculous to me. My comic gets ~20k to ~30k uniques per month, and I sell maybe 1 or 2 items per month. That’s a profit of like $10. The half pixel model says I should be making $1000? Either I’m doing something terribly wrong, or this model is wildly optimistic by several orders of magnitude.

Here’s a screengrab of my stats if you don’t believe me: http://www.tinyghosts.com/monthlystats.jpg

Some strips are harder to monetize than others, but you’re not running any ads for one. That would clear you at least $200-300 a month.

[…] Ben Gordon critiques the business model presented in How to Make Webcomics. Then Gary Tyrell critiques the critique at […]

[…] from profits generated by a strip or activity fostered by that strip, you’ll probably greatly enjoy Gary Tyrrell digging into one critique of a profit model that’s been presented. I think in general we don’t have enough samples that last for a long enough period of time to be […]

I would think as a webcomic journalist who has been coasting on press releases for some time

I thought acting as the mouth-piece of their favorite webcomic artists was what “webcomic jornalist” meant!

So is there any truth to Gordon’s accusation that you talk about your colleagues behind their backs?

“once for insulting colleagues behind their back (in private writing sent to me)” – Gordon

You ever going to publicly explain that rubbish article you wrote about my work? I am referring to your readers, who may not have been informed of the magnitude of your reckless reporting. Be a man for once.

I just noticed someone asking if you insult people behind their backs. Get to work on your apology/retraction, because I still have the emails. You know — the ones you pretend to forget.

I can’t believe you have to be threatened to get you moving. Why don’t you put this fake blog out to pasture and get a hobby that suits you, like mixing DNA with dog ticks?

RSS feed for comments on this post.

Years Later, My Prob/Stats Professor Continues To Haunt Me

Contact Us

Archives

Search

A Good Start