Can you talk about your analysis process? As a layman it feels a little hard to know where to start. What are some of the steps that you take when you’re digging into a question?
Sometimes I have a particular question. Is fanfic mostly porn? Or what are the big femslash fandoms, because I know femslash can sometimes be frustratingly hard to find. Or when The Daily Dot said, “What are the biggest ships? We have no way to know.” Whenever I have a specific question to answer, my first step is to spend a while mapping out what an answer to the question would look like. I have to write things out to think them through, so I would just open a doc and type something like, “What if I had [X] data? Would that be helpful? How would I analyze it?” So there’s my planning phase: what is the data that I’m trying to get and what will it show me?
But with a lot of the fandom stats that I do, it’s really more exploratory. For example, I started out without a more specific question than, “what does AO3 look like?” When that’s the case, I start out by gathering some data, and then I try different things and make it up as I go. Which generally involves making a lot of different graphs and staring at them, looking for interesting patterns.
A lot of times the data gathering I do is very simple. Well it’s not simple if you don’t understand AO3’s search and tagging system, which has a lot of complexity. AO3 does such a nice job trying to gather together related tags and return related results, but it’s built more for the reader than the researcher.
But anyway, I realized early on that when I was curious about a particular tag, that the sort and filter bar would give me a lot of numbers for that tag. For example, I could click on the Sherlock fandom tag, and it would give me a breakdown of all the ratings, warnings, etc. And then I could just copy them into a spreadsheet and make graphs from those. So that was one of the first things I did. And then in some cases where I didn’t just want to look at a single tag, I would use the Works Search page instead. But most of my methods just come down to doing a few AO3 searches and writing down the number of results I get for each search. E.g., how many fanworks use each of the ratings.
I usually stick my numbers into a Google Docs spreadsheet, and then I make whatever kind of graphs from it that makes sense. It helps that I have a scientific background that helps me think about how to visualize data. But Google Docs will suggest graph types to you if you highlight the data. So a lot of that stuff that I do is stuff that anybody could do, if they have the patience to work with AO3 search and spreadsheets.
My whole model of data gathering is to be as lazy as possible with the stats; to grab as little data as possible, as quickly as possible. So I’ve ended up writing a few programs to grab things easily, but I’m still going for just grabbing those big aggregate numbers, like how many things are there in this category? Some people have done some nice work to grab every single fic and its data in a fandom or a category. Then they can do means and more complicated stats that you actually need the entire whole set of data to do. I just don’t have the patience to do that.
There are lots of other fandom statisticians at this point, some of whom have very different approaches. Centrumlumina is really awesome, every year she goes through and finds the biggest ships. She actually does it in a very careful and comprehensive way, and she cares a lot about representation. She labels each ship as slash, het or femslash, and whether they involve people of color. She’s also the person who did a two thousand person AO3 user survey to understand the demographics of AO3 users, and she’s done a lot of excellent follow up analysis based on that survey data.. She’s fantastic — I believe she wants to study this in grad school but she’s already doing really cool things.
I’ve also been working with other fandom stats enthusiasts esgibter, annathecrow, and fffinnagain to try to develop essentially a set of outside APIs for AO3, so that anyone can enter any AO3 tag or ship or fandom and get back stats and graphs about that tag. It’s still a work in progress and hasn’t launched yet. But I would like it for be the case that anyone could use it and get the basics of whatever they’re curious about in fandom.
Are you currently doing stat research outside of the Sherlock fandom?
Periodically. The majority of the fandom stats I do aren’t just about Sherlock but are about fandom and fanworks more generally. And sometimes I do analyses that aren’t directly connected to fandom. For instance, I’ve recently been working on stats about gender representation and bias in television, though I’m also going to compare that eventually to representation in fandom.
I also did a recent non-fandom-specific analysis about apocalypses. I attended a Bay Area con about sci-fi and mystery and fantasy stories, called FOGcon. They had a panel on fictional apocalypses and how they changed over time. Why were they so popular now? And as I often do I was like, is it true that they are more popular now? Or are we just more aware of them for some particular reason? Have the types of apocalypses changed? People were asking these questions in the panels, and I thought, this seems like the kind of thing that Wikipedia would have a list about. And sure enough, Wikipedia has a list of apocalypses in fiction sorted by date and type of apocalypse. So I took that data and made some graphs about how fictional apocalypses change over time, and io9 covered it.
I’ve also been thinking recently about issues outside of entertainment — #BlackLivesMatter and police violence against minorities. One of my questions is: are things horrible now in a way that it has always been, or are we just more aware of it now? Is it changingly horrible? What is changing and what is an effective way to try and fix it? I’m still in the midst of trying to understand what data I could look at to try to get these answers, and I don’t know whether or not I’ll be able to find the right data to do stats.
But yes, there are a variety of things in my life outside fandom that prompt me to ask: what data will help me better understand what’s going on here? And in some cases I also ask, how might I have a positive impact here — either by shining a light on a phenomenon or by identifying factors that are correlated with positive results? But all that said, fandom is the domain that I keep coming back to — in part because of the whole community around it, the interactivity and the response of fans. I haven’t had any other thing that’s been such a consistent source of stats in my life.
What are some of the most surprising realizations that you had from looking at fandom stats?
One of them was that I was shipping one of the biggest ships that there was. That really surprised me, because I had come into fandom looking only for that and didn’t have anything for comparison. Also, Transformers can have three kinds of sex.
I love the things that I learn about individual fandoms, or individual alternate universes — things that are not necessarily common tropes in the fandoms that I’m in. I’ll often look at the most common tags for a given fandom and end up asking, what is sticky sex? What is spark sex? I’ll go ask people from those fandoms (or google it) and learn. I’ve had a lot of small discoveries that are totally wonderful and entertaining, as well as giving me views into other corners of fandom. Those are always fun.
In the big picture, I think I’ve mainly realized how much my experience of fandom is shaped by the corners I inhabit, and how many things are not true for all fans. A lot of things that are true in Sherlock fandom are not true for some other fandoms. Some fandoms don’t have any Omegaverse, for example. I’ve learned that what is common, what is popular, varies a lot on some dimensions. I’ve also been very pleasantly surprised by the diversity of fandom creations, AUs, tropes, and ideas that are out there.
Do you feel like you have a bird’s eye view on fandom now that you’ve looked at it from a stats point of view?
In one sense yes, I feel that I’m a lot more informed about what my corner of fandom is like, and what’s more broadly true of fandoms on AO3 especially. So sometimes when people have questions or beliefs about particular parts of fandom, I’m actually able to provide a broader perspective (e.g., when people assume that most fanfiction is pornographic, or that Mystrade is the second largest Sherlockian ship on all websites, or that teen AUs are the most common AUs, i can jump in and offer relevant data). But my knowledge is very biased towards AO3 and Tumblr, and to a lesser extent, Fanfiction.net, because those are the platforms I research the most. Every time I look at Wattpad, which has a much younger demographic than AO3, I find ways that my assumptions about fandom don’t hold universally.
A while ago I looked at the usage of languages besides English on AO3 and fanfiction.net and how many things were English and which things got translated into other languages. And it got me thinking about how much more fanfiction there must be out there — in other languages, other cultures — than what I usually see on AO3 and the other familiar sites. For instance, when some fans asked to translate my fics into Chinese and Russian, they wanted to post their translations on foreign-language forums that I had never heard of, but which I gather are very popular in their native languages. So I’ve become very aware of the gaps in my understanding, internationally and cross culturally. I haven’t really figured out a great approach yet to try to get a more international perspective (though some readers have sent me recommendations of archives that are popular in their cultures), but it’s one of the things I think about a lot.
It would be really interesting to find out what the trends and tropes and the kinds of stories were popular cross culturally.
Yes, it would! But even on Wattpad it’s a bit hard to to research, due to their more limited search functionality — and that’s mostly in English. From what I have been able to tell, though, that site’s fanfiction is mostly RPF — boy bands and YouTubers being two of the most popular subjects. There are some of the more familiar (to me) fandoms that I think the staff are trying to encourage to be more active there, but many of them don’t have as much of a presence. The platform also has different conventions, like tagging femslash as girlxgirl or gxg. So even just within our culture there is possible to have such different fandom experiences on different platforms, and to be very unaware of each other’s.
When it comes to other languages, it’s even harder for me to start to understand what’s common, and what varies across cultures, because I can’t even understand the websites. At that point, I start to need to rely on fans in those cultures to give me guidance and help me understand what life is like in those very different corners of fandom.