the site of Adam Stevens

Citizen science

First of all I’d like to say that I’m sorry for not posting in a while I got distracted playing video games and getting a lot of lol wins. On the back of a short discussion on Twitter following Alice Bell’s great article on Science TV, I’ve been prompted to flesh out my thoughts on the Zooniverse.

Zooniverse is a (growing) repository of projects that aim to involve the general populace in data processing for a diverse range of fields (from galaxies to cyclones to bats).

Let me say, right now, that I don’t want to denigrate Zooniverse. It’s a wonderful idea and has helped to link people to the science that their taxes pay for.

But (and you might have guessed there was a ‘but’ coming), let’s not pretend that it is something that it is not. It’s not, as far as I’m concerned, “citizen science”. It is data crunching, plain and simple, and I think it could be so much more. I’ve had a go at a number of the different projects and they were entertaining for about five minutes, after which I was often left thinking “yes, but what does it mean if that galaxy is elliptical and that one is spiral?” or similar.

Zooniverse often seems to be touted (along with projects like Foldit) as a new paradigm of citizen engagement with science, where non-scientists can really “make a difference”.

The exoplanet hunting part of Zooniverse got a lot of attention last year when it was featured on Stargazing Live and Dara O’Briain and Brian Cox used a lot of very excited adjectives to describe the public response and result of finding (yes, folks!) a whole new planet, which was no doubt exaggerated for televisual effect.

This apparent success clearly meant that this year’s Stargazing Live had to have a new ‘interactive’ element. The result was Planet Four, where people can annotate and categorise orbital pictures of the martian surface, in search of ‘black fans’ that can apparently tell us a lot about what’s going on. Some of my colleagues had some… interesting things to say about the project, but I hate throwing out baseless criticism, so here’s my attempt to be a little constructive.

It was hailed on this year’s Stargazing Live as another huge success (though nothing was said other than “Well done folks, you’ve looked at an awful lots of pictures), in which the UK population had provided a massive help to Mars science. Ignoring the potential issues with the validity of the science behind the whole endeavour, what Planet Four offers is a massive stack of newly categorised HiRise images. We have thousands of these already, thousands still uncategorised, and untold thousands of other images from other Mars missions. There aren’t enough Mars scientists in the world, and the ones there are don’t have enough time, to look at all these images, so yes, the people that take part in Planet Four are performing a valuable service.

These categorised, itemised images, though, are what happens before the science starts. Looking at these images, clicking, sorting, categorising, isn’t the science. The science is in the interpretation that happens afterwards. Connecting people to the science behind their Zooniverse clicks is difficult, and requires careful thought, probably from people cleverer than I, but sometimes it can be something simple.

Planet Four is apparently aiming to map “features [that] indicate wind direction and speed”. So why not ask people to say what they think the wind direction is? Add a button to the side to add an arrow (or more than one) to the picture to show the wind direction. Better yet, let them compare two images and say which one has a stronger/faster wind. Yes, ‘scientists’ might balk and say that people can’t make these kind of inferences without all the necessary information and a background in planetary science, but it would have the potential to provide interesting results and lifts the activity from mere number crunching to people making real inference from real data.

This simple change, I feel, would elevate the whole exercise, making it real science and stopping what I think is somewhat patronising slave labour. I’m sure others could come up with better, more insightful ways of doing this, for the other Zooniverse projects as well, but let’s not lose sight of the aim of really connecting people to the actual science, not just the data collection bit at the beginning.

EDIT: I’ve open this post for comments. I’m interested to hear what you think.

7 Responses

  1. True the current Zooniverse is not perfect, true it could be better.
    But your criticisms are a bith harsh because you don’t take into account the learning curve and its impact on society.
    In the small solar system body field, we have citizens involved in our work for quite a while, i will say more than 20 years. They started by simply taking a few snapshot of asteroids with their backyard telescopes, but today several of them can record accurate lightcurve, manage large mailing lists to coordinate campaignS and search for precovery in database. Some of them have even envisioned and coded their own software which are used by amateur and professional astronomers.
    This is true as well for the monitoring of Jupiter and Saturn which now relies mostly on amateur astronomers.
    If you ask these citizen scientists how they started to be involved, most of them reply that they started with a small project that they read about in an astronomy magazine. It did NOT start by attending lectures, taking classes in university etc… These citizen science projets give an opportunity to people to find an interesting research topic in science, learn about with their peers (without having to face the arrogance of some scientists) and several will choose to go to the next step. They have a valuable impact for teenagers who dont have science-litterate parents, live in remote area or are too shy to reveal to their classmates their interest in those “geeky” activities.
    If you had spent some time exploring the PlanetHunter Zooniverse you would have seen some interesting discussions about exoplanets, and several signal detected by Kepler. For instance the first variable transits detected by Kepler published last year by scientists who interpreted this signal as an exoplanet losing its material due to the proximity of its star, had been already discovered and discussed in PlanetHunter.

    In summary, we could improve Zooniverse, some projects are not similar in quality and involvement of the public. However they represent a valuable access point to science for the public. It is valuable to criticize them, but I guess the best way to make them better is to be involved.
    Thanks for taking the time to share your thoughts.

    This text need some edits. I will post a more detailed and clearer version on my blog soon.
    Clear skies
    Franck M.

    February 17, 2013 at 5:05 pm

  2. I’m a little frustrated reading this post, because it feels it’s based on a passing look at one Zooniverse project, rather than an appraisal of their overall methodology. It also assumes that people don’t get anything out of light-touch engagement, which I don’t think is the case at all.

    So firstly, on light-touch engagement, e.g. classifying or annotating an image. This can be a very satisfying, enjoyable thing for some people to do. I don’t have as much time as I’d like to devote to Zooniverse projects, but when I do log in, I really enjoy the simple processes that I’m asked to carry out, and I do feel like I’m contributing to something larger. I know that my classifications and annotations, alongside everyone else’s, form vital data that simply cannot be acquired any other way.

    There are indeed thousand and thousands of images and datasets from all sorts of projects that simply cannot be processed without the involvement of large numbers of people. This isn’t patronising slave labour, it’s the opportunity for people to engage in a genuinely important part of the scientific process, and one they’ll only do if they enjoy it.

    As for the depth of engagement, I’d suggest that you take a look at the GalaxyZoo project, going right back to the beginning. The community has been and still is very involved in the science, even to the point of driving some of it.

    Four years ago, I wrote this for the Guardian:

    And you can see there that even in the early days, people got very, very involved with the science. They weren’t and aren’t automatons.

    On the rest of the web, the 1-9-90 rule is well acknowledged. The ideas is that 90% of your users will lurk, 9% will engage only slightly, and only 1% will engage fully. Not everyone wants to or has the time to get hip-deep in a project. That 1-9-90 ratio changes from site to site, but I’d guess that Zooniverse has a much higher percentage of fully- and lightly-engaged people than many other websites, which shows just how much people enjoy it.

    Obviously different projects have potentially different needs, so it might not always be possible to involve people to the same depth every time. And some projects might go through stages, starting with simple tasks and getting more involved as the project progresses. But to write the Zooniverse off as being just a click farm is doing it and its communities a disservice.

    I doubt you’d find any of the Zooniverse team balking at the idea that the general public are capable of making really quite nuanced interpretations of what they see. The original GalaxyZoo project found that, actually, the public are very, very good at it. They require the right kind of information to allow them to assess each image, but Zooniverse projects are good at providing that information in the form of a tutorial.

    Finally, not only are some Zooniverse members getting involved in the science beyond categorisations/annotation, they are also being credited on the papers that the team produced. Their contribution is both essential and acknowledged. To me, that shows that the Zooniverse team is committed to providing as much opportunity for people to get deeply involved as they can.

    Disclosure: I’ve written about and worked with the Zooniverse team in the past.

    February 17, 2013 at 5:15 pm

  3. admin

    Thanks all for your comments.

    It seems I have missed a big part of what the Zooniverse is about, and what it sets out to achieve. I appear to have struck a chord with some and riled others, but appreciate the considered responses.

    It is wonderful to see that after taking part many go on to do amazing things, and that in itself is going to be difficult to replicate without a lot of effort.

    Let me say again: I think Zooniverse is great and has every business doing what it does, providing a link to real scientific data and (I know now) providing an avenue through which people can fall into deeper science. I can definitely appreciate for that.

    My initial post was half-formed thoughts hastily scribbled down in a tea shop, but I think the majority of it holds. My main point, I think, is about being honest about what projects like this are, and striving to think of how we can make them more than that.

    February 17, 2013 at 9:59 pm

  4. Jean Tate

    The serendipitous discovery aspects of Zooniverse projects seems to be one thing you missed, Adam. From the original (Galaxy Zoo, a.k.a. GZ), such discoveries have led to a couple of new terms (voorwerpjes, a.k.a. AGN clouds, and Green Peas), and, indirectly, a dozen or so papers. I do not know what examples there are from other projects – I’m much less familiar with them – but I’d be surprised if there are simply none (watch out for something surprising from the Andromeda Project though).

    Then there’s Lens Zoo (LZ): you do know, don’t you, that several (GZ) zooites – ordinary amateurs, not professional astronomers drawing a salary from the University of X – are core LZ team members? If gumbosea (a GZ zooite now a member of that core team) had had your attitude (“… had a go at a number of the different projects and they were entertaining for about five minutes, after which I was often left thinking …”), he would not have gone on to develop code for analyzing SDSS images, and as a result discovering many good lens candidates. Nor would Alice (one of the oldest ‘oldbie’ zooites) have decided to enrol in an MSc astrophysics program. Nor …

    Yes, the Zooniverse can certainly be improved; yes, it is very heavily top-down driven; yes, perhaps only 1% (or fewer) of the ~800,000 are really ‘into it’; … but Zooniverse projects are most certainly doing far more than just producing ‘click data’ from zooites acting as zombie automatons.

    February 18, 2013 at 5:02 pm

  5. I’m not really bothered by the idea that Zooniverse projects tend to be relatively entry-level on the analytical front. Citizen science projects generally balance research, education, and advocacy, and I don’t see the fact that different projects reach different balances of those three as a problem.

    Zooniverse may be a prominent player in citizen science, but it’s absolutely not the only one. There are hundreds of projects out there, and several make higher-level demands of their participants. As others have commented, simply sparking interest is a major accomplishment.

    February 18, 2013 at 9:41 pm

  6. I think your sentence here really hits at the main issue many people have with ‘data-mining’ heavy citizen science projects.

    “Connecting people to the science behind their Zooniverse clicks is difficult, and requires careful thought, […] but sometimes it can be something simple.”

    I would argue though, as some have already, that the real power of these projects comes from inspiring people to become active participants in science research – this must always be the first step in engagement. Ebird, one of the original CS style projects, allowed birders who might not be aware of the research to contribute and learn in the process. The extent of participation in Zooniverse projects, as you have clearly stated, is not always so great. But if you can think of those ‘clicks’ as steps toward improving their science literacy and interest, we then just need to provide them the materials to move to the next level in understanding – which is currently lacking I agree. We’ve got the breadth in CS projects, but not the depth.

    It’s a similar issue with science-inspiration videos like TED. The viewer sees (for instance) this amazing conservation issue brought to light, but there are no links or additional information provided to allow them to learn more or take action on that issue.

    February 24, 2013 at 3:59 pm

  7. Thanks for sharing this. We’ve seen an increase in the participatory aspects of “citizen science” opportunities beyond the human cognition functions available via the Zooniverse platform. There are close to 600 curated, searchable citizen science projects featured on with more added every week. Data collection and data sorting are still tops among the actions requested by the project organizers but the ability for participants to start to draw upon conclusions and mash up the data with other data sources and even help build tools and sensors to enable the process (cross pollinating DIYers/Makers with traditional citizen scientists) presents some exciting paths forward.
    There are also approaches underway to harness the power and knowledge of citizen scientists to help inform related policy matters. Literally giving a voice to the participants well beyond the tasks of data collections and sorting. See
    Thanks for the opportunity to weigh in.

    February 25, 2013 at 6:44 pm