The State of JS Survey Is A Farce

The State of JS is a survey that has been running for a few years now, which surveys front-end developers and aims to find out what they’re using, what they love, what they’re interested in learning and what they’re not interested in knowing.

The survey sounds good in theory, it gives you insight into the state of front-end development and the various tools, libraries and frameworks people are using.

In practice, the survey is a farce. The 2018 version of the survey saw over 20,000 respondents complete the survey. While 20,000 respondents seem quite low given the number of developers out there who identify as front-end or Javascript developers, the actual issue here is the data, in this case, is biased. When you use biased data, you get a biased result.

The survey on the front-end frameworks page makes a really bold and exaggerated claim:

The front-end remains the key battleground for JavaScript. But now that the dust has cleared, it’s starting to look like only two combatants are left standing…

Based on the extremely limited dataset it might look like that, but this is an erroneous and highly inaccurate statement to make. While many use React and Vue, this does not mean people have abandoned other choices in favour of them.

In the enterprise, choices like Angular and Ember still reign supreme because they’re more verbose and verbosity is generally favoured in the enterprise because it more often than not results in a less error prone result. Angular in terms of enterprise popularity, in particular, is quite high.

And I have seen people using Npm stats as a metric for determining popularity, in the enterprise more often than not, packages are not being installed using Npm. The metrics are also skewed here once again because it doesn’t take into account that not everyone who downloads a package through Npm is building something with it (could just be curious).

In the conclusion for the front-end libraries section, the survey then doubles-down on the erroneous statement of React and Vue being the only choices:

The other story of those past couple years is the fall of Angular. While it still ranks very high in terms of raw usage, it has a fairly disappointing 41% satisfaction ratio. So while it probably isn’t going anywhere thanks to its large user base, it’s hard to see how it will ever regain its place atop the front-end throne.

Why does this matter?

It’s only a silly survey and while a little over 20,000 respondents filled it out, it’s dangerous.

The issue here is that managers, CTO’s and CEO’s are going to potentially see this survey and use it as justification to abandon other solid choices in a desperate attempt to be seen as modern and relevant.

This isn’t about being angry that React or Vue are increasing in popularity, I think Vue is great and I have worked with it before. I also worked for a company building a Netflix-like product for South American content which used React and that was also great as well.

I am sure you have seen what happens when you send developers to conferences, they come back excited and giddy wanting to change the world and use all of these new libraries they saw at the conference, this survey is the same, it’s hype fodder.

The crux of the matter here is the State of JS survey is perpetuating false claims based on seemingly biased information and in the process turning front-end development into a schoolyard popularity contest by declaring winners and losers.

The survey is a good idea, but it needs more data

StackOverflow has an annual developer survey they do and their 2018 survey got around 100,000 respondents, but the downside is they cover a broader spectrum that isn’t just front-end development like the State of JS does.

If the State of JS wants more accurate results, they need a tonne more data. And not just more data, but they need to work to eliminate the bias from their survey. And to remove the bias, it’s clear if they can get more people to complete the survey it might help. But even so, the people who run the survey seem to be users of React, so there would always be an element of bias.

Going back to the StackOverflow survey for 2018, the section for frameworks libraries and tools is rather interesting to look at. Even though it includes non-front-end choices, Node.js is the top result followed by Angular and then React.

Out of 100,000 respondents, 36.9% are using Angular, 27.8% are using React. With the following footnote:

Node.js and AngularJS continue to be the most commonly used technologies in this category, with React and .Net Core also important to many developers.

Then under the Most loved, dreaded and wanted frameworks, libraries and tools section we see angular get a 54.6% score. Worth noting that React is second here with 69.4% saying they love it.

As you can see, the two surveys produce two different results. One is focused on a specific area of development, the other is focused more broadly. I would love to see StackOverflow run a more targeted survey to see what the results are.

The State of JS survey clearly has a marketing problem and hopefully, over time the number of people who participate goes up. It’s a great idea, but at present feels like it is only being marketed to React and Vue developers, creating this confirmation bubble that React and Vue are the only choices (they’re not).

I found the gender disparity in the results (over 90% male) to be quite concerning. It really highlights that we need to do more to get women into front-end development, but it also highlights once again that the limited number of people this survey is being marketed to is a huge problem in terms of skewing the data.

One thing that is clear from this survey, is developers are choosing to use frameworks, libraries and tools based on their popularity and not whether or not their choices actually align with the requirements of the business or customers they’re developing for.

The front-end space in the last four years has honestly turned to shit, with people flinging mud and swinging their proverbial digital dicks around claiming that React is the king and that Vue is the new messiah.

There is an entire ecosystem of great frameworks and libraries to choose from, and in 2018, very little difference between them (except how you build apps). While people buy into false claims like the virtual DOM being faster than the real DOM, really regardless of what you choose; Angular, Aurelia, Ember, React or Vue, you’re going to be making a great choice for building modern web applications.

It is an exciting time to be a developer if you like picking sides and criticising people for the choices they make (especially if they’re less popular options).

The TL;DR here is to take the results of this survey with a grain of salt. It is not indicative of the industry whatsoever and is highly inaccurate, it’s interesting to see what 20,000 front-end developers think, but that’s about it.

Make decisions based on the results of this survey at your own peril.

One response to “The State of JS Survey Is A Farce”

  1. […] I published a blog title which I titled, The State of JS Survey Is A Farce in which I expressed criticism that the State of JS survey is highly inaccurate, biased and […]

Leave a Reply

Your email address will not be published. Required fields are marked *