The State of Test Coverage in Rails

One of the things that we look for when examining the health of a Rails application is the percentage of test coverage the app has, using a tool like simplecov. This should only test the code your team has written and not 3rd party libraries (one reason why it’s important to keep your 3rd party dependencies separate from the rest of your code).

In the Rails Survey this year we asked a new question about how many existing Rails applications the respondents work on have decent test coverage (above 80%).

In the results, we found that 53% answered that all or most of their apps have decent test coverage. The remaining percentage is made up of people who answered some, none, or half of the apps they work on have less than decent test coverage.

The statistics here don’t look great. Although we’re happy to hear that there are at least 1047 developers out there with Rails apps with good coverage, there are a lot out there that believe the majority of their applications have less than optimal test coverage.

When we inherit projects at Planet Argon, test coverage is one of the biggest factors we look into while learning the code base. There are some great advantages to a good test suite. First and foremost, you’re more likely to know that you’re breaking things while making changes.

Another reason is it can really be a clue to future developers about what a piece of code is supposed to be doing. Lastly, it’s really important when you are making a major upgrade to the application - say the Rails or Ruby versions. Your tests will clue you in on problematic areas and dependencies that need to be resolved with your upgrade.

It’s worth noting that we’re using 80% as a good guideline, but also keeping in mind a test could touch the code and not test any relevant edge cases. It could show that the code is ‘tested’ when it’s really not asking the correct questions about if code is completing its mission.

Getting your application’s test coverage up is a slow but worthwhile process. I would encourage developers in the ‘less than decent’ range to start writing tests for each piece of code they write. Over time, this will bring up your test coverage and ideally would catch on at your organization to how code should be written.

Generally Rails developers out there should be writing tests for most or all of the code they’re writing, especially the complicated areas. I think often times we shy away from testing when things get more difficult, which can absolutely defeat the purpose. Testing should aid as a guide for you to make sure your assumptions about your complex feature is doing what you’d expect it to!

While there’s obvious room for improvement in the Rails community, overall I’m not sure that we have a great comparison with how other communities are doing. Is the state of test coverage in Python or Django any better than in our community? We can’t say for certain. At the very least, if your apps have good test coverage, this data should be encouraging that you’re on the right track and may be ahead of some of your peers.

If you answered that most of your apps do not have decent test coverage, or you’re reading this and realizing that they don’t, here are a few helpful resources for learning more about increasing your test coverage in Rails.

• Your Guide to Testing in Ruby on Rails 5
• Rails test coverage: Measuring what matters with SimpleCov
• How Much Testing is Too Much?

Click below to view more results from our 2018 survey of the Rails community.