Quality Gates: how we integrate automatic code checks into our processes

The principle of Quality Gates helps solve problems in the code at an early stage, before it becomes overgrown with dependencies. If there is duplication in the code, a problem with variables, or there are not enough tests, it does not "pass through the gates" and is returned to the author. As a result, the code becomes cleaner and clearer, bugs are easier to fix and pop up less often.

Three reasons to implement Quality Gates

1. The company adjusts the quality of its products. All teams have common requirements for code quality, a common vision of programming styles, product security, and product quality in general. Moreover, the company can centrally update the requirements for all projects at once.
2. Developers get rid of another piece of manual work.
3. The team receives an automatic report with the values of key metrics and an explanation of why a particular code or algorithm is considered bad.

Step-by-step implementation plan

You should start experimenting with Quality Gates with a static code analysis. This method helps teams eliminate common errors, clean up the code, get rid of the rough edges and some security gaps. We emphasize the word "some" as, in order to avoid vulnerabilities in the product, special checks are required.

For static analysis we chose SonarQube, a popular open source solution that supports a couple of dozen programming languages. An important thing for us is that there is integration with our tool for version control, TFS, so we can create ready-made pipelines with code checks already enabled.

Image

Useful conclusions of the pilot results

Having performed test runs on several dozen projects, we were able to assess the effectiveness and limitations of the tool.

1. "From a bird's eye view" we clearly saw how different teams form their own style – developers unconsciously form a common approach to writing code and, as a result, the same shortcomings appear within each team. With Quality Gates, this situation can be fixed for everyone at once.
2. The first results with hundreds of errors on a large project can easily prove demoralizing. It is important to understand that you need to pay attention not to the total number of problems, but to their diversity. The fewer types of errors there are, the faster you can fix everything.
3. For iOS products, additional tools are needed – the basic version of SonarQube does not support Swift and Objective-C. In the paid version, the verification rules for iOS in SonarQube are considerably fewer than for Java/C#. Besides that, there is no simultaneous verification of Swift and ObjC, and most of our products contain code in both languages.
4. To fully protect the product, additional means are absolutely necessary. We compared the list of vulnerabilities that SonarQube found in one of the applications with the results of a full security audit. Only a single match was found – SonarQube lacks dynamic checks, and the standard list of rules needs to be expanded.

Conclusion

Static analysis is a great starting point for implementing Quality Gates. It gives a quick effect and helps you decide where to go next.

The general quality requirements allow you to run checks based on these rules in any project, and update them centrally for all at once. The result of the checks will be a report with the values of key metrics and an explanation of why a particular code or algorithm is considered bad; by improving these values, the team will improve the quality of the product and develop their skills.