When having to choose from a wide range of products, a simple consumer might wander off into the realm of reviews to get a better sense of what he’s getting with each choice he can make. These reviews will mostly talk about the functionality, the design, about the little niceties, and they will definitely give utterance to a handful of negative aspects as well. This can be fine for someone who doesn’t care about the innards of his newly bought hoover, but, in a market where every second customer can be just as knowledgeable as the merchant, every little detail counts, even if it’s buried underneath dozens of cool features. And this is especially true if you’re developing plugins, which, as their name implies, will always be part of a bigger system, and not any system, but a system of the same kind.
Evaluating the quality of software has never been an easy or straightforward task. But, because of their persistent and efficiency-seeking nature, software engineers have devised a few strategies for grading their own work, apart from the good old code review, i.e. telling someone to take look over your code. You can now do static code analysis on your whole project, measure the percentage of code that’s covered by at least one test, see the number of pull requests the repository has on GitHub etc. The whole idea is that there are now a lot ways to appreciate a piece of software, and what I’m going to do over the course of a few weeks will be to analyse some of the most popular plugins on WordPress. Now I’m not going to talk about the functionality, or lack thereof. Instead, I will define a set of rules that we use to assess our in-house plugins and apply them on WordPress’ most sought-after.
The reason for which we are starting this series on plugin assessment is to encourage good practices and the development of higher quality software products. We believe that no feature can be of such importance to justify bad coding or poor internal structure. With this series, we also hope to raise awareness of the tools available and give insight to anyone who might be interested in the way we do quality control.
The three plugins that we picked are All in One SEO Pack, WordPress SEO by Yoast, and Wordpress SEO Plugin by SQUIRRLY.
#Static Code Analysis
We’re using Code Climate, which is a free service for public repositories, to do all our automatic code checks. This app finds duplicated code, unused variables, and helps you keep a better structure by telling you which methods are too complex and need to be divided. Every source file gets graded from A to F, while the whole project gets a general score between 0 and 4, with 4 being the best. A nice touch is that it helps you keep track of your project’s evolution in terms of code quality, by scanning your project on every commit.
#All in One SEO Pack
Its score is 0.44 which is pretty low. The reason for this is that almost half of its source files are high in complexity and have many duplicated lines. This leads to a ratio of 6/18 files that have an F rating which ultimately drags the whole score down.
There’s one file called aioseop_module.js which triggers over 100 code smells. This is mostly because of a different coding style, and it does not influence the final score very much, as it still manages to get a D.
#WordPress SEO by Yoast
This one gets a score of 1.56. It has far less complexity and duplication than the first one, but it’s not really there yet. It still has 5 sources of chaos, each labeled with an F, the worst of which contains almost 400 lines of duplicated code, as reported by Code Climate.
#Wordpress SEO Plugin by SQUIRRLY
This plugin manages to beat the rest by a hair’s breadth with a score of 1.6. It’s far from being flawless, though, and it has much of the same issues as WordPress SEO by Yoast: 5 source files with an F and a quite a bit of code duplication in a few files.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Coding Style
We evaluate if a plugin conforms to the WordPress Core Handbook by using PHP_CodeSniffer with the WordPress Coding Standards plugin. We then count the number of errors and warnings and add them up, with the errors having a weight of 1 and the warnings a weight of 0.5. The sum gets divided by the number of PHP files in the repository, value which we call the WordPress Style Error Rate (WSER), and then it gets scaled on a scale from 0 to 500. 0 means a score of 2, while 500 or more means a score of 0. The higher the score, the better.
You might think it would be a better idea to use the number of lines when performing the calculation instead of the number of files, but we do have a reason for doing it this way. There’s a tendency of many plugins to stuff a lot of functionality into a single file. Thus, the fewer files a plugin has, the lower its final score will be, reflecting its bad design.
#All in One SEO Pack
The All in One gets a big WSER of 762.87. This is because respects only a part of the official style guide. Its final score in this department is 0.
#WordPress SEO by Yoast
This one gets a pretty good WSER of 91.22 and, while it doesn’t follow the style guide completely, it does meat most of the requirements. The final score it gets is 1.64.
#WordPress SEO Plugin by SQUIRRLY
Last but not least, Squirrly’s plugin gets a pretty average WSER of 216.66, showing that it does know quite a bit about the style guide. The plugin gets a score of 1.13.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Structure
When considering the readability and extensibility of a software product, the internal architecture of the code is very important. God classes and big files should be avoided. Another sign of bad structure can be feature envy, where a class uses another class’ methods too often. All of these are very hard to evaluate automatically which is why this section will be evaluated manually. Every plugin will receive a score between 0 and 1 with a step of 0.25 alongside with an explanation for the given mark. 0.25 will be awarded for keeping a good folder structure, another 0.25 for keeping the number of lines of code below 800 for every file, and the last 0.5 will be awarded subject to the reviewer’s appreciation of the code. A higher score means a better structure.
#All in One SEO Pack
All the source files of this plugin are placed into the root folder, creating a completele mess and proving that it doesn’t deserve the 0.25 for the folder structure. Also, with a healthy 2722 lines of code in the aioseop_class.php file, it loses the other 0.25 points as well.
Upon closer inspection, we can notice that the main class, located in the same aioseop_class.php file, has a big pile of array constant definitions, whose place is in a separate file. This class also has a lot of functionality that could have been divided into a few smaller classes which provide functionality, leaving only plugin initialization for the main class to handle. Another sign of bad structure is the presence of JavaScript code within a few methods. This code could have been placed in separate .js files. Apart from this and a few places where the code violates Demeter’s Law, the code is not that bad. Its final score for structure is 0.25.
#WordPress SEO by Yoast
This plugin has a pretty good folder structure. Although it may be beneficial to make a few new folders and categorize some source files a tad further, it’s definitely much better than what we saw previously. Unfortunately, there’s at least one file with over 800 lines of code—1947 in inc/class-wpseo-options.php, to be precise. This leaves WordPress SEO with 0.25 points up to this point.
Again, we can find some hard-coded arrays in a function of admin/class-pointers.php. In the same file, we also have some embedded JavaScript which makes the whole thing harder to read. Apart from this and a few other minor issues, the plugin has a somewhat comprehensible architecture which helps it score another 0.25, summing it up to a final score of 0.5.
#WordPress SEO Plugin by SQUIRRLY
This one has some relatively tidy folders too, and, exactly like the one that came before, it has one file that goes over the 800 lines mark. (882 lines in models/SQ_Frontend.php) This adds up to 0.25.
Taking a closer look at the code reveals an unquestionably better structure than the other two plugins. There are no huge god-classes lying around, no constant array definitions inside of methods, and the methods have a generally smaller size and complexity. With the 0.25 from before and the extra 0.5 it gets now for keeping it clean and simple, it arrives to a steady score of 0.75.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Coverage
Coverage is the percentage of lines that are executed in at least one test. Covering your whole project with testing means that whenever you try to change something and somehow break functionality that was working before, you’ll immediately know where and why. Healthy testing can also encourage faster development and deployment, because a build passing all the tests has high chances of being released. The score we will give for the coverage part is the ratio between covered lines and uncovered ones multiplied by 3. A higher score means better testing.
#All in One SEO Pack
The All in One SEO Pack has no written tests, resulting in a score of 0.
#WordPress SEO by Yoast
Yoast’s plugin manages to cover a modest 7.96% of all lines. It’s score will thus be 0.24.
#WordPress SEO Plugin by SQUIRRLY
The same story as before: plugin has 0 tests, leading to a similar value of 0 for its score.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Size
When dealing with WordPress plugins, size matters. Nobody wants to download a simple plugin whose size is huge. As we see it, there has to be a correlation between the complexity of the plugin and its size. We thus give a score of 1 to any plugin that uses a maximum of 40 bytes of data within the whole project with every line of code that is has, and a score of 0 to any plugin that exceeds the aforementioned size by a factor of 5 or more. Any other value will get an intermediary value between 0 and 1, with 1 being the best.
This is the bash script we used to compute the scores. It takes as an argument the folder containing the WordPress plugin that you want to assess.
#!/bin/bash
lines=$(find $1 -regex '.*.(php|js|css|html)' -exec wc -l {} ; | awk '{ s += $1 } END { print s }')
lines=$(($lines * 40))
size=$(du -k --exclude=.git $1 | tail -1 | awk '{ print $1 }')
size=$(($size * 1024))
echo "$size $lines" | awk '{ result = 1 - ($1 - $2) / ($2 * 4); if (result < 0) result = 0; if (result > 1) result = 1; printf "%.2fn", result }'
#All in One SEO Pack
With a seemingly small size for it’s amount of code, the All in One SEO Pack gets a score of 0.44.
#WordPress SEO by Yoast
This one gets a smaller score of 0.29, mostly because of its 11MB size.
#WordPress SEO Plugin by SQUIRRLY
WordPress SEO snatches the crown, this round, by keeping a small size, scoring 0.60.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Maturity
Having a product that’s been heavily worked on means, apart from the obvious additional functionality, better code quality and fewer bugs. The maximum score here will be 1 from every plugin which has at least 1000 commits. Anything below will get a proportional score. Higher scores suggest maturity within the project.
#All in One SEO Pack
The plugin gets a score of 0.46 for its 464 commits.
#WordPress SEO by Yoast
1846 brings this one a perfect score of 1 in the maturity test.
#WordPress SEO Plugin by SQUIRRLY
The plugin only gets a score of 0.07 as it only has 71 commits.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Liveliness
The last important aspect we want to take into consideration is the amount of activity the project had in the last few months. Software liveliness means that the project has a smaller chance of being hacked and a much higher probability of getting bug fixes. The points will be awarded in the following manner: 0.5 if there’s any commit that’s not older than a year and 1 for any commit that’s at most 2 months old. Again, we are looking for the highest score.
#All in One SEO Pack
All in One SEO Pack is actively maintained and receives a score of 1.
#WordPress SEO by Yoast
The same goes with WordPress SEO, which also scores a 1.
#WordPress SEO Plugin by SQUIRRLY
The last plugin’s latest commit is more than two months old but less than one year old, for which reason it will only get a 0.5.
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Final Scores
- All in One SEO Pack
- WordPress SEO by Yoast
- WordPress SEO Plugin by SQUIRRLY
#Overall score
#Conclusion
When taking everything into consideration, a clear winner emerges. WordPress SEO by Yoast is the best of the bunch when adopting a software engineer approach for the assessment. It has a larger code base, it’s very active and it has at least some meager testing. Coming in second is WordPress SEO Plugin by SQUIRRLY. While it has the best structure of the three, it’s much younger and not very active. The last one in our list, All in One SEO Pack, is quite far from being a threat to the first two: it has a lot of programming problems, it doesn’t follow the guideline, and it has a very poor structure.
This evaluation brings forth a few common issues which are in dire need of being solved. All of the tested plugins could benefit greatly by improving their structure and sticking closely to the object oriented programming principles. Apart from this, a free tool such as Code Climate would aid the team in keeping track of their code quality. Pausing development for a while to write tests would bring huge gains to each and every one of these projects, and it would certainly ease their jobs later on.
A great place to start evaluating your plugins in terms of quality would be on the PHP Awesome repository.
For more on WordPress hosting, plugins, quality assessment, and much more, keep an eye on the Presslabs blog.
Disclaimer: Presslabs is not affiliated in any way with the authors of the aforementioned plugins. The code on which this comparison is based dates September 25th, 2014