On rating Drupal modules ... where

Harry Slaughter recognizes the need for some sort of evaluation system for the huge number of Drupal modules available on Drupal.org. However, I feel he gets the diagnosis wrong.

As far as I can tell, the primary reason for not having a rating system for modules is fear. Module developers in particular are concerned with the fairness of ratings. They are concerned with "gaming" of ratings. They are concerned that inexperienced or "dumb" end users may unfairly give a bad review of a module simply because they don't understand how to use it. These are all reasonable concerns. But they are concerns shared by other OSS projects as well. Sure you will see "bad" reviews, giving a module the lowest possible rating along with some inane review such as "tis modules sukcs BEWARES" :) But who cares, it's just noise that will be drowned out by valid reviews. It works for other OSS projects, and it can work for Drupal.

It's not fear, it's time and energy. Configuring ratings on Drupal.org takes work -- volunteer work, so far. Regarding ratings, it's also a matter of figuring out the proper metrics for evaluating a module. Some measures that come to mind immediately include scalability, ease of use, ease of administration, extensibility (interaction with other modules), as well as aggregated metrics of the status of issues (how long they're open, how many, etc.), number of downloads....

How do you measure that with basic ratings? It's not so easy. Even the architecture and business logic of a ratings system has to be well thought out.

I feel Harry also gets the remedy wrong:

John Forsythe has released what I believe is the first site dedicated to rating and reviewing Drupal modules drupalmodules.com. No doubt this site will be a source of controversy as developers voice their concerns. But we need this resource now.

I encourage my entire audience (hi, mom!) to register at drupalmodules.com and to submit reviews for both your favorite and most hated Drupal contributions. This is a great way for non-techies to contribute to the community. The site is young, and there is naturally a shortage of ratings on the site now, but that will change as the site brings on more users.

Maybe this database will eventually make its way to Drupal.org. For now we can show our support for this type of system by helping build out the database at drupalmodules.com.

I don't think private metrics efforts will get imported into Drupal.org, for risk of skewing the results. And I feel there's some downside to splitting community dialogue into disparate sites scattered around the web. I suppose perhaps it's inevitable -- "scratch your own itch" and all -- but my preference is for Drupal.org-focused efforts.

We're having open discussions about redesigning Drupal.org on groups.drupal.org/drupal-org-redesign-analysis, including implementation of some ratings system.

Angie is co-leader of this effort, and has been putting a lot of energy into making it rock. Kieran is also a leader in this effort, and is looking for team leaders.

I've signed on, as have a number of others. While stop-gap sites that fork and fragment module discussions may have some value to some, I feel we benefit most from gathering the resources of the full community. Rather than build out a remote pantry, let's fix up our own kitchen. Drupal.org is our collective home. Redesign is a lot of work. But as Jack Aubrey would say, "Well, then, there's not a moment to lose!"

Join us!

Comments

Does supporting resources outside of Drupal.org hurt the community? Laura talks of Risk, Splitting The Community, and Disparate Sites, but is that really the case?

DrupalModules.com patterns itself after a number of other successful community sites, including DrupalSites.net, ThemeGarden.org, and DrupalDojo.com. There are dozens of other influential, unofficial Drupal resources in the community. Perhaps you run one yourself.

These sites don't take anything away from Drupal just because they are not under the Drupal.org domain name. In fact, they contribute diversity, which is a critical part of any sustainable ecosystem.

Like those sites, I believe DrupalModules.com is extending the community in exciting and important ways.

Drupal.org serves a vital function as a Support Forum and Issue Queue. It's my opinion that trying to graft a rating and review system onto that mission critical framework is not the right thing to do. It will detract attention that would be better paid to the more important aspects of Drupal.org, increase costs for the association, and require even more maintenance and moderation than the forums.

If Drupal is to have an official rating and review system, I believe it needs to stand on its own. A dedicated module rating and review site will always be more flexible and better able to serve the public than one squeezed into the gaps.

There are many important reasons to favor a dedicated solution. For example, users can benefit from the latest, cutting-edge technology, like the new Module Finder feature I just released. It finds modules as you type, updating at a rate of 10 times a second, highlighting keywords and building rating graphs in real time. The Drupal.org architecture just wasn't designed with this kind of thing in mind.

The precedent was set when Drupal shut down the Theme Garden last year. Live theme previews, site showcasing, module reviewing: These projects all needed their own room to grow. And there will be more. This is a good thing. Drupal has a healthy, growing community, filled with passionate users, and ignoring their individual efforts will hurt everyone in the long run.

Drupal.org does not have to be the only site for Drupal content. And it isn't. The great thing about the internet is that everything is just a click away. Whether this resource ends up at DrupalModules.com or Modules.Drupal.com, it's the content users are after.

I have invested considerable time and money into helping bring that content to the community. There is no advertising, and there are no membership fees on DrupalModules.com. What I have created is a labor of love, and it will remain free and open to everyone.

In closing, I would like to invite the entire Drupal Community to show their support for expanding the Drupal ecosystem instead of contracting it, and I would like to thank those of you who are already supporting my efforts.

--
John Forsythe, from DrupalModules.com

Agree with you, it's not fear.
It's lack of resources (time, energy, people...)

I think Drupal.org should remain neutral and not start a silly rating system because ultimately it will create fear of rejection in the developer community for these items which are being submitted as contrib modules under good intentions and free software terms.

I much rather have a third party site like john's where people review and rate items, and where people who care about that can go there and find those ratings.

It would be detrimental to centralize such a thing when it is not a value add for everyone. I personally don't care much for rankings and reviews because I have no way to know how knowledgeable the user is that is rating it.

Centralization would be a net negative because we are talking about free software that (unlike most firefox extensions) WILL be modified, so I rather have someone submit a crappy module with potential rather than nothing at all because they fear they'll get a bad rating.

It is simply anti-development and anti-free software philosophy to create such a vindictive system.

I don't see vindictiveness as a real danger. And focusing attention is not centralization -- we're talking about an open source community. Decentralized means forked, and is that really where we want to go?

Laura, you're a great asset to the community and your PopSci write-up was EPIC! But I gotta' say, I'm with John on this one.

I'm not so interested in the reasons why there's not an 'official' module rating system on drupal.org. In fact, I'm not even sure that drupal.org is the right platform for such a system. But I'm estatic that someone has jumped in to fill the void, and they've done so with a really high-quality effort.

I've got my own humble little drupal-related site, and I've received my share of criticism for 'operating outside the drupal.org umbrella.' But drupal as a platform and a community is bigger than drupal.org as a website. I think that more sites, more soap boxes, and more voices can only help Drupal grow. Diversity is a good thing. Alternative voices and alternative opinions are a good thing.

I'm happy to volunteer time and energy to the efforts at drupal.org, and I'd gladly take on additional responsibilities. At the same time, I feel like I'm providing a valuable service to the community through my own little site too. And my stuff pales in comparison to the asset that I think John's site will become.

Like it or not, a year from now there will be hundreds of drupal-related sites. Far from fragmenting the community, I believe that this will greatly enrich the community. The Drupal community is going to grow beyond the ability of drupal.org to contain it. As this happens, I hope that community leaders resist the temptation to try and control it.

I can see the appeal in the short term for a private site providing ratings, but for the long-term health of the greater community I would like to see that kind of energy put into making our collective home better. But perhaps I am in the minority in terms of viewing Drupal as a long-term investment of time and energy. For the person who just wants a website now and isn't thinking much about where Drupal will be next year, such concerns may seem minor by comparison.

Like it or not, a year from now there will be hundreds of drupal-related sites. Far from fragmenting the community, I believe that this will greatly enrich the community. The Drupal community is going to grow beyond the ability of drupal.org to contain it. As this happens, I hope that community leaders resist the temptation to try and control it.

Drupal is open source. It's not about control. It's about working together. Balkanized efforts have little value in the long term, it seems to me, unless the community needs to fork to remain healthy.

Hi, Laura

"Fear" was probably a poor word choice, but the basic idea remains a valid one. The assumption by many is that the average Drupal user is not qualified to rate Drupal modules. It boils down to the believe that "open ratings are useless" (http://www.archive.org/details/Drupal.org_redesign). That's what I interpret as a "fear" that allowing Joe User to rate a module which he very well may not have a deep understanding of is going to invalidate the results of a rating system.

I couldn't disagree more. I think in some ways a relatively un-educated opinion on a module has more value than that of an expert in that it better reflects the point of view of the average user. A views expert doesn't need to see a review on the views module. While a beginner would benefit greatly from seeing an array of opinions that would likely point out that while views is a phenomenal tool, it can be difficult to use. It's not going to help him to see 4 five star reviews from experts all pointing out the power of the views API. He wants to know how the Drupal admin next door feels about the module too.

I'm not saying that drupalmodules.com is *the* solution to the problem, rather a good start at a solution. Rather than taking offense to John's effort and my support of that effort, we should be congratulating John on a job well done and working with him to see if his great work can be used to help fulfill the number one request from the Drupal community (http://groups.drupal.org/node/6636).