drupal

Reflections on Drupal Quality Assurance

We recently launched ReviewDriven, a distributed quality assurance platform, which is the culmination of months of work and knowledge gained working with Drupal and its community over the past 3+ years. I look forward to feedback from the community regarding ReviewDriven, and being able to fund further development of this service and, at the same time, improving the Drupal quality assurance ecosystem.

Since ReviewDriven is a major event in my life and Drupal career it caused me to reflect on how I got to this point. From my humble beginnings in the Google Highly Open Participation Contest (GHOP), where my testing roots were planted, I received encouragement and mentoring which inspired me to continue working with open source. Previously, I had been interested in contributing to open source, but had never found a place to plug in. Since my start with Drupal I have contributed in a variety of ways to openSUSE, the Linux kernel, and KDE.

After my initial introduction to Drupal, I was received with enthusiasm and the community helped me get to Drupalcon Boston 2008. I took part in the GHOP and SimpleTest presentations. During the coding sprint after the conference, I was approached by Kieran Lal who offered to help me get to a testing sprint in Paris. Again the Drupal community along with some help from Google made it possible for me to attend. Not only was it my first time out of the United States, but I got to spend a few days working closely with some of Drupal's best. During the sprint Drupal took a major step towards realizing automated testing with the introduction of SimpleTest (or rather a fork) into Drupal core.

In the months after the sprint I pushed hard to maintain, add to, and improve the tests in core. At the time patches were committed without much thought given to the tests so keeping the tests passing was a full time job. After discussions with Kieran I ended up taking over the testing.drupal.org (now qa.drupal.org) effort. After a radical redesign and plenty of work we managed to deploy testing.drupal.org and enable integration with the issue queue once we finally got all the tests passing. With the integration also came the adoption of the current "tests always pass" ideology and requirement to include tests with patches which has revolutionized Drupal development. The system even caught some interesting drupal.org bugs.

Again thanks to community support I was able to attend Drupalcon DC and give a presentation with Kieran on the testing saga. The conference was a lot of fun in general and gave me a chance to meet all the people who I had been working with fairly regularly. Later that year I was accepted to Google Summer of Code (GSoC) for the second time and I worked part-time for Acquia as an intern over the same summer. After an exciting summer, with help from the community, I attended Drupalcon Paris 2009 where I gave another presentation on SimpleTest and the automated testing system with Kieran. After a productive Drupalcon we deployed the second version of the automated testing system and continued to improve the system.

Before the end of the year I was hired full-time by Examiner to lead their quality assurance effort. The opportunity provided me with first-hand experience on how quality assurance can fit into an enterprise Drupal development workflow. Additionally, I was able to spend a portion of my time improving the automated testing system so that we could enable partial testing of contributed projects. Examiner required a slightly different approach to testing which formed the basis for SimpleTest 7.x-2.x. Examiner sponsored the development team to attend Druaplcon San Francisco 2010 during which I gave a talk at the Core developer summit on Quality Assurance in Drupal 8, a SimpleTest presentation, and a productive BoF on testing.

In addition to the specifics mentioned I have been blessed to work with and learn from many skilled Drupal developers, and to contribute to Drupal core and contrib which has further refined my skills. My Drupal career has been a great learning experience in addition to being fun and exciting. I look forward to continued involvement with and support from the Drupal community!

Drupal 8 thoughts: configuration management and improved installation

I have been doing a lot of work related to easing the process of building a site from scratch on an individual machine and thus dabbling with configuration management and related topics. Since configuration management is one of the Drupal 8 key initiatives I figured I would share some thoughts I had on the outer fringes of the topic with more to come in the future.

Installation

One area of Drupal that has always seemed a bit odd to me has been the installation process/system. The process can play a key part in configuration management across machines and made it impossibly to implement a basic environment system during installation without hacking core. To remedy this issue I believe the installation system can be much improved, simplified, and made much more consistent with the rest of Drupal which will in tern make it easy to implement my environment system in contrib or core.

The installation system in Drupal 7 was re-factored/rewritten quite a bit and has thus been quite improved, but I think the direction of the installation system could be changed to make it much better. Currently the installer attempts to fake systems in core, like the cache, and actually duplicates a lot of code found elsewhere for module management and what not. Why not simply package a minimal database dump, similar to how the update tests used a Drupal 6 dump written in DBTNG, that can be installed to create an extremely minimal Drupal installation. At that point the installer can act like any other module and provide forms to complete the process. All modules in addition to the required modules can be installed through the standard process invoked from the modules page, but done in an automated fashion through the installer.

The reasons why this approach is beneficial are: 1) install.inc and install.core.inc could be virtually removed, 2) profiles would no longer be "hackish" during the install phase (solve the current issues related to dependency resolution and what not being the same for modules and profiles), 3) hooks like hook_system_info_alter() would work properly for profiles and modules during early installation phases, and 4) remove race conditions in general caused by maintain two sets of the same code.

Environments

In addition, my environment.module would work without hacking core. This concept is nothing new in the development world, but is something I feel would be a great candidate for Drupal 8.

<?php
/**
 * Implements hook_system_info_alter().
 */
function environment_system_info_alter(&$info$file$type) {
  if (!empty(
$info['dependencies'])) {
    
$environment environment_get();
    if (!empty(
$info['dependencies'][$environment])) {
      
$info['dependencies'] = array_merge($info['dependencies'], $info['dependencies'][$environment]);
    }
    foreach (
$info['dependencies'] as $key => $dependency) {
      if (!
is_numeric($key)) {
        unset(
$info['dependencies'][$key]);
      }
    }
  }
}

/**
 * Get the current environment.
 *
 * @return
 *   The current environment: production, staging, or development.
 */
function environment_get() {
  return 
variable_get('environment''production');
}
?>

The above code allows for the environment to be configured in the settings.php file for a site.

<?php
$conf
['environment'] = 'development';
?>

During installation a profile (or module) can perform different tasks or conditional code based on the environment. For example, generated users can have a simple password on a development machine and complex ones on production or staging machines.

<?php
function my_profile_install() {
  if (
environment_get() == 'development') {
    
// Do cool stuff that only devs get to see.
  
}
}
?>

Another cool feature that I have a use-case for on a site I am working on and seems to generally be useful is to enable modules based on the environment. Instead of having to do that in hook_install() or related it makes sense to have a way to specify that in a .info file. The above code allows for the following.

name = My profile
description = ....
version = 0.1
core = 7.x
 
dependencies[] = block
dependencies[] = dblog
 
dependencies[development][] = views_ui
dependencies[development][] = fields_ui
dependencies[development][] = devel
 
dependencies[production][] = integration_with_third_party
dependencies[staging][] = integration_with_third_party

Having this type of functionality in core would hopefully encourage better development practices and seems like a great feature to have. I have a number of scripts in combination with drush make, and the above environment utility that allow me to build out a fully functional site on a new box with a single command. I plan to cleanup the scripts, document them, and provide them in a followup post. As always I would love to hear your thoughts on this subject.

Git bisect saves the day

I have been working on a private project for quite some time using git. Yesterday, I noticed that one of the views on the site was taking around 10 seconds to generate instead of less than a second like it used to. I scanned through the recent commits, scratched my head, and messed with a bunch of stuff, but to no avail. I reverted to a commit from over a month ago just for kicks and sure enough the view rendered quickly again. So I decided it was time to learn how to use git bisect.

From git documentation:

Find by binary search the change that introduced a bug

I had read in passing the general idea behind git bisect and that you could use a script that returned pass or fail to automate the process. After reading through the man page I confirmed that a script may be used and simply needs to return exits status code 0 for pass and 1 for fail. So I figured I could write a script that manually executes the view and check to see if the amount of time required was over a certain threshold. Interestingly it seems the view executes much faster using the script than inside a normal page request. Thus the threshold used is much lower then one might expect.

I created two files since I wanted to use drush php-script and follow the documentation's recommendation by placing the scripts outside the repository. The first script is a wrapper that simply changes to the directory in which Drupal is installed and then executes the drush command.

git_bisect.sh

#!/bin/bash
cd /path/to/drupal
drush php-script ~/check_view.php

check_view.php

<?php
drupal_flush_all_caches
();

$view views_get_view('MY_CUSTOM_VIEW');
$view->set_arguments(array(3)); // Test data.
$start microtime(TRUE);
$view->execute();
$stop microtime(TRUE);

// 0: pass, 1: fail
$diff $stop $start;
$status $diff 0.3 0// Threshold.

var_dump($diff);
var_dump($status);

// If exit(0) is called drush still views it as abnormal shutdown and sets code
// to non-zero so only call when we want abnormal shutdown.
if ($status != 0) {
  exit(
1);
}
?>

My case was a bit more complex since code beyond a certain point was incompatible since I had to backup a related module to work with old revisions.

078d60ad18b73ec356436a7ea30528c95c9c4844 (bad)
3f1cfca83821a6b2d694cf228e5d8af3db20922f (good)

I ran the following inside the repository directory.

git bisect start 078d60ad18b73ec356436a7ea30528c95c9c4844 3f1cfca83821a6b2d694cf228e5d8af3db20922f --
git bisect run ~/git_bisect.sh

I ended up with the following result (-- indicates where I scrubbed data for privacy).

running /home/boombatower/git_bisect.sh
float(0.43191289901733)
int(1)
Drush command terminated abnormally due to an unrecoverable error.
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[50dcca7e9cec514c2bcc24156cd8b4622eb2cd3e] -- message --
running /home/boombatower/git_bisect.sh
float(0.53287100791931)
int(1)
Drush command terminated abnormally due to an unrecoverable error.
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[37e793d693a75a55470e6a92f5e3f30649ee2214] -- message --
running /home/boombatower/git_bisect.sh
float(0.18644404411316)
int(0)
Bisecting: 1 revision left to test after this (roughly 1 step)
[77ddb40b26fe2436cd7a15549109ffa9095d6995] -- message --
running /home/boombatower/git_bisect.sh
float(0.51487994194031)
int(1)
Drush command terminated abnormally due to an unrecoverable error.
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[08d99c98f4b9d837775db47770bc125727d93dc6] -- message --
running /home/boombatower/git_bisect.sh
float(0.1781919002533)
int(0)
77ddb40b26fe2436cd7a15549109ffa9095d6995 is the first bad commit
commit 77ddb40b26fe2436cd7a15549109ffa9095d6995
Author: --
Date:   --
 
    -- message --
 
:040000 040000 a4d3a8cb990d0eff7a7dd87c941a3f12b768feaf 10ab97d60a9b03a56862828e0f9f66cf7f4ef6b4 M      --
:100644 100644 7254a3027edfb35e10b948a9dcd994a9fbdd44a3 0a8ef37931373929328fe6458bf1f595549d265a M      --
bisect run success

Sure enough the "first bad commit" was indeed the commit that caused the performance issue. Very cool!

Drush, Drush Make, other Drupal packages, and development setup for openSUSE

I have recently added drush and drush make packages to my openSUSE repository. For more information or to report bugs on the packages please visit their respective project pages: drush and drush_make.

To install the packages you can use the one-click installers provided by the build service or manually add my repository and install the packages as shown bellow.

su
zypper ar http://download.opensuse.org/repositories/home:/boombatower/openSUSE_11.[2 or 3]/ home:boombatower
zypper in drush drush_make

Also note my existing Drupal packages: drupal-dev and drupal-vhosts, as well as the LAMP Drupal one-click pattern. The latter package (drupal-vhosts) is very useful in setting up a multi-drupal version, multi-subdomain work environment.

To use simply install and run the command to point the virtual hosts to the directory containing your Drupal code.

su
zypper in drupal-vhosts
drupal-vhost /path/to/main/software/directory

Either edit the hosts file directory or use YaST -> Network Services -> Hostnames to add an entry for every Drupal version you wish to run (package currently supports 6, 7, and 8). The relevant lines from my /etc/hosts file are as follows.

127.0.0.1       d7x.loc 
127.0.0.1       d6x.loc 

For my setup I use /home/boombatower/software for all my code with Drupal cores in drupal-7 and drupal-6 directories respectively. If you want to have subdomains for your sites just add more entries to /etc/hosts and use the respective Drupal sites directories.

Personally, I then create symbolic links to all my modules so that the code resides in the root of the software directory, but can be used by any respective site. This makes the paths to modules and what not much shorter and easier to reference from multiple specific sub-sites and what not. For example to link pathauto to the all modules directory for Drupal 7 I would execute the following.

ln -s ~/software/pathauto ~/software/drupal-7/sites/all/modules

Or from within the sites/all/modules directory as I tend to do.

ln -s ~/software/pathauto .

Also note, to enable mod_rewrite and get clean URLs to work simply go to YaST -> System -> /etc/sysconfig Editor then Network -> WWW -> Apache 2 -> APACHE_MODULES and add rewrite to the end of the line. You can do so manually of course as well.

In order for the virtual host changes and apache module addition to take effect you will need to restart apache and for the /etc/hosts changes you need to restart the network which you can do with the following commands run as root.

rcapache2 restart
rcnetwork restart

The end result of all this work is beautiful URLs like: http://d7x.loc/node/1, http://foo.d7x.loc/user, and http://d6x.loc/.

I also create a similar structure within MySQL. First, I set an easy to remember MySQL root password since there really no reason for it not to be easy to remember and it is helpful when having to enter it a lot.

mysqladmin -u root password EASY_TO_REMEMBER_PASSWORD

Next setup a drupal user in MySQL and give the user all permissions to d7x* and d6x* named databases which allows us to use a single user for all our drupal sites (much easier to remember login info) without having to update privileges all the time. I name my databases the same as virtual hosts, so for d7x.loc I would have d7x as the database name and for foo.d7x.loc I would have d7x-foo.

CREATE USER 'drupal'@'localhost' IDENTIFIED BY  'EASY_TO_REMEMBER_PASSWORD';
GRANT USAGE ON * . * TO  'drupal'@'localhost' IDENTIFIED BY  'SAME_EASY_TO_REMEMBER_PASSWORD' ;
 
GRANT ALL PRIVILEGES ON  `d7x%` . * TO  'drupal'@'localhost';
GRANT ALL PRIVILEGES ON  `d6x%` . * TO  'drupal'@'localhost';

Anytime you want to add a database for a new site simply run the following.

CREATE DATABASE  `DATABASE_NAME` ;

Enjoy your fancy development environment!

General quality assurance roadmap and request for help

As you may have seen in my previous post I have been getting back into the swing of things after some time away. I have a number of projects that I need to attend to, but as always not enough time to work on them all. Bellow you will find a list of my most important projects that need attention. If you have time available and an interest in helping out I would appreciate any extra hands. A number of the items do not have links since I have yet to spec them out completely, but I will update this post as I fill them in.

  • Parse project .info files: present module list and dependency information - needed in order for contrib testing to work 100% of the time and to move it out of beta so anyone can enable testing of their project.
  • PIFT 2.3 (drupal.org side of qa.drupal.org) - In order to make the most of the qa framework a number of improvements need to be made, mainly options and information display on drupal.org so everyone can take advantage of all the features the system already provides.
  • PIFR 2.3 (qa.drupal.org and clients) - A fair amount of testing of the current development code base, a few more bug fixes, and features are needed to round out the next release.
  • A few remaining cleanup items for SimpleTest in Drupal 7 core.
  • SimpleTest 7.x-2.x - The branch will be used for continued development during Drupal 7 life cycle and already contains a number of improvements over the base Drupal 7 testing system. It was designed so that it can coincide with the Drupal cores framework and allow for tests to use either framework (one framework per module). The switch module, which no longer works (need to look into), allows you to switch between the contrib and core versions with a single click. The switch module also could be used by other projects attempting to replace core modules and can do so with a small amount of abstraction. With the next PIFT/PIFR release I plan to support projects that wish to use the 7.x-2.x API for testing.
  • Code coverage - Reports for all tests (patches or at least core/contrib commits).
  • Refactor/Rewrite SimpleTest for Drupal 8 - I have an extremely detailed roadmap in my head (and some paper notes) and a small code base that needs to be made public.
  • PIFR 3.x - Same as above, lots of plans, but not very much publicly available yet.

Feel free to read through the issues, information, and code and let me know if you need any help getting started. Thanks!

Drupalcon SF - Quality assurance thoughts

Preface
Before I get to the actual body of this post I would like to give an explanation for my somewhat distant behavior the last month or so since Druaplcon SF and the reason for this post being so long in the making. I have been going through some life changes and issues that have required most of my attention and left me with little to time for the Drupal community. I have resolved the issues that were consuming my time and I am looking forward to picking up where I left off. Hopefully, you will see a lot more activity from me in the near future.

Summary
We had an educational discussion during the quality assurance break-out session at the Core Developer Summit. During the session we discussed the following topics.

  • JavaScript testing for Drupal
  • Site-builder testing tools
  • Drupal core performance tests
  • Ensuring its easy to start testing

I was charged with leading the discussion and taking notes. The following are my notes of the conversation that took place during the session.

  • JavaScript testing for Drupal
    • Use testswarm to crowd source JavaScript testing.
    • Either, test HEAD/branches only and do so against tagged versions or wait for a single browser result to come back and do on patches.
    • Determine list of browsers/configurations we official support and that must pass JavaScript tests.
    • Look into themes breaking JavaScript, possibly run core JavaScript tests against contributed themes.
    • Selenium seems to have limited run-ability.
  • Site-builder testing tools
    • Provide base set of tests to ensure a Drupal server is well.
    • Provides the ability to run tests against non-Drupal sites which can be useful when porting sites, working with sites that are not entirely written in Drupal, and for checking third-party integrations.
    • Maintain site-builder tools in 7.x-2.x branch of SimpleTest in contrib.
    • Possibly provide a slimmed down version of SimpleTest for use outside of Drupal.
  • Drupal core performance tests
    • Does not have to be complicated, bug simply provide a consistent benchmark.
    • Something like loading several URLs a number of times on the same server and configuration.
    • Have a scripted setup containing lots of content on server.
    • Provides another use-case for extracting the SimpleTest browser for use in core and elsewhere.
    • Simple graph of performance over time.
    • Possible initial performance suite
      • View several anonymous pages
      • Login
      • Create a node
      • Make a comment
      • View several administration screens
      • Load modules page (historically one of the slowest)
      • Logout
  • Ensuring its easy to start testing
    • Use Selenium IDE combined with simpletest_selenium to make it easy to create basic tests.
    • Submit native Selenium IDE output with bug reports to make it simple for developers to re-create bug and check if bug still exists.
    • Could also be used by experienced developers to create a good basis for a test.

Thoughts

After letting everything digest I have a number of thoughts regarding the discussion and ideas that were presented as well as a few additional pieces of information. First of all I want to share my thoughts on JavaScript testing, as I am not sure I was able to properly present this idea in person.

I look at JavaScript testing the same way I look at the current PHP based testing we do. We assume a number of things work and are tested by other organizations. As such we do not duplicate those testing efforts ourselves which is a wise decision. What I mean by that is we assume the PHP language to work as expected, the SQL language and database engine to work, and a number of other components to function. No where in our testing system do we attempt to ensure that the PHP language constructs behave as they should. We should treat jQuery as the language that it is and assume, just as with PHP, that the language functions properly in all supported environments.

The implications of the above may not be clear. What the above implies is that we do not spend time ensuring that our components and JavaScript interactions function in all browsers, environments, and operating systems. Instead we leave that job to the folks at jQuery whom already do extensive testing. Drupal should focus on ensuring that the widgets/components that core provides, such as the form API autocomplete and the ctools framework function properly. This means that we can use a tool like Crowbar or Webkit to interpret the JavaScript/jQuery and run our tests in that manor.

Attempting to test our JavaScript implementation on the infinite number of environments that exist adds a large amount of complexity to our work-flow and, as far as I can tell, very little gain. Unless someone can come up with a solid reason why we need to run our JavaScript tests on lots of environments I do not feel the idea is worth any more consideration. Oddly enough the proponents of it seemed willing to delude to waiting for one environment to return before reporting the results on drupal.org. It seems we have a lot of interest around the idea with little concern given to the implementation and cost vs benefit.

I propose we evaluate JavaScript testing frameworks with this in mind. We also need to be aware that we do not need to re-test the whole of Drupal using a JavaScript testing framework. On the contrary we need to ensure that our components and interactions work in a generic form and leave the actual testing of the final operations such as submitting a form to the already existing PHP tests. Maintaining two suites of tests that cover the same ground would be a big mistake that I hope we do not make.

Selenium vs Drupal Testing

To solidify this point further let's compare the popular JavaScript testing framework Selenium to our current PHP testing framework. After you boil down the features and purpose of the two systems you discover that they both focus on the same key ability, that being to submit forms and perform actions as a user would. The area that Selenium allows us to test that our current system does not is in regards to JavaScript interpretation, while our current system allows us to test the PHP API directly, interact with the database, and even perform unit testing. So in order to give ourselves a fully rounded test framework we simply need to fill in the small bit that our current system does not give us.

More specifically, we need to be able to test our JavaScript behaviors and components built on top of jQuery. Something more along the lines of qUnit seems appropriate since it focuses on doing just that. We will most likely develop some wrapper code for Drupal specific things, but the library provides us with a much closer starting point. There is already a patch that takes us most of the way.

Plans

The site-builder tools discussed will be maintained in the SimpleTest 7.x-2.x branch and hopefully committed to Drupal 8 once development has begun. I will continue to work on improving the tools provided to site-builders in regards to testing in the 2.x branch and will also provide back-ports of these tools to the SimpleTest 6.x-2.x branch. Since these tools are relatively new I appreciate feedback.

Automated testing 2.2 and the future

Most of you have probably seen the drupal.org front page post concerning the deployment of the automated testing system version 2.1 and the subsequent beta phase of contrib project testing. This post will provide additional details regarding the recent deployment and result of that deployment.

Contributed project testing
Without a doubt the biggest new feature provided by the 2.1 update was the addition of the ability to test contributed projects. The reason this took so long to support, was not due to lack of support from the testing master server or testing clients, but rather in the determination of module dependencies. In order to test contrib projects that have that have dependencies on other contrib projects the dependency relationships must be determined so the dependencies can be checked out by the testing clients.

The beta status will continue until the dependecy parsing component is complete and able to determine dependencies with 100% accuracy. I have documented what needs to be done in order to complete the code and will begin working on it, but any help would be appreciated.

The biggest issue that needs to be decided, in part by the community, is the way in which Drupal 7 style dependency information can be provided in Drupal 6 module .info files. Once we have a consensus that does not break compatibility with Drupal 6, core then the system needs to be updated to support Drupal 7.x style dependency restrictions in both 7.x and 6.x compatible projects.

On the positive side of things, we were able to track down issues with contrib testing after enabling reviews on a small number of contrib projects. With the help of the project maintainers reporting issues, I was able to fix the problems relatively quickly. Now that we have deployed the changes I have gone ahead and enabled testing on 31 projects (included Drupal core) that had requested to be included in the beta phase. It is great to see this much interest and enthusiasm by the Drupal community in embracing testing. The following is a list of the 31 projects that are currently taking advantage of the automated quality assurance system.

Brief look at of ATS 2.2
I wrapped a few other bug fixes along with the contrib testing bugs into a new release, which was deployed this morning. You can get a feel for what the contrib testing bugs were that we squashed in the initial testing phase below.

PIFR 6.x-2.2, 2010-01-27
------------------------
- Bugs:
   * #695350: Provide 'last' field in pifr.retrieve() and correct query.
   * Events should only be triggered when trigger modules is available.
   * Remove notices when test record is saved before client record.
   * #695278: Test list should be generated from root directory.
   * #695278: Module 'tests' directory should be searched.
   * #696044: Patches are not being applied properly to contrib projects.
   * #696194: Cannot preview client test information.
   * Do not add SimpleTest as dependency if it has already been added.

Coder review
In an effort to expand Drupal quality assurance beyond straight forward testing, I have written the initial integration for coder and coder_tough_love. Since Drupal core does not currently "pass" the review I have not enabled it on the main Drupal core tests, but have instead provided a stub test to demonstrate the functionality and provide and public set of results to work against.

It is my goal that Drupal 7/8 core and coder/coder_tough_love will be cleaned up to the point where there are no false positives from coder and Drupal itself is clean so that the review can be enabled on all Drupal core patches and commits.

I will queue the stub test for review every few days, or upon request in IRC.

Notification e-mails
Another new feature provided in the 2.1 update was the addition of a trigger/actions implementation that allows e-mails to be sent from qa.drupal.org upon a test result. The system is not designed to send e-mails to specific project maintainers and such, but instead to notify test client maintainers of a client confirmation failure (client no longer working) and the core maintainers upon a bad commit that break Drupal core.

I would like to extend this system to provide general test e-mails for project maintainers and other interested parties, but that work needs to be done on the drupal.org side of things.

Realizing the system's power
Something that may not be apparent to everyone is that the pifr/pift system can be used for more then just reviewing patches and commits. Not only can we extend the system to perform performance analysis, test coverage analysis, textual analysis, and automatic tests/security reviews, but we can use the system as a generic platform for performing distributed operations on intelligent triggers.

For example, we could use the system as a distributed platform for generating the API documentation seen on api.drupal.org. The system would automatically rebuild the documentation on commit and due to its distributed nature might even allow for the expansion of api.drupal.org for contrib projects (need to talk to infrastructure).

An advanced plugin API is provided for implementing additional functionality for the system. I have created an easy to extend Drupal specific implementation that should make it easy to build Drupal specific additions. Please contact me in IRC/email or the issue queue if you are interested in developing a plugin.

The future and beyond
I have identified items that I would like to have completed for the next installment of the system, created issues for them, and provided a summary below. I would appreciate any help from the community in completing them.

Project Issue File Review (PIFR) (qa.drupal.org & clients)

Project Issue File Test (PIFT) (drupal.org)

Project Info (drupal.org)

Drupal quality assurance requires a decision

Many of you may be familiar with my previous post Diaries of a Core Developer and that sadly nothing has changed. We had a lot of good discussion in the comments, but as far as I understand nothing has been done to improve the development workflow.

Drupal QA has and is currently having issues due to a complex problem with the core design of SimpleTest. I documented this problem some time ago, and made several attempts to refactor SimpleTest (before and after). The patches were quite large since they did a proper overhaul of the system. I was told to break them into smaller pieces so they would be easier to review, but after discovering how intertwined the code was and not receiving any other real feedback I was burned out after a week of working on the patch (and the lack of time others were willing to spend).

Just as the database layer and other large changes had large patches and lots of follow-up patches, even SimpleTest itself was introduced in a large patch, so does this refactoring. I briefly described what needs to be done and some of the benefits in the original issue. You can find more details on the refactoring in one of my later consolidation issues in which I attempted to explain the design and work through the issue.

As I explained in the issue there are some hackish ways we can work around the problem for the time being. I am happy to implement those fixes, given that I am not wasting my time. I still believe an overall refactoring is in order to fix a number of problems including proper isolation of test processes and a clean way of gather errors.

Before I commit a large amount of time, again, to refactoring the system I would like confirmation that someone will actually review the patch, those involved are fine with the changes, and that we will see this through to be committed (preferably some assurance from Dries of webchick). Of course this leads back the fact that I have no authority as a maintainer to make this crucial decision and that the core maintainers are bogged down reviewing ALL patches, instead of distributing the load to the sub-maintainers and core maintainers reviewing sub-maintainers patches and overall changes.

Pages

Subscribe to RSS - drupal