drupal

Error message

  • Warning: Illegal string offset 'field' in DatabaseCondition->__clone() (line 1901 of /home/boombato/public_html/drupal7/includes/database/query.inc).
  • Warning: Illegal string offset 'field' in DatabaseCondition->__clone() (line 1901 of /home/boombato/public_html/drupal7/includes/database/query.inc).

Fresh SimpleTest backport - 2.9

Considering:

  • the last official backport was April 23 2009
  • a number of very cool features have been added in core in addition to incremental changes
  • people have been asking
  • Drupal 7 is in "code slush"

it seems appropriate to perform another backport. I finished the bulk of the backport during Drupalcon Paris and have been tweaking and fixing bugs from feedback since then.

In order to ensure that all the new features from Drupal 7 were available it was necessary to create a patch against Drupal 6 core which needs to be applied before installation, as described in INSTALL.txt.

Please update and report any issues in the queue and have fun with the new features and proper error reporting!

test run

Drupal 7: debug() and SimpleTest->verbose()

Recently, I have made two major improvements to debugging in Drupal 7, the addition of a general debug function and a verbose mode for SimpleTest. The two additions make it much easier to debug problems quickly through the use of a consistent method. Take a look at what chx said via twitter:

Writing #drupal code? Check the new function debug(). Writing #drupal tests? Check $this->verbose(). And debug() works too. AWESOME!

General debug function
The general debug function can be used at any point after Drupal is bootstrapped, although the limitation may be removed in the future. The function provides a very simple wrapper to dump data through the use of var_export() and print_r(). When used normally it will display data based on the "Logging and errors" settings provided in Drupal 7 core. If using a dev version the debug information will be displayed using drupal_set_message() as shown in the screenshot below.

debug normal

The exciting part about the new debug() function is that it also works during testing. The debug() function can be placed inside the test itself or in any other part of Drupal and it will be picked up and displayed in the test results as shown below.

debug test

SimpleTest verbose mode
Another exciting new debugging tool that is extremely useful when writing tests is the new verbose mode for Drupal 7 SimpleTest. The verbose mode can be enabled on the SimpleTest settings page.

verbose setting

Once enabled SimpleTest will automatically record the page as it was seen by the SimpleTest browser after each drupalGet() and drupalPost() call. A link is then placed in the test results that will display the page the browser saw and some meta data related to the request.

verbose link

Page 1

verbose page1

Page 2

verbose page2

Manual verbose
In addition to the automatic message provided by SimpleTest custom verbose data may be dumped using DrupalWebTestCase->verbose() which can be used in a test as shown.

<?php
$this
->verbose($data);
?>

If the data to be dumped in not available in the test, but in the code being tested a function is provided that may be accessed by including the DrupalWebTestCase as shown below.

<?php
require_once drupal_get_path('module''simpletest') . '/drupal_web_test_case.php';
simpletest_verbose($data);
?>

Summary
By adding these debugging tools to Drupal 7 the developer experience involved in writing a test has been greatly improved. These methods can still be improved and as such please feel free to file issues in the Drupal 7 SimpleTest issue queue. Also note that this work was sponsored by Acquia as part of my Summer Internship.

openSUSE one-click for LAMP and Drupal

I decided to play around and see if I could create an YaST Meta Package for:

  • LAMP stack
  • Drupal/SimpleTest required PHP extensions
  • phpMyAdmin

After a quick read of the Build Service/Tutorial and a talk with cyberorg in IRC (#suse/#opensuse-buildservice) I was able to upload my pattern to the openSUSE build service.

It was quite simple as I expected since openSUSE's build service and such seem to be quite nice. I have plans to fix up some of my other Drupal packages, add more, and work on personal projects. If you use openSUSE and give it a try, let me know if it works or you have any feedback.

My repository is available at:

Help D7 SimpleTest maintainer and usablity expert get to Drupalcon Paris 2009

The scholarships provided by the Drupalcon Paris team will not cover all the accommodation expenses. Bojhan and myself (boombatower) have decided to stay together and share the expenses. In order to make it to Paris we need to raise at least $500 USD, otherwise I will not be attending.

Take a look at our user pages to get an idea of the work we have done for Drupal.

I plan to give two sessions: Introduction to testing with Drupal: SimpleTest and Sept 2nd, Drupal 7 release party. How continuous integration testing made this plausible.

Bojhan has two presentation proposals as well: Building blocks for your module's UI and How Open Design will drive Drupal 8.

We appreciate your donations and hope to accomplish much at Drupalcon Paris.

Update restore pattern

Recently I have been working on confirming that the Project Issue File Review (PIFR) and Project Issue File Testing (PIFT) update paths from 1.x to 2.x work properly. I was attempting to do so with a copy of the live data from testing.drupal.org and drupal.org. Doing so would ensure that the somewhat delicate upgrade path functioned properly.

I ran into the issue of resetting the data after a failed update attempt. Doing so is not a quick and simple task, especially for the PIFR upgrade which requires renaming the old tables (I may move into udpate path itself) so the data can be converted and placed in the new schema. To solve the problem I developed the following script, or pattern, that can be used to reset for another update attempt. The first script is the one used for the PIFT update which is much simpler. The pift_data.sql contains the live data from drupal.org and similar with the PIFR update.

I placed the scripts in index.php, just as I do with all my scripts, after the <?php drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL); ?> line. The scripts can then be ivoked by appending ?restore=true to the URL to your site. Obviously I commented out the one I was not using.

I found that having these scripts helped ensure that I tested the update path extensively and as such they seem like a good thing to share.

PIFT update reset script

<?php
if (isset($_GET['restore'])) {
  
header('Content-Type: text/plain');
  
ini_set('display_errors'1);
  
set_time_limit(0);

  echo 
"resetting...\n";

  
db_query('TRUNCATE TABLE {watchdog}');

  
db_query("UPDATE {system}
            SET schema_version = %d
            WHERE name = '%s'"
6000'pift');

  
$tables = array(
    
'pift_data',
    
'pift_test',
    
'pift_project',
  );
  
$ret = array();
  foreach (
$tables as $table) {
    if (
db_table_exists($table)) {
      
db_drop_table($ret$table);
    }
  }

  echo 
"importing d.o dataset...\n";

  
$parts parse_url($db_url['default']);
  
$command "mysql -u{$parts['user']} -p{$parts['pass']} " substr($parts['path'], 1);
  
exec($command ' < /home/boombatower/download/pift_data.sql');

  echo 
"finished...\n";

  
print_r($ret);

  exit;
}
?>

PIFR update reset script

<?php
if (isset($_GET['restore'])) {
  
header('Content-Type: text/plain');
  
ini_set('display_errors'1);
  
set_time_limit(0);

  echo 
"resetting...\n";

  
db_query('TRUNCATE TABLE {watchdog}');

  
db_query("UPDATE {system}
            SET schema_version = %d
            WHERE name = '%s'"
6000'pifr');

  
$tables = array(
    
'pifr_branch',
    
'pifr_client',
    
'pifr_file',
    
'pifr_log',
    
'pifr_project',
    
'pifr_result',
    
'pifr_result_detail',
    
'pifr_result_detail_assertion',
    
'pifr_test',
    
'pifr_file_old',
    
'pifr_result_old',
    
'pifr_server',
    
'pifr_log_old',
  );
  
$ret = array();
  foreach (
$tables as $table) {
    if (
db_table_exists($table)) {
      
db_drop_table($ret$table);
    }
  }

  echo 
"importing t.d.o dataset...\n";

  
$parts parse_url($db_url['default']);
  
$command "mysql -u{$parts['user']} -p{$parts['pass']} " substr($parts['path'], 1);
  
exec($command ' < /home/boombatower/download/tdo.sql');

  echo 
"renaming...\n";

  
db_rename_table($ret'pifr_file''pifr_file_old');
  
db_rename_table($ret'pifr_result''pifr_result_old');
  
db_rename_table($ret'pifr_log''pifr_log_old');

  echo 
"importing pifr_server schema...\n";

  
exec($command ' < /home/boombatower/download/pifr_server.sql');

  echo 
"finished...\n";

  
print_r($ret);

  exit;
}
?>

Tags:

Firefox userContent.css for drupal.org issue queue

After seeing a reference to http://userstyles.org/styles/11133 by tha_sun in IRC I went ahead and played with it a bit. I ended up with a very simple version that that just removes the blocks and makes the issue table full width. While I was at it I customized Google.

If you do not know where to put this checkout, per-site custom CSS in Firefox.

@namespace url('http://www.w3.org/1999/xhtml');
 
@-moz-document url-prefix('http://drupal.org/project/issues/') {
  #spacer {}
 
  #contentwrapper {
    background: none !important;
  }
 
  #main {
    margin-left: 5px;
    margin-right: 5px;
  }
 
  #threecol, #content, #squeeze {
    width: 100% !important;
    margin: 0 !important;
    padding: 0 !important;
  }
 
  .sidebar .block {
    display: none !important;
  }
}
 
@-moz-document url('http://www.google.com/') {
  #spacer {}
 
  #ghead, td[align='left'], td[width='25%'], #body > center > font, input[name='btnG'], input[name='btnI'], #footer {
    display: none;
  }
 
  #main > center {
    margin-top: 250px;
  }
}

Acquia internship

Acquia logo

This Summer I will be working part-time as an intern for Acquia. I am very excited to be working with Acquia and having the chance to spend more time improving things that I have interest in. To clarify I will be working on projects that benefit the entire Drupal community. The items I will be working on are improvements to projects I have either started or that I am heavily involved with.

During the discussion of the internship I came up with the following goals that were then prioritized by Dries.

Primary goals

  • Finalize testing of contributed modules and Drupal 6.x projects/core.
  • Add executive summary of test results on project page.
  • Extend the SimpleTest framework so we can test the installer and update/upgrade system.
  • Improve and organize SimpleTest documentation
  • Work on general enhancement of Drupal 7 SimpleTest.

Secondary goals

  • Provide on-demand patch testing environment for human review of patches.
  • Finish refactoring of SimpleTest to allow for a clean implementation of "configuration" testing.
  • Analyze current test quality and code coverage, and foster work in areas requiring attention.

I will post updates on some of the more interesting items as they are accomplished. Additionally, I would like to give a special thanks to Kieran Lal for his mentoring and help in finding me sponsorship.

FOLLOW UP:
To clarify I will still be participating in Google Summer of Code 2009, which was explicit in my agreement with Acquia.
Follow up post by Dries.

Automated Testing System - Statistics

I decided to pull gather some statistics about the automated testing system. These statistics were collected on Wednesday, May 6, 2009 at 4:00 AM GMT. Automatic generation of these statistics along with analysis is a feature I have in mind for ATS 2.0. I appreciate donations to the chipin (right), as this project requires a lot of development time.

From the data you can see that the test slaves have been running tests for the equivalent of 200 days. The system has been running for 192 days and not all the data was included since some of it is inaccurate. That means the system has saved 200 days of developer's time! It is clear that the ATS is a vital part of test driven development. Additionally the time that would have been spent fixing regressions and new bugs has been drastically lowered.

Item Function Value
Time testing SUM 17,310,047 seconds (~288,500 minutes, ~200 days)
Test run (test suite) COUNT 42,351
MAX 3620 seconds (~60 minutes)
MIN 17 seconds
AVG 804 seconds (~ 13 minutes)
STDDEV_POP 783 seconds (~13 minutes)
Test (patch, times tested) COUNT 6,953
MAX 86
MIN 1
AVG 10
STDDEV_POP 15
Test pass count MAX 11,453
MIN 0
AVG 4,265
STDDEV_POP 4,910
Test fail count MAX 6,989
MIN 0
AVG 9
STDDEV_POP 155
Test exception count MAX 813,795
MIN 0
AVG 160
STDDEV_POP 9,893

One item you may notice is the maximum test exception count of 813,795. The patch that caused that many exceptions proved that our system is scalable! The patch is much appreciated. :)

Saved the current test result breakdown.

Result distribution

The average test run length for all the active test slaves can be seen below. This data is only looking at the latest test run for each patch in the system.

Test slave Average test length*
4 730 seconds
5 1,753 seconds
7 1,352 seconds
8 576 seconds
9 2,438 seconds
10 1,942 seconds
12 1,161 seconds
16 217 seconds

* Excludes test runs that do not pass initial checks and fail before running test suite.

Average Test Length

Pages

Subscribe to RSS - drupal