The Paris sprint went off without a hitch. We accomplished several goals that will move Drupal into its next stage of development. Among the things accomplished was the addition of the SimpleTest module to Drupal core. Inclusion of a common testing suite with Drupal core will provide a basis for complete testing coverage which was stated as one of the goals for Drupal 7. A necessary component of this initiative is a method for measuring the success of testing. This will be done with "code coverage" tools which measure the percentage of code that is being tested. To put all of this work to good use an automated testing farm is being created which will run the tests on all patches to core. Work on these three areas was unified and pushed forward at the sprint.
An enormous amount of preparation has been completed over the past month and a half in order to prepare SimpleTest for inclusion in the core. The work has included API cleanup and documentation, functional test writing, statistic collection, and unit test planning. The first three objectives were finalized at the sprint before SimpleTest was added to core. Major strides were made on the last objective at the sprint and was planned to become part of core within the next week or two. The whole idea of unit testing is still somewhat up in the air.
Without the sprint the final pieces necessary to bring SimpleTest into the core would have taken much longer to finalize. Thanks again to all those who sponsored my trip to the sprint.
The current plan is for SimpleTest to become part of core in the near future. Once this happens the tests will need to be updated with patches to core. However, at this point SimpleTest is not part of core and whenever core changes break the tests someone, usually me, has to go through and figure out why they broke, make sure that the code they test doesn't actually have a bug, and fix them.
As we draw nearer to the Unit Testing Sprint we are attempting to plan and prepare for the sprint. Having to go through and fix tests whenever the core is changed can be quite time consuming. It would be helpful if we could begin the practice of updating tests along with the core before SimpleTest is officially part of core. Any help in this department would be much appreciated.
Recently a number of the tests broke due to changes in the core. If you would like to help please see the list of tests that should pass and make sure that they still do. Specifically, it appears that commit #109998 has broken over four of the tests and it would be appreciated if others could take the initiative and fix those tests.
Thanks in advance.
It appears that I have created some confusion about what exactly the sprint will be about. In my previous post I stated:
The sprint will take place April 19th - 21st. The sprint will coincide with DrupalCamp Paris in order to attract Drupal enthusiasts to help with the unit testing. Anyone interested should plan on attending the sprint at some point during their stay. There will be instructional time for those planning on contributing at the sprint.
The sprint has come together rather quickly over a very accelerated planning period matched by the accelerated development timeline. The idea of inviting other Drupal enthusiasts to come, learn, and help out in the unit testing sprint was based on some original ideas thrown around in the planning process, but will not be occurring due to logistical and time restrictions.
I realize that some people have made plans to attend the sprint based on the previous description. I am sorry for this miscommunication and thank you for your enthusiasm. If you are interested in coming to Paris to help write tests, contact the testing sprint organizer Kieran Lal.
If you would still like to help there are several ways you can get involved.
- Read the SimpleTest issue queue
- Keep up to date on what needs to be done by reading groups.drupal.org/unit-testing
If you have any other questions please feel free to contact me.
I will be attending the Paris Coding Sprint to work on unit testing for Drupal 7 thanks to those who generously supported me and Google. Leslie Hawthorn was able to secure funding from Google to sponsor both Charlie Gordon and Jimmy Berry (me). Another thanks to those who gave generously to help me get to the sprint.
- Greg Dunlap
- Josh Koenig
- Michael Anello
- Thomas Moseler
- Kent Bye
- Paul Chvostek
- Bonnie Bogle (Development Seed)
The sprint will attempt to get a jump start on unit testing in order to postpone the code freeze for Drupal 7. This is a crucial initiative in advancing Drupal as a whole.
For more information on the sprint see my previous post.
An overview of the plan to get SimpleTest ready for core can be found at the SimpleTest Roadmap for Drupal 7 post which links to the specific discussion on unit testing. Over the weeks since Drupalcon Boston 2008 we have accomplished an enormous amount of cleanup and testing. The SimpleTest module has been transformed into the future of Drupal testing.
At the sprint we intend to continue this by moving into unit testing instead of functional testing which is almost complete. The accelerated development schedule is necessary in order to meet Dries' deadline, which is fast approaching, for complete coverage of Drupal core and extend the developement cycle. Testing will not only extend the cycle of development, but also make Drupal more stable by ensuring that changes to the core do not break Drupal. The tests will provide a wall of protection as illustrated below.
Due to my role in the SimpleTest area it is important that I make it to the sprint. The work that will be completed will not just help one area of Drupal, but Drupal as a whole since it will allow faulty patches to be much more easily recognized which will make reviewing core patches much easier.
Many things need to be completed before the coding sprint. The first of which is the completion of the unit testing plan. Once completed I will update SimpleTest Unit Testing module to use the decided standards and generate the unit test stubs. The module automatically parses all core files and determines what functions need to be tested and what functions each of them are dependent on. The stubs will then be committed to the SimpleTest repository.
All major changes to the SimpleTest API need to be finished so that any tests written at the sprint won't have to be updated. This is almost complete, but there are still a few remaining issues.
The remaining functional tests should be completed. All the functional tests should then be reviewed to ensure that they pass on HEAD, follow coding standards, use the latest SimpleTest API, and test the necessary functionality. I have already begun this process the results of which are posted in the functional tests post. By finishing functional testing it will allow all the focus to be diverted to unit testing which is key in order to get the most accomplished.
The sprint will take place April 19th - 21st. The sprint will coincide with DrupalCamp Paris
in order to attract Drupal enthusiasts to help with the unit testing. Anyone interested should plan on attending the sprint at some point during their stay. There will be instructional time for those planning on contributing at the sprint.
What I Will Accomplish at the Sprint
At the sprint I will accomplish several things in coordination with the other developers at the sprint.
- Help make a significant progress in the unit tests that need to be written. Hopefully we can finish over 50% (over 960 unit tests) of the 1921 tests that need to be written for over 111 files.
- Quickly react to any issues that arise and adjust the plan as needed.
- Work with Karoly to ensure test creation tools for functional and unit tests are good as they can be. I will be recruiting other developers to help write tests so I need to make sure these tools are very easy to use.
- Work with Dries to ensure unit test module I am writing provide the coverage he needs for Drupal 7.
- Work with Kevin Bridges and Douglas Huber to make sure he can easily lead the Drupal 7 maintenance effort with the tests I am writing. These tests may have to be updated hundreds of times, and I need to ensure we agree on how these tests should change as core changes.
- Work with Rok to ensure the automated patch tests can run the functional tests and unit test I am writing. Help debug the tests if there are patch testing automation problems.
In order to accomplish this a group of knowledgeable developers will gather at the sprint in addition to others not coming only just for the sprint.
- Karoli Negyesi
- Rok Žlender
- Charlie Gordon
- Jimmy Berry
- Kevin Bridges
- Douglas Hubler
- Dries Buytaert
After the sprint there will most likely still be some unit tests to write, but a major portion will be done and the guidelines will be affirmed. At this point developement will need to continue in order for the unit tests to be completed before the deadline.
The community will need to rally in order to complete unit testing by the deadline only 90 days after the sprint, but the progress made at the sprint will be necessary in order for this to be plausible.
My Contributions to Date
- I have written 9 functional tests or 27% of the functional tests once they are all completed.
- So far I have officially reviewed 6 of the tests and will continue to review and make sure that all the tests pass and test the necessary functionality.
- Made 318 "commits" (as the cvs log counts them) in the past two weeks.
- Dealt with months of backlogged issues and played a major role in the cleanup and preparation of SimpleTest for core. (over 4 pages of issues)
- Spent numerous hours debugging, planning, and training others for SimpleTest.
In order for this to be accomplished I need financial support. I live in the United States and will need to fly to Paris in order to contribute at the sprint. If you are in a position where you can help donate, anything helps.
You can contribute using the Chip In widget above or by clicking the following link Contribute (no-flash).
I am Jimmy Berry and I would like to attend the coding sprint in Paris to work on unit testing for Drupal 7. I, boombatower, have become a major contributor to the SimpleTest module, both in functional tests and as maintainer. I have been working to get SimpleTest cleaned up and ready for inclusion in the Drupal 7 core.
References and Additional Reading
I have contributed a number of SimpleTests for Drupal and plan on continuing my work after GHOP is over.