Item4893: Test suite does not work with standard release

For example, one might want to develop plugins against the current release, rather than the develop branch. In doing so, one may want to also run the TWikiSuite to ensure one did not break anything.

If one wants to set up unit tests there are a number of obstacles:

• The UnitTestContrib is not available on twiki.org, but must be retrieved from svn. There is no documentation how to retrieve just a subdirectory and not the whole release.
• The unit tests themselves are not available on twiki.org, but must be retrieved from svn.
• The required CPAN libraries are not documented.
• The file lib/MANIFEST is required to run many test suites. This is not obvious from the error message without actually studying the failing source code. Why this is needed is unclear. This file is not in the standard release but has to be found on svn.
• The unit tests then also require files that are not present in the release. For example, a number of tests rely on TWiki/LoginManager/TemplateLogin.pm which is not in the standard release. This becomes problematic as adding such files may cause problems.

The long and short of my whining is that I think we should maintain the unit tests that held when a release was made. (Unit tests may become outdated as develop progresses, for example, or as in the current case, they may not work on the current release due to newly added dependencies.)

-- TWiki:Main/ThomasWeigert - 26 Oct 2007

We have always made a complete copy of the MAIN directory when we created a release branch. And we always copy the entire release branch when we create a tag of each exact release (+ all the ,v files that are added only to tags).

So what additional things do you propose Thomas?

-- TWiki:Main.KennethLavrsen - 27 Oct 2007

Thomas is right, we need to be working in that direction. I moved the unit tests to a contrib as the first step in the process that would lead to the tests being runnable against a release, rather than always requiring SVN. It's just a question of time. However I'm reclassifying this as an Enhancement.

-- TWiki:Main.CrawfordCurrie - 27 Oct 2007