Create Selenium automatic test cases for list creation, removal and indentation

[Martin Kou]

The Selenium automatic test engine is working quite well in branches/features/selenium-tests after [999] and [1000]. It's time to put it to some real use.

The list editing component in FCKeditor 2.5 is now completely implemented in JavaScript. It is working pretty well right now but it has no specifications. We should have at least a rough specification or feature tests for it as its behavior is quite complicated.

Some reference test cases can be found in ​http://www.mozilla.org/editor/list-rules.html. But our test cases should not be entirely the same since our list editing component is XHTML 1.1 compliant and thus behaves slightly differently compared to Midas.

[Martin Kou]

The test cases are added in [1001]. A few minor list command bugs were discovered and fixed while writing the test cases.

[Frederico Caldeira Knabben]

Right at this moment, we should be concentrated on "bug regressions", as we have very few specifications to base our tests (we'll be working on it too). This will be an important usage case for our testing environment. What does it means?

It means that, whenever possible, we should have one or more TCs for each bug at our dev site. We have seen that at Opera and that's really and interesting way to get things stable.

For that, we should have a dedicated SVN repository, out of the trunk, so it can be checked out independently. It should be a web based system, and we can standardize it so everybody can have the same "hosts." DNS entry, pointing to it. We could have a URL like this:

    http://t/fckeditor/ts/visual/1387/001.html

The above URL is formed by:

• "t" : the standard "hosts." entry to use for the tests repository. In this way, we can make references to the TCs in the tickets, reaching them with a mouse click.
• "fckeditor" : the sub-project name.
• "ts" : the test group. In this case, "ticketing system", indicating that here we have tests for tickets. We may have tests that are not related to tickets too.
• "visual" : the test type. I'll explain it later.
• "1387" : the ticket number
• "001.html" : the final testing page. It should be a sequential number, padded with zeros (three chars).

For the ticket groups, we could have:

• "ts" : ticketing system, containing a folder for each ticket number.
• "ds" : design, containing a folder for each piece of code to be tested. We could have folders for classes or global objects.

Not all tests can be done automatically, unfortunately. Some TCs need user input, which can be automated by test frameworks like Selenium. Others need visual verification, which can be achieved by humans only. So we should group our TCs by test framework and model. Some possible types:

• "unit" : xUnit TCs. Those are usually very specific test cases which are usually created during code development. A unit TC is usually restricted to a function, or a class.
• "visual" : TCs that need user input, but can be automated by a framework like Selenium.
• "interactive" : TCs that need user input or user validation, which cannot be achieved with automated tests (#1340, for example).

We can develop this idea further. These are the first things that came to my mind.