[DRAFT]

Testing

Testing is an important part of the development process, often considered less important than coding - and wrongly so. This page contains some general tips and guidelines to CKEditor testing.

Please note that specific test cases will be included in CKTester.

Preparing Testing Environment

Before the testing process begins, the testing environment that would simulate real user activities should be prepared.

CKEditor Version

When the editor source code is frozen for testing, the selected version should be downloaded.

• All testing should be done on the release version, created with CKReleaser.
• The release version has an entirely different loading sequence and different source code because of the compression process.
• The release version should be loaded into the web server, so that it would run as a normal web application.
• HTTP and SSL protocols on the page should be tested, which means that CKEditor must be loaded to a HTTP server with SSL support.
• Note that some CKEditor samples depend on PHP, thus it would be useful to have access to a web server with PHP support.

Browsers

The editor version under test should be run in  all supported browsers. Each browser should be run with its default configuration or in the safe mode.
Usually each tester will have one or more browsers to test assigned.
Browsers should be run with and without the error console enabled. Note that this is:

• the usual way for the developers using CKEditor to work,
• an easy way to find errors that happen, but are not that easy to spot because the editor seems to be working fine.

Testing Phase

When the environment is ready, the testing process can begin.

Manual Tests

1. Check all the tickets closed in the current milestone to find the list of things with a higher chance of finding a bug. Note that everything should be tested anyway, but this may help in catching even more bugs.
2. Different editor configurations:
   • HTML samples distributed with CKEditor should be launched and tested, including different skins and UI language direction.
   • Every button in toolbar should be checked.
   • Every dialog window should be checked to see if it loads proper data.
   • Every drop-down list should be checked to see if it contains a set of items, as defined in the plugin configuration file.
3. Switching views between WYSIWYG and Source to see if the code is properly interpreted:
   • By entering content in the Source mode and switching to WYSIWYG, and then back.
   • By entering content in WYSIWYG and switching to Source, and then back.
4. Inserting content:
   • Image plugin and Flash plugin: if the dialog window loads proper data and preview.
   • Insert Table plugin: if it creates a table as expected, if the Table Properties dialog window loads proper data.
5. Copy and Paste:
   • Copying and pasting between different editor instances,
   • Copying and pasting between editors running in different browsers,
   • Copying and pasting content from an external editor (like Microsoft Word),
   • Copying and pasting by using keyboard shortcuts and context menu.
6. Drag and Drop
7. Context menu to see if it works properly and according to the selection.
8. BIDI options: Check if the editing works properly when RTL language direction is set.

CKTester

 CKTester can be used to automatically verify existing design tests and ticket tests for regressions. The manual test cases of CKTester could also provide some guidance for further testing.

Reporting Issues

When a bug was found during the testing process:

• Make sure in which version of CKEditor the error that you have found appeared for the first time. It is important to check whether it is a regression or an old bug. Previous versions of CKeditor can be downloaded from here.
• Our revision server  http://rev.ckeditor.com can be used to check the exact revision that introduced the issue. Binary search can be used if it is of a large unsure commit range.
• A bug identified in a specific browser may also affect others. It would be good to check if the problem is generic.
• Fill the bug ticket according to TicketSpecs.
• When new tickets are opened for bugs found during the testing phase, do not start to work on the tickets immediately. Wait for others to confirm the issues and possibly find further flaws in the same area. We will have a separate period of time for bug fixing.
• If a bug looks critical, notify the team as soon as possible (i.e. if fixing it may cause the testing phase to be restarted, sometimes even a trivial change may cause unexpected problems).