Changes between Version 2 and Version 3 of Ticket #4218
- Timestamp:
- Aug 9, 2009, 5:57:03 PM (15 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #4218 – Description
v2 v3 1 CKTester is our internal universal testing forge, it should be capable of running tests for different projects1 CKTester is our internal universal testing swarm, it should be capable of running tests for different projects 2 2 with different testing tools easily while remain simplicity, below are some basic ideas of the project 3 3 based on the previous effort on 'FCKtest'. [[BR]] … … 5 5 === Test Cell === 6 6 A '''test cell''' is basically a plain html file reside in individual project's testing source, in which test suites/test cases 7 are written. It'll be executed inside an iframe/popup of the runner pageso '''full page life cycle'''7 are written. It'll be executed inside an iframe/popup of the fort so '''full page life cycle''' 8 8 ( from page load to page destroy ) is guaranteed when performing the test.[[BR]] 9 9 10 Each cell has only key responsibility of '''reporting''' the sucess/failure/time of it's execution to the runner, this is11 done by cross-frame communication with parent runner window.[[BR]]10 Each cell has only key responsibility of '''reporting''' the sucess/failure/time of it's execution to the fort, this is 11 done by cross-frame communication with parent/opener window.[[BR]] 12 12 13 The cell's testing environment, including testing target, runtime libraries, etc. will be all injected by it's belonging14 runner,initiatively or passively, the only required effort in the cell is to include the bootstrap file.13 The cell's testing environment, including testing target, runtime libraries, etc. will be all injected by the fort, 14 initiatively or passively, the only required effort in the cell is to include the bootstrap file. 15 15 {{{ 16 <script type="text/javascript" src="{ path-to-runner}/bootstrap.js"></script>16 <script type="text/javascript" src="{CKTester_ROOT}/bootstrap.js"></script> 17 17 }}} 18 It could choose to manually pull in resources provided by the runner also, 19 e.g. A cell want to load an testing target manually, it could have: 20 {{{ 21 <script type="text/javascript" src="../../bootstrap.js"></script> 22 <script type="text/javascript"> 23 CKTester.require( '${myProjectRoot}/myTarget.js' ); // inline variable ${myProjectRoot} 24 </script> 25 }}} 26 Note that inline variables like above used in the path are not determined by the runner but are defined in '''testing profile''' which know well about the 27 testing stage( target ) location. 18 19 A cell could be running in '''two modes''': 20 1. Managed Mode : It's the default mode when it's running in a batch of cells by the forge, where the cell is running in an iframe. 21 2. Standalone Mode : This mode is adapted when a cell is running separately, on the opposite of 'Managed Mode', the cell will be running inside a popup window, and it doesn't require to report to the fort. 28 22 29 23 === Profile === 30 24 A '''tests profile''' is a js file contains information about running a bunch of test cells. 31 25 It a function declaration which return a configuration object, contains: 32 1. Cell variables : A object contains key/value pair definition of variables which are to be used by individual test cell when communicating with runner.33 1. Cells list : A list of key/values as:34 * Key : The test cell html '''file path''', with each directory within the path is resolved as a tag;26 1. Cell variables : An hash of variable names and their values which are often embeded in profile paths to indicate location. 27 1. Cells list : A list of key/values: 28 * Key : The test cell '''file path''', where all the paths are relative to the runner; 35 29 * Value : An array of '''tags'''. 36 It will basically looks like: 30 1. Cell resolvers : An array of function which were used to manipulate each cell's attributes at run-time, e.g. Inject necessary dependencies for 31 cells when encounter certain tags. Note that The fort already has a few default resolvers, usages including : converting the cell path to tags; calculate resource path for each cell, etc. 32 A profile will basically looks like: 37 33 {{{ 38 34 CKTester.profile = function () … … 40 36 return 41 37 { 42 variables : [ [ 'ckeditorRoot', 'path-to-CKEditor-root'] ],38 variables : { 'CKEDITOR_ROOT' : 'path-to-editor-root' }, 43 39 cells : [ 44 [ ' tt/unit/htmldataprocessor/1' ], // Implicit tags declaration by file path.45 [ ' ckeditor/3210.html' , [ 'ticket', 'unit', 'htmldataprocessor'] ]// Explicit tag declaration.40 [ 'editor/tt/unit/plugins/htmldataprocessor/1' ], // Implicit tag declaration. 41 [ '3210.html' , [ 'tt', 'unit', 'htmldataprocessor'] ] // Explicit tag declaration. 46 42 ] 43 cellResolvers : [ ... ] 47 44 }; 48 45 } 49 46 }}} 50 The '''tag''' plays an important role as an identifier to denote the following aspects of a test cell: 51 1. Environments of the cell, e.g. if it has tags 'unit' and 'editor', it denote a unit test running by YUITest, along with the CKEditor testing stage is required to run the test. 52 2. Categories of the cell, when it's been run in a runner with a specific criteria. E.g. A url criteria as 'http://t/runner?cells=tt,plugins' will run only the ticket test of plugins ( with 'ticket' and 'plugins' tags ). 47 48 It's important to note that '''tag''' plays an important role as the following indicators: 49 1. Environments( dependencies ) of a cell, e.g. if the cell has tags 'unit' and 'editor', it denote a unit test running by YUITest, along with the CKEditor testing stage is required to run the test; 50 2. Categories of the cell, when it's been run in a runner with a specific criteria. E.g. A url criteria as 'http://t/runner?cells=tt,plugins' will run only the tests of category 'ticket' and 'plugins' ( with 'ticket' and 'plugins' tags ); 51 3. Any reasonable denotation. 53 52 54 53 55 === Runner === 56 Test runner is bridge between the testing tools/framework ( e.g. Selenium ) underneath, 57 and the testing resources( our various test cases ), it's resided in CKTester project. 54 === Fort === 55 The '''Test Fort''' is a bridge between the runner tools/framework ( e.g. Selenium ) underneath, and the testing resources, so it's above the level of any TC or TS. It's resided in CKTester project.[[BR]] 58 56 59 The runner is responsible for delegating the running of all the tests to the under-laying tools/framework and collect57 The fort is responsible for delegating the running of all the tests to the under-laying tools/framework and collecting 60 58 the testing result neutrally. It consist of a UI to interact with and providing report. 61 59 62 The runner is required to be running with a specified profile, from which it will register all the defined tests cells .63 Upon requesting by a specific ''' testing criteria''' on running, it shouldfiltering down the registered cells60 The runner is required to be running with a specified profile, from which it will register all the defined tests cells in it. 61 Upon requesting by a specific '''criteria''' on running, it filtering down the registered cells 64 62 to only execute the satisfied ones. 65 63 66 Before runner execute each test cell, an extensible '''environment resolver''' is running there to determinatethe required/requested64 Generally, before runner execute each test cell, an extensible '''cell resolver''' is working out the required/requested 67 65 resources of a specific cell by analysing the '''tags''', guarantee the cell always has all the dependencies 68 66 it should receive. These resources are typically … … 70 68 * The testing target 71 69 * The testing library 72 * Any resources dedicate the one kind of test70 * Any resources dedicated to the cell. 73 71 74 72 === Testing Request === 75 A request is simply a loading of the runner page, which is the beginning of the testing life cycle. 76 1. The request must contains a profile param in the url to specify the location of profile of this running. 77 1. The request could optionally consist of a criteria which is a list of '''tags''' or '''path''' to filter down the test cells to run, could be 78 specified via : 79 * A ampersand separated url params; 80 * A predefined JSON file denote the criteria; 73 A request is simply a page load of the fort, which form the begin of the testing life cycle. 74 1. The request must use url param to specify the location of profile file relative of the fort. 75 1. The request url could optionally consist of a criteria which is a list of '''tags''' or '''path''' to filter the test cells to run, it could be 76 specified in various ways: 77 * A comma separated param in URL; 81 78 * Specified through UI of the test runner page; 82 Some example testing requests look like :83 * Running only the ticket s test of unit test : http://t/runner.html?profile=../profile.js&cells=tt,unit84 * Running a specific test : http://t/runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/185 * Running two specific tests : http://t/ runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/1,../editor/dd/selenium/visual/186 79 Some example testing requests look like ( I've remove the 'profile' param for clear understanding ): 80 * Running only the ticket tests of unit test type: http://t/fort.html?cells=tt,unit 81 * Running all the '''stable''' tests : http://t/fort.html?cells=stable 82 * Running two specific tests : http://t/fort.html?cells=../editor/tt/unit/htmldataprocessor/1,../editor/dd/selenium/visual/1 [[BR]] 83 Note that since it's not convenient to specify a cell path within URL, it's a shortcut that open the cell url directly would have it redirect to correct url back to the forge.
