|
| QF-TEST |
| Release Notes |
|
|
|
|
| This document lists changes to QF-Test since version 3.0. |
|
| Version 3.1.1 - July 28, 2009 |
| |
| New features: |
| |
- The transition to the new check mechanism is now complete. Custom Checkers are now fully documented and can be created for all check data types. All 'Check' nodes now have a 'Check type identifier' attribute and the old 'Check editable', 'Check enabled' and 'Check selected' nodes were replaced with 'Boolean check' nodes with a matching 'Check type identifier'.
- It is now possible to bookmark test-suites, run-logs and individual nodes in a test-suite so they are directly accessible via the new sub-menu »File«-»Bookmarks«.
- The new ClassNameResolver - though also useful for SWT and Swing applications - is an important milestone for AJAX web testing. With its help and the new pseudo class hierarchy for DOM nodes a deeply nested hierarchy of DIV nodes can be turned into a concise component hierarchy with expressive class names that vastly improve component recognition.
- 'Procedures' can now automatically be parameterized via the sub-menu »Parameterize node« in the popup menu for the 'Procedure' node. This feature is work in progress and will need more fine-tuning so we hope for your feedback.
- When creating testdoc documentation it is now possible to follow 'Test call' nodes and include the target 'Test-case', 'Test-set' or test-suite into the generated documentation by activating the respective option in the dialog for interactive use or via the command line argument -testdoc.followcalls in batch mode.
- Several more variants for converting and (un)packing nodes were added.
|
| |
| Version 3.0.2 - March 27, 2009 |
|
| New features: |
|
- Support was added for Eclipse/SWT versions 3.5M6 and 3.4.2. bullet
- The method callProcedure was added to the class DaemonRunContext of QF-Test's daemon API. This method and the existing method runTest now also support direct passing of parameters to this call.
- The Jython and Groovy run context rc now provides the convenience methods getBool, getInt, getNum and getString for simplifying lookup of typed variable values in scripts.
- It is now possible to specify proxy settings for a web SUT directly in the quickstart wizard or via the procedure qfs.web.browser.settings.setProxy in the standard library test-suite qfs.qft.
- An entry for Swing JInternalFrame components has been added to the template file for automatic procedure creation.
|
|
| Version 3.0.0 - December 5, 2008 |
|
| New features: |
|
- Cross-platform and cross-browser web testing with web2.0/AJAX support.
- Integration with HP/Mercury Quality Center and Imbus TestBench.
- Groovy as a second scripting language.
- New Jython version 2.2.1.
- Improved Jython and Groovy script editor with syntax-highlighting.
- Automated creation of basic procedures.
- Extended and unified functionality for 'Check...' and 'Wait...' nodes.
- Direct support for negated checks.
- Split run-logs enable long-running tests without memory problems.
- 64-bit support for SWT on Windows and Linux.
- Extensions and an external driver for the daemon API.
- New concept: "Expected failure" for 'Test-case' nodes.
- Directing and documenting manual tests.
- Extended undo support.
- Immediate vs. lazy variable bindings.
- Image API for creating and saving screenshots.
- Crypting of text input for password fields.
- Recursive sorting and transformation of nodes.
- Revised search with highlighting of results.
- Extensions to the standard library test-suite qfs.qft.
- New tutorial chapters for advanced features.
|
|
| Version 2.2.1 - January 24, 2008 |
|
| New features: |
|
- QF-Test now supports Eclipse/SWT 3.4 milestone 4.
- In addition to locating the next error, it is now possible to jump directly to the previous error in a run-log.
- A new option has been added that limits the number of error messages that are logged in case an SUT contains code that accesses Swing components from the wrong thread and the error message itself has been further improved.
- Progress information shown in the status-line of a test-suite window now includes information about whether the test is currently running, has been paused or has finished.
- Mouse click events with a 'Click count' of 2 or more are now shown as "Double click" or "Multi click" in the tree for easier recognition.
|
|
| Version 2.2.0 - December 5, 2007 |
|
| New features: |
|
- The new 'Data binder' nodes 'Excel data file' and 'Database' enable data-driven testing with data read directly from an Excel file (without the CSV detour) or from a database. Excel support is provided for all operating systems.
- Data for data-driven testing can now also be provided dynamically through a new Jython module and new 'Procedures' in the standard library qfs.qft.
- During replay of a test, progress information is now displayed in the right side of the status bar, showing the overall state as well as the test-case statistics similar to the report. After completion of a test-run the final result will remain visible until another test is started.
- If the 'Comment' attribute of a node contains variable references, it is now expanded at test execution time and stored in the 'Annotation' attribute of the corresponding node in the run-log. For 'Test-set' and 'Test-case' nodes the expanded value is also passed through to the report.
- Nodes in the test-suite can now be sorted alphabetically by selecting the respective nodes, right-clicking and choosing »Sort nodes« from the context menu.
- The new 'Package' shellutils in the standard library qfs.qft provides platform independent 'Procedures' for typical file operations like copy, move or delete and the setText 'Procedures' can now also handle multi-line text.
- The Jython method rc.logMessage now accepts an optional second attribute that can be set to inform QF-Test that the message must not be removed even from a compact run-log.
- QF-Test now logs a warning if it encounters components with ambiguous names. The warning can be suppressed via the option "Log warning for ambiguous name".
|
|
| Version 2.1.1 - October 29, 2007 |
|
| New features: |
|
- Eclipse/SWT version 3.3.1 is now also supported.
- The warning message for non-unique IDs in a test-suite has been improved. Only actual component IDs are now taken into account and the warning is already shown when copying components to a test-suite that cause a duplication of IDs.
- The warning messages for structure and feature mismatch have been improved. They are now more informative and can be suppressed via the options "Log warning for feature mismatch" and "Log warning for structure mismatch".
- The default memory for QF-Test has been increased to 256 MB. This is now configurable via QF-Test's Java configuration dialog on Windows and when running setup.sh or setup.ksh on Unix. If the memory is specified on the command line with -J-Xmx..., the command line setting has precedence, so this change is fully backwards compatible.
- When started on Windows with the command line argument -dbg QF-Test now opens a console window to show the debug messages in. Redirecting QF-Test's output to a file is no longer required.
- Two new 'Procedures' - qfs.swt.sash.moveSash and qfs.swt.sash.moveSashTo - for moving an SWT Sash component by a relative amount or to a specific position are now available in the standard test-suite library qfs.qft.
|
|
| Patch version 2.1.0-p3 - September 26, 2007 |
|
| New features: |
|
- The new feature for detection of events coming from the wrong thread has proved to be very effective. For better understanding the error message has been improved to provide helpful information about the reason of the error and what to do about it. It cannot be said often enough: This feature detects potentially serious problems in the SUT and informs you about them. For additional information, please see the option "How to handle events coming from the wrong thread".
|
|
| Version 2.1.0 - September 12, 2007 |
|
| New general features: |
|
- 'Data drivers' are now also allowed inside 'Test' nodes, so data-driven testing becomes more flexible and is now available outside the strict 'Test-set' context.
- The new 'Data loop' node greatly simplifies looping at 'Test-case' level. It and all other 'Data binders' now can also bind the iteration count for the current loop to an optional variable specified in the 'Iteration counter' attribute.
- The 'Dependency' resolution mechanism has been rewritten from scratch. It is now possible for a 'Dependency' to have multiple 'Dependency references' to the same base 'Dependency' with different values for the 'Characteristic variables'.
- Interaction with external editors is now much more flexible and works well with multi-file editors like JEdit or UltraEdit.
- Interaction with Java WebStart has been improved so that QF-Test can now also handle security-related WebStart dialogs like error messages during application download or certificate confirmation requests.
- New-style reports have been improved. They now optionally include warnings and additional options are provided for selecting whether warnings, errors and exceptions should be listed or not. Also, optional thumbnails with configurable size are now displayed for screenshots in the error listings and screenshots for individual windows are handled in addition to the main screenshot (or screenshots in case of multi-head displays which are now also handled).
- QF-Test now tries to prevent deadlocks caused by SUT bugs where GUI components are accessed from outside the AWT event dispatch thread. An error message is issued instead, the severity of which is defined by the option "How to handle events coming from the wrong thread".
- The rather technical names 'MPRC' and 'PTR' for mouse and key events have finally been dropped and replaced with the more intuitive 'Mouse click' and 'Keystroke'.
- A lot of new icons for the run-log tree help improving the readability of run-logs.
- The standard test library qfs.qft has been greatly enhanced. Please see the tutorial for an overview and its pkgdoc documentation for details.
|
| New SWT features: |
|
- Image checks are now supported for SWT.
- Drag&Drop operations for SWT can now also be recorded directly and are replayed more reliably.
- The native SWT dialogs ColorDialog and FontDialog are now fully supported.
- 'Selection' events have become more flexible and powerful. Instead of an integer, the 'Detail' attribute is now a string that can represent different kinds of actions and choices, including files, directories, colors or a font representation for SWT dialogs. The 'File selection' event node is now obsolete for SWT.
- In case of errors and exceptions screenshots of individual windows of the SUT are now stored in the run-log.
|
| New scripting features and API extensions: |
|
- The Jython API method rc.check has been complemented with a new method rc.checkEqual which is more convenient for comparing a dynamically determined value to an expected value.
- Writing resolvers in Jython has been drastically simplified. The new Jython module resolvers takes care of all the cumbersome details, so a NameResolver can now be implemented and registered with only 3 lines of Jython code.
- Output resulting from print statements in Jython 'Server script' nodes is now multiplexed and copied to QF-Test's terminal view.
- The daemon API has been extended with methods to query running daemons about the objects created by them and the tests currently or recently being executed. Thus it is possible to re-connect to daemons and continue managing them after some kind of system or network failure without having to restart all daemons.
|
|
| Version 2.0.5 - August 16, 2007 |
|
| Bugs fixed: |
|
- Incompatible change: The 'Procedure' qfs.swing.popupmenu.selectItem in the standard library suite qfs.qft is now named correctly (it was previously misspelled as selecItem).
- Replay of a 'File selection' event for SWT could cause an exception.
- Multiple concurrently opened modal SWT dialogs or shells are now handled correctly.
- The special variables ${qftest:client.output.} and ${qftest:client.exitcode.} now also work for multi-thread tests.
- New-style reports for multi-thread tests were not created correctly.
- Error messages from a failed rc.check call were not listed in new-style reports.
- Test counts in the test-suite summary of a new-style report could be wrong.
- Installing QF-Test in a directory with non-7-bit characters, e.g. German umlauts, could cause problems during SUT startup.
- Auto-saving is now turned off in runtime mode. Because test-suites cannot be written in runtime mode, the auto-saver could lead to empty temporary files being created.
- Closing a run-log while creation of a report for the log was still in progress could cause an exception.
- Starting an SUT with Java version 1.3 could cause an exception.
- The server-side-only plugin directory .../qftest/plugin/qftest was not properly included on the classpath on Windows.
|
|
| Version 2.0.4 - July 3, 2007 |
|
| New features: |
|
- Added support for SWT/Eclipse version 3.3 "Europa".
- The tutorial has been revised to better conform with QF-Test version 2.
- When updating checks from the run-log of a failed test-run, regular expressions are now preserved where possible. A warning is issued if a regular expression cannot be retained because it does not match the new value.
|
| Bugs fixed: |
|
- Several report problems were fixed: The same test-cases could be displayed multiple times in the report details, the legend for 'skipped' and 'failed' tests was swapped and the test-case count and resulting percentage in the overall summary was wrong in case of skipped tests.
- JMenuItems are now excluded from the recently introduced focus fix for Swing buttons.
- Replay of key events on SWT Text and Combo widgets now also works correctly with Gtk version 2.10.
- Reuse of running QF-Test instances via the -reuse command line argument now also checks for matching user and QF-Test version.
- In 'SUT script' nodes rc.getComponent(...) now also works correctly for sub-items of SWT components.
|
|
| Version 2.0.3 - May 16, 2007 |
|
| New features: |
|
- Instrumentation of Eclipse/SWT based applications has been greatly simplified and can now be performed automatically as part of a test before starting the SUT. This step has also been integrated into the "Quickstart Wizard".
- Support for Eclipse 3.3 was upgraded to the latest milestone 3.3M7.
- Selecting or closing a tab in an SWT CTabFolder control is now captured and replayed as a 'Selection' event node. This makes it possible to select a currently invisible tab in a CTabFolder with many items and closing a tab is now independent of geometry. Additionally, 'Procedures' for generically selecting or closing CTabFolder items are now provided in the standard library qfs.qft.
- New special variables for accessing client information have been added: ${system:clients}, ${system:clients.all} and ${system:client.exitcode.<name>}.
|
| Bugs fixed: |
|
- For Swing applications, replay of an MPRC event on a button can happen so quickly that the focus doesn't get transferred to the button between the PRESSED and RELEASED events. In that case the current focus owner doesn't lose the focus before the action on the button is triggered which can lead to subtle problems in case the application relies on a FocusListener being invoked first. QF-Test now prevents the problem by explicitly transferring focus to the button first.
- The placeholders '%b', '%p' and '%P' for use in command line arguments did not work correctly when creating reports.
- Fixed a rare NullPointerException caused by updating the title of a run-log window that is just about to be closed.
- The tree in the run-log did not display 'Test-set', 'Test-case' and 'Dependency' nodes correctly.
- Synchronizing tests across multiple machines with rc.syncThreads did not work correctly for tests started in a QF-Test daemon with TestRunDaemon.createContext().
- Decimal numbers are now supported for SWT Spinner widgets.
- When manually executing multiple selected nodes in one go, binding the result of a 'Procedure call' node as a local variable could temporarily interfere with the global variables.
|
|
