ВУЗ: Не указан

Категория: Не указан

Дисциплина: Не указана

Добавлен: 01.01.2026

Просмотров: 2646

Скачиваний: 0

ВНИМАНИЕ! Если данный файл нарушает Ваши авторские права, то обязательно сообщите нам.

Vaadin TestBench

Parameters.setScreenshotComparisonTolerance(1.0);

Parameters.setScreenshotRetryDelay(10);

Parameters.setScreenshotComparisonCursorDetection(true);

Parameters.setCaptureScreenshotOnFailure(true);

}

23.6.2. Taking Screenshots on Failure

Vaadin TestBench takes screenshots automatically when a test fails, if the captureScreenShotOnFailure is enabled in TestBench parameters. The screenshots are written to the error directory defined with the screenshotErrorDirectory parameter.

You need to have the following in the setup method:

@Before

public void setUp() throws Exception { Parameters.setScreenshotErrorDirectory("screenshots/errors"); Parameters.setCaptureScreenshotOnFailure(true);

...

}

23.6.3. Taking Screenshots for Comparison

Vaadin TestBench allows taking screenshots of the web browser window with the compareScreen() command in the TestBenchCommands interface. The method has a number of variants.

The compareScreen(File) takes a File object pointing to the reference image. In this case, a possible error image is written to the error directory with the same file name. You can get a file object to a reference image with the static ImageFileUtil.getReferenceScreenshotFile() helper method.

assertTrue("Screenshots differ", testBench(driver).compareScreen(

ImageFileUtil.getReferenceScreenshotFile(

"myshot.png")));

The compareScreen(String) takes a base name of the screenshot. It is appended with browser identifier and the file extension.

assertTrue(testBench(driver).compareScreen("tooltip"));

The compareScreen(BufferedImage, String) allows keeping the reference image in memory. An error image is written to a file with a name determined from the base name given as the second parameter.

Screenshots taken with the compareScreen() method are compared to a reference image stored in the reference image folder. If differences are found (or the reference image is missing), the comparison method returns false and stores the screenshot in the error folder. It also generates an HTML file that highlights the differing regions.

Screenshot Comparison Error Images

Screenshots with errors are written to the error folder, which is defined with the screenshotErrorDirectory parameter described in Section 23.6.1, “Screenshot Parameters”.

For example, the error caused by a missing reference image could be written to screenshot/errors/tooltip_firefox_12.0.png. The image is shown in Figure 23.13,

“A screenshot taken by a test run”.

Taking Screenshots on Failure

529


Vaadin TestBench

Figure 23.13. A screenshot taken by a test run

Screenshots cover the visible page area in the browser. The size of the browser is therefore relevant for screenshot comparison. The browser is normally sized with a predefined default size. You can set the size of the browser window with, for example, driver.manage().window().setSize(new Dimension(1024, 768)); in the @Before method. The size includes any browser chrome, so the actual screenshot size will be smaller.

Reference Images

Reference images are expected to be found in the reference image folder, as defined with the screenshotReferenceDirectory parameter described in Section 23.6.1, “Screenshot Parameters”. To create a reference image, just copy a screenshot from the errors/ directory to the reference/ directory.

For example:

$ cp screenshot/errors/tooltip_firefox_12.0.png screenshot/reference/

Now, when the proper reference image exists, rerunning the test outputs success:

$ java ...

JUnit version 4.5

.

Time: 18.222

OK (1 test)

You can also supply multiple versions of the reference images by appending an underscore and an index to the filenames. For example:

tooltip_firefox_12.0.png tooltip_firefox_12.0_1.png tooltip_firefox_12.0_2.png

This can be useful in certain situations when there actually are more than one "correct" reference.

Masking Screenshots

You can make masked screenshot comparison with reference images that have non-opaque regions. Non-opaque pixels in the reference image, that is, ones with less than 1.0 value, are ignored in the screenshot comparison.

530

Taking Screenshots for Comparison


Vaadin TestBench

Visualization of Differences in Screenshots with Highlighting

Vaadin TestBench supports advanced difference visualization between a captured screenshot and the reference image. A difference report is written to a HTML file that has the same name as the failed screenshot, but with .html suffix. The reports are written to the same errors/ folder as the screenshots from the failed tests.

The differences in the images are highlighted with blue rectangles. Moving the mouse pointer over a square shows the difference area as it appears in the reference image. Clicking the image switches the entire view to the reference image and back. Text "Image for this run" is displayed in the top-left corner to identify the currently displayed screenshot.

Figure 23.14, “The reference image and a highlighed error image” shows a difference report with three differences. Date fields are a typical cause of differences in screenshots.

Figure 23.14. The reference image and a highlighed error image

23.6.4. Practices for Handling Screenshots

Access to the screenshot reference image directory should be arranged so that a developer who can view the results can copy the valid images to the reference directory. One possibility is to store the reference images in a version control system and check-out them to the reference/ directory.

A build system or a continuous integration system can be configured to automatically collect and store the screenshots as build artifacts.

23.6.5. Known Compatibility Problems

Screenshots when running Internet Explorer 9 in Compatibility Mode

Internet Explorer prior to version 9 adds a two-pixel border around the content area. Version 9 no longer does this and as a result screenshots taken using Internet Explorer 9 running in compatibility mode (IE7/IE8) will include the two pixel border, contrary to what the older versions of Internet Explorer do.

23.7. Running Tests in an Distributed Environment

A distributed test environment consists of a grid hub and a number of test nodes. The hub listens to calls from test runners and delegates them to the grid nodes. Different nodes can run on different operating system platforms and have different browsers installed.

Practices for Handling Screenshots

531

Vaadin TestBench

A basic distributed installation was covered in Section 23.2.2, “A Distributed Testing Environment”.

23.7.1. Running Tests Remotely

Remote tests are just like locally executed JUnit tests, except instead of using a browser driver, you use a RemoteWebDriver that can connect to the hub. The hub delegates the connection to a grid node with the desired capabilities, that is, which browsers are installed in a suitable node. The capabilities are described with a DesiredCapabilities object.

For example, in the example tests given in the example folder, we create and use a remote driver as follows:

@Test

public void testRemoteWebDriver() throws MalformedURLException {

//Require Firefox in the test node DesiredCapabilities capability =

DesiredCapabilities.firefox();

//Create a remote web driver that connects to a hub

//running in the local host

WebDriver driver = TestBench.createDriver( new RemoteWebDriver(new URL(

"http://localhost:4444/wd/hub"), capability));

// Then use it to run a test as you would use any web driver try {

driver.navigate().to( "http://demo.vaadin.com/sampler#TreeActions");

WebElement e = driver.findElement(By.xpath( "//div[@class='v-tree-node-caption']"+ "/div[span='Desktops']"));

new Actions(driver).moveToElement(e).contextClick(e)

.perform(); } finally {

driver.quit();

}

}

Please see the API documentation of the DesiredCapabilities class for a complete list of supported capabilities.

Running the example requires that the hub service and the nodes are running. Starting them is described in the subsequent sections. Please refer to Selenium documentation [http://seleniumhq.org/docs/07_selenium_grid.html] for more detailed information.

23.7.2. Starting the Hub

The TestBench grid hub listens to calls from test runners and delegates them to the grid nodes. The grid hub service is included in the Vaadin TestBench JAR and you can start it with the following command:

$ java -jar vaadin-testbench-standalone-3.1.0.jar \ -role hub

You can open the control interface of the hub also with a web browser. Using the default port, just open URL http://localhost:4444/. Once you have started one or more grid nodes, as instructed in the next section, the "console" page displays a list of the grid nodes with their browser capabilities.

532

Running Tests Remotely



Vaadin TestBench

23.7.3. Node Service Configuration

Test nodes can be configured with command-line options, as described later, or in a configuration file in JSON format. If no configuration file is provided, a default configuration is used.

A node configuration file is specified with the -nodeConfig parameter to the node service, for example as follows:

$ java -jar vaadin-testbench-standalone-3.1.0.jar -role node -nodeConfig nodeConfig.json

See Section 23.7.4, “Starting a Grid Node” for further details on starting the node service.

Configuration File Format

The test node configuration file follows the JSON format, which defines nested associative maps. An associative map is defined as a block enclosed in curly braces ({}). A mapping is a key-value pair separated with a colon (:). A key is a string literal quoted with double quotes ("key"). The value can be a string literal, list, or a nested associative map. A list a comma-separated sequence enclosed within square brackets ([]).

The top-level associative map should have two associations: capabilities (to a list of associative maps) and configuration (to a nested associative map).

{

"capabilities":

[

{

"browserName": "firefox",

...

},

...

],

"configuration":

{

"port": 5555,

...

}

}

A complete example is given later.

Browser Capabilities

The browser capabilities are defined as a list of associative maps as the value of the capabilities key. The capabilities can also be given from command-line using the -browser parameter, as described in Section 23.7.4, “Starting a Grid Node”.

The keys in the map are the following:

platform

The operating system platform of the test node. Can be WINDOWS, LINUX, or MAC.

browserName

A browser identifier, any of: android, chrome, firefox, htmlunit, internet explorer, iphone, opera.

Node Service Configuration

533

Vaadin TestBench

maxInstances

The maximum number of browser instances of this type open at the same time for parallel testing.

version

The major version number of the browser.

seleniumProtocol

This should be WebDriver for WebDriver use, or Selenium for tests in the HTML format.

firefox_binary

Full path and file name of the Firefox executable. This is typically needed if you have Firefox ESR installed in a location that is not in the system path.

Server Configuration

The node service configuration is defined as a nested associative map as the value of the configuration key. The configuration parameters can also be given as command-line parameters to the node service, as described in Section 23.7.4, “Starting a Grid Node”.

See the following example for a typical server configuration.

Example Configuration

{

"capabilities":

[

{

"browserName": "firefox", "maxInstances": 5, "seleniumProtocol": "WebDriver", "version": "10",

"firefox_binary": "/path/to/firefox10"

},

{

"browserName": "firefox", "maxInstances": 5, "version": "16",

"firefox_binary": "/path/to/firefox16"

},

{

"browserName": "chrome", "maxInstances": 5, "seleniumProtocol": "WebDriver"

},

{

"platform": "WINDOWS", "browserName": "internet explorer", "maxInstances": 1, "seleniumProtocol": "WebDriver"

}

],

"configuration":

{

"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy", "maxSession": 5,

"port": 5555, "host": ip, "register": true,

"registerCycle": 5000, "hubPort": 4444

534

Node Service Configuration