Selenium integration testing

[harsha.galla]

Isomorphic,

We want to go the selenium way for testing the UI code. We want to be able to use selenium to maintain test suites which would be integrated into maven for code coverage and other purposes. I have read through the selenium documentation present in the smartgwt docs. Also read through the basics from Selenium website.

As per selenium docs, there are two approaches, the older RC way and the new WebDriver way. Which one would best support SmartGWT. Also I have read some of the threads where people were discussing that scLocator generated by smartgwt is not consistent and would require us to use unique ID's(by SetID()) for components to be able to locate them.


I followed the instruction given in the selenium document in smartgwt docs. I was able to simulate a test using selenium IDE which runs fine. But when I export the test to JUnit4 webdriver class, the resulting java class created has nothing but the first line which would open the url being tested and others as errors:

package com.example.tests;

import java.util.regex.Pattern;
import java.util.concurrent.TimeUnit;
import org.junit.*;
import static org.junit.Assert.*;
import static org.hamcrest.CoreMatchers.*;
import org.openqa.selenium.*;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.Select;

public class Seleniumtestcase {
	private WebDriver driver;
	private String baseUrl;
	private StringBuffer verificationErrors = new StringBuffer();
	@Before
	public void setUp() throws Exception {
		driver = new FirefoxDriver();
		baseUrl = "http://127.0.0.1:8888/";
		driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
	}

	@Test
	public void testSeleniumtestcase() throws Exception {
		driver.get(baseUrl + "/MarketIntelligence.html?gwt.codesvr=127.0.0.1:9997#OD");
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//TabSet[ID="isc_Report_0"]/paneContainer/]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=originType||title=Origin||value=Airport||index=0||Class=SelectItem]/[icon='picker']]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=originType||title=Origin||value=Airport||index=0||Class=SelectItem]/pickList/body/row[originType=Domestic||5]/col[fieldName=originType||0]]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=originType||title=Origin||value=Domestic||index=0||Class=SelectItem]/[icon='picker']]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=originType||title=Origin||value=Domestic||index=0||Class=SelectItem]/pickList/body/row[originType=International||6]/col[fieldName=originType||0]]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=destinationType||title=Destination||value=Airport||index=3||Class=SelectItem]/[icon='picker']]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//DynamicForm[ID="isc_MarketCriteria_0"]/item[name=destinationType||title=Destination||value=Airport||index=3||Class=SelectItem]/textbox]]
		// ERROR: Caught exception [Error: unknown strategy [sclocator] for locator [scLocator=//TabSet[ID="isc_Report_0"]/paneContainer/]]
	}

	@After
	public void tearDown() throws Exception {
		driver.quit();
		String verificationErrorString = verificationErrors.toString();
		if (!"".equals(verificationErrorString)) {
			fail(verificationErrorString);
		}
	}

	private boolean isElementPresent(By by) {
		try {
			driver.findElement(by);
			return true;
		} catch (NoSuchElementException e) {
			return false;
		}
	}
}

We havent used selenium before, can you point us in the right direction which would best suit our needs to be able to integrate selenium tests in to maven and generate code metrics etc.

Also any samples (could be of the showcase project you guys have) would be helpful.

Thanks.

[Isomorphic]

We support and recommend Selenium classic over WebDriver, it has several key advantages.

When there is no component ID specified, locators will sometimes incorporate automatically generated IDs which have a sequence number (eg isc_Object_355). If your test has unpredictable execution order (for example, two simultaneous network operations take place and either may complete first, and both generate UI components on completion) then these IDs will not be stable from run-to-run. They will likewise not be stable if you test part of an app and then embed it in a larger app and try to use the same script. Either make your tests predictable or use IDs judiciously to avoid these problems.

Your test is failing because the target page did not load our Selenium extensions. See the user guide in the "selenium" directory in the SDK.

For best practices we recommend setting up Selenium RC so that it runs a suite of tests every time someone checks into your source control, and reports them to your team somehow (email, CI server dashboard, etc). This is not directly related to Maven. Some CI servers have partial support for this. See also SauceLabs for the ability to run your Selenium RC browsers in the cloud, on demand.

If you would like samples created for this setup, it would be a valid Feature Sponsorship to ask us to write an article for the public wiki (wiki.smartclient.com) showing an example setup.

[p27939]

Is there a plan to move forward to support and recommend WebDriver, given that RC has been officially deprecated?

According to the front page of Selenium’s website http://seleniumhq.org/

“Selenium WebDriver is the successor of Selenium Remote Control which has been officially deprecated.”

Thanks.

[Isomorphic]

The developers working on WebDriver have made this claim, as is predictable when you are trying to get people to try out what you consider to be latest and greatest.

The feeling in the community at large is different, and various ways of measuring activity suggest there is far more activity around Selenium RC (such as: almost 3x more Google results for "Selenium RC" than WebDriver).

We have plans to look at WebDriver support, and if you are interested in accelerating this, you can do use the Feature Sponsorship program to do so.

Currently, no plans to recommend WebDriver over Selenium RC. To actually recommend it over Selenium RC it would have to have compelling advantages and minor drawbacks and this is definitely not the case.

[acarur01]

We can't find anything on how to use classic selenium with tablets. All we see are WebDrivers - Do you have any site that you can point us to so that we can start testing on android/ipad and blackberry tablets?

[Isomorphic]

You have probably also noticed that the sites talking about WebDriver for mobile also mention that's it flaky if it works at all..

You can run your Selenium tests using Selenium Core and manually directing the device to the Selenium test suite based on core. Details are here.

[Depado]

There is a reason if everyone wants to use Webdriver :
- Faster testing (unit and integration) because it does not open a remote control browser
- Faster testing because you can execute all your tests in one browser

The only reason why we can't use it is because : scLocators are not recognized. So you can use Selenium RC without any problems, but we want to use Webdriver. People are actually trying to make it work in Java directly like this :

public By byScId(final String locator) {

		  final String id = locator
		      .replaceAll("'", "")
		      .replaceAll("\"", "");

		  return new By() {
		    public List<WebElement> findElements(SearchContext ignored) {
		      throw new UnsupportedOperationException("findElements");
		    }
		  
		    public WebElement findElement(SearchContext context) {
		      if (!(context instanceof JavascriptExecutor)) {
		        throw new NoSuchElementException("context must implement JavascriptExecutor");
		      }
		      
		      JavascriptExecutor js = (JavascriptExecutor) context;
		      
		      return (WebElement) js.executeScript(
		          "var o = window[arguments[0]]; " +
		          "if (!o) { return null; } " +
		          "var scLocator = '//' + o.getClassName() + '[ID=\"' + arguments[1] + '\"]'; " +
		          "return window.isc.AutoTest.getElement(scLocator)",
		          id, locator);
		    }
		  };
		}

I did not test, but I will in the following days because it will improve my testing time A LOT. The only thing I'm worrying about is : What if the scLocators are working, but my programm needs more stuff, will it work without the user-extensions.js ?

Wait and see.

[Isomorphic]

This thread has a comprehensive response on WebDriver support as well as pros/cons of WebDriver.

[tellington]

We support and recommend Selenium classic over WebDriver, it has several key advantages.

When there is no component ID specified, locators will sometimes incorporate automatically generated IDs which have a sequence number (eg isc_Object_355). If your test has unpredictable execution order (for example, two simultaneous network operations take place and either may complete first, and both generate UI components on completion) then these IDs will not be stable from run-to-run. They will likewise not be stable if you test part of an app and then embed it in a larger app and try to use the same script. Either make your tests predictable or use IDs judiciously to avoid these problems.

Your test is failing because the target page did not load our Selenium extensions. See the user guide in the "selenium" directory in the SDK.

For best practices we recommend setting up Selenium RC so that it runs a suite of tests every time someone checks into your source control, and reports them to your team somehow (email, CI server dashboard, etc). This is not directly related to Maven. Some CI servers have partial support for this. See also SauceLabs for the ability to run your Selenium RC browsers in the cloud, on demand.

If you would like samples created for this setup, it would be a valid Feature Sponsorship to ask us to write an article for the public wiki (wiki.smartclient.com) showing an example setup.

A) "When there is no component ID specified..."

How does one specify a component ID?

Our code creates a DynamicForm populated with FormItems. DynamicForm has a setID() method but the FormItem does not.

I have tried FormItem.setAttribute() and setProperty(). However, when I execute those methods, I get something like this:

<div ...$89="theValueSet"...</div>

instead of

<div ...ocsErrorItem="theValueSet"...</div>

Where '$89' is the key used across all the FormItems where I try doing the set methods.

B) "...your target page did not load our Selenium extensions.'

I did execute a server, stand-alone, with the provided extensions and then executed an IDE-generated Junit test using Selenium RC. Just as in the IDE, my RC-executed test case completed successfully.

I then switched the order of two of the check box FormItems and re-ran the RC test...it failed because the scLocator value for each element changed.

Does this really mean that even just a creation order change will necessitate completely recreating a test case in the IDE in order for the scLocator magic to work in the RC and the IDE?

Do the element ids, for FormItems, change with every execution? Or are they static for every execution for a given build?

[Isomorphic]

You set a component ID via setID().

We can't tell if your other question is about setting ID or something else.

Yes, if you set no IDs at all, there is no definite way to identify components so there will be order dependencies. As the best practices section in the docs says, you really just need to set one or a handful of IDs to and all child components end up locatable by the parent's ID, without needing IDs of their own.