Master
ThoughtWorks
菜单
关闭
  • 产品及服务
    • 概况
    • "客户体验、产品及设计业务线 "
    • 数据战略、工程及分析业务线
    • 数字化转型及运营业务线
    • 现代化企业、平台及云业务线
  • 合作伙伴
    • 概况
    • 汽车企业
    • 医疗企业
    • 公共服务机构
    • 清洁技术,能源与公用事业
    • 媒体和出版业
    • 零售业和电商
    • 金融和保险企业
    • 非盈利性组织
    • 旅游业和运输业
  • 洞见
    • 概况
    • 特色

      • 技术

        深入探索企业技术与卓越工程管理

      • 商业

        及时了解数字领导者的最新业务和行业见解

      • 文化

        分享职业发展心得,以及我们对社会公正和包容性的见解

    • 数字出版物和工具

      • 技术雷达

        对前沿技术提供意见和指引

      • 视野

        服务数字读者的出版物

      • 数字化流畅度模型

        可以将应对不确定性所需的数字能力进行优先级划分的模型

      • 解码器

        业务主管的A-Z技术指南

    • 所有洞见

      • 文章

        助力商业的专业洞见

      • 博客

        ThoughtWorks 全球员工的洞见及观点

      • 书籍

        浏览更多我们的书籍

      • 播客

        分析商业和技术最新趋势的精彩对话

  • 加入我们
    • 概况
    • 申请流程

      面试准备

    • 毕业生和变换职业者

      正确开启技术生涯

    • 搜索工作

      在您所在的区域寻找正在招聘的岗位

    • 保持联系

      订阅我们的月度新闻简报

  • 关于我们
    • 概况
    • 我们的宗旨
    • 奖项与荣誉
    • 多元与包容
    • 领导层
    • 合作伙伴
    • 辅助功能
    • 新闻
  • 联系我们
China | 中文
  • United States United States
    English
  • China China
    中文 | English
  • India India
    English
  • Canada Canada
    English
  • Singapore Singapore
    English
  • United Kingdom United Kingdom
    English
  • Australia Australia
    English
  • Germany Germany
    English | Deutsch
  • Brazil Brazil
    English | Português
  • Spain Spain
    English | Español
  • Global Global
    English
博客
选择主题
查看所有话题关闭
技术 
敏捷项目管理 云 持续交付 数据科学与工程 捍卫网络自由 演进式架构 体验设计 物联网 语言、工具与框架 遗留资产现代化 Machine Learning & Artificial Intelligence 微服务 平台 安全 软件测试 技术策略 
商业 
金融服务 全球医疗 创新 零售行业 转型 
招聘 
职业心得 多元与融合 社会改变 
博客

话题

选择主题
  • 技术
    技术
  • 技术 概观
  • 敏捷项目管理
  • 云
  • 持续交付
  • 数据科学与工程
  • 捍卫网络自由
  • 演进式架构
  • 体验设计
  • 物联网
  • 语言、工具与框架
  • 遗留资产现代化
  • Machine Learning & Artificial Intelligence
  • 微服务
  • 平台
  • 安全
  • 软件测试
  • 技术策略
  • 商业
    商业
  • 商业 概观
  • 金融服务
  • 全球医疗
  • 创新
  • 零售行业
  • 转型
  • 招聘
    招聘
  • 招聘 概观
  • 职业心得
  • 多元与融合
  • 社会改变
软件测试技术

Speeding up Test Execution with Appium

Sai Krishna Sai Krishna

Published: Jan 29, 2016

Background

Mobile Test Automation is increasingly becoming very important. Almost all web applications are responsive these days and it's very important to test how the application works across devices. The same is true with the native application as well. At the same time, the number of devices and the custom OS versions on devices are also vast. This means that it's harder for a tester to manually run the automated tests over a list of devices to get device coverage and quicker results over every feature development.

Hence there is a need to run the tests parallelly, which provides faster feedback about the quality to your team.There are a couple of open source tools in the market to automate mobile applications. One of them is Appium, a cross platform automation tool, which can run tests on both Android and iOS. There are a number of good blogs, issue threads on github and other discussion forums, which talk about how to run Appium Android tests in parallel across multiple devices with the help of Selenium Grid.  

Existing Challenge

Although there is a lot to learn from these helpful discussions, the setup is expensive and hard to maintain. Here is why: every instance of Appium server with the Android UDID has to be registered with the hub (check how selenium grid works with appium), which means that if you want to run your tests across ten devices, you need to run ten nodes.

Imagine how much system memory you will need if you have to run your tests across fifteen or twenty devices. Also, this becomes more tedious when you have to run your tests with CI across devices.

Solution

I did a lot of reading on how to make this process (of running tests across devices) easier and came up with the below mentioned approach, which would help everyone run tests in a distributed fashion, to get quicker results and to run tests concurrently, to get better device coverage. In the image below,  you can see that the devices are connected in parallel and tests are executed across the devices.

Appium Connections

Right now with v3.0.3, this tool supports only running Android (Native,Hybrid and Web) tests across devices in parallel; support for iOS parallel execution over real devices will soon be implemented.

Usage is very simple. All you need to do is to add the dependencies to your project, create a runner class and connect N+1 devices to a single machine.

Implementation details

To Run in Parallel Mode

The goal is to get device coverage, so we trigger N threads for N devices in parallel and push all the tests to each of the N threads. The Appium server generates runtime ports, starts the server and pushes the tests to the devices.

To Run in Distributed Mode

The goal is to get faster feedback on every feature completed. So we trigger N threads for N devices in parallel. However, instead of pushing all tests to all devices, we distribute the tests across the available threads, and the Appium server generates runtime ports, starts the server and pushes the tests to the devices.

Executor: TestNG

For example: pom.xml should look like this:

<repositories>
	<repository>
		<id>jitpack.io</id>
		<url>https://jitpack.io</url>
	</repository>
</repositories>

<dependencies>
	<dependency>
		<groupId>com.github.saikrishna321</groupId>
		<artifactId>AppiumTestDistribution</artifactId>
		<version>3.0.3</version>
	</dependency>
<dependencies>

Runner.java should look like this:

public class Runner {
	@Test
	public static void testApp() throws Exception {
		ParallelThread parallelThread = new ParallelThread();
		parallelThread.runner("com.test.site");
	}
}

Initialize the AppiumParallelTest runner, which takes care of triggering the Appium session in parallel.

public class UserBaseTest extends AppiumParallelTest {

	@BeforeMethod()
	public void startApp(Method name) throws Exception {
		startLogResults(name.getName());
	}

	@AfterMethod()
	public void killServer(ITestResult result) {
		endLogTestResults(result);
		getDriver().resetApp();
	}

	public AppiumDriver getDriver() {
		return driver;
	}

	@BeforeClass()
	public void beforeClass() throws Exception {
		driver = startAppiumServerInParallel(getClass().getSimpleName());
	}

	@AfterClass()
	public void afterClass() throws InterruptedException, IOException {
		killAppiumServer();
	}
}

You need to specify the Android APK file path, LaunchActivity and TargetPackage of the Android application under test, in the config.properties file.

APP_PATH=/Users/saikrisv/Documents/workspace/PagePatternAppium/build/wordpress.apk
APP_PACKAGE=org.wordpress.android
APP_ACTIVITY=org.wordpress.android.ui.WPLaunchActivity
RUNNER=parallel
APPIUM_JS_PATH=/usr/local/lib/node_modules/appium/bin/appium.js
APP_WAIT_ACTIVITY=org.wordpress.android.ui.accounts.SignInActivity
APP_TYPE=native
BROWSER_TYPE=chrome

You can trigger the test in two different ways -Distribute and Parallel - which means that  when you set the Runner to ‘Parallel’ in the config, the same set of tests run on all the devices connected, which helps us get device coverage.

When you set the Runner to ‘Distribute’ in the config, the framework will distribute the tests across devices, which helps us to get quicker results.

Roadmap

  1. Implement parallel run at method level. Currently, when the execution mode is set to Distribute,  the tests are distributed at class level to the devices. In future, the scope is to distribute tests at method level.
  2. Implement parallel run for iOS real devices for Native,Hybrid and Web apps. Current scope is to run tests in parallel across Android devices. The same implementation will be added to iOS real devices.
  3. Implement ScreenCast as attachment for Android tests. Current scope is to attach a screenshot on failure to the reports, with future improvements to include adding screen-recording as well for test failures
  4. Image Comparison for UI Verification(using ImageMagick API). Add visual assertions to compare two images and validate

The framework does generate detailed HTML reports (using Extent API) which includes AppiumServerLogs and ADBLogs. For  test failures, it includes screenshots.

Appium Report 1

Appium Report 2

Appium Report 3

Project links

  • Take a look @ the repo AppiumTestDistribution
  • Sample Test Project GitHub link to the sample example

Reference links

  • Extent Reports
  • http://appium.io
  • WordPress Android App
  • How appium works for  Android and iOS
  • Setup of Appium on OSX and Windows
This post was designed and produced by Kaifeng Zhang

Master
政策声明 | 现代奴役声明 | 辅助功能
Connect with us
×

WeChat

QR code to ThoughtWorks China WeChat subscription account
© 2021 ThoughtWorks, Inc.