About Thuc Nguyen

Thuc Nguyen is Product Owner of TestArchitect – LogiGear’s flagship product which helps simplify creating and maintaining automated tests without coding. Thuc has a great passion for Agile, product management, UX design, and especially complex test automation problems.

How Can Artificial Intelligence be Applied in Automation Testing of Web Apps?

There are several interesting web app automation scenarios that we can improve using Artificial Intelligence (AI):

  • Increase test execution stability (self-healing automation) by letting AI to automatically locate web elements when the primary locators fail. This feature already appears in some cutting-edge automation tools like Mabl.
  • Increase automation productivity by using Natural Language Processing (NLP) to automatically translate manual test cases into automated test cases. In theory, this capability should be feasible but I haven’t seen any open-source or commercial solutions available on the market yet.
  • Increase test coverage by auto-generate input parameters that could exhaustively test the API under test using advanced algorithms such as Pairwise.
  • Reduce test execution time by only running the tests that are impacted by a certain code commit instead of exhaustively running all of them after every code commit. For instance, check out the Test Impact Analysis feature of Microsoft Azure DevOps.
  • Reduce failure debugging time by auto analyzing test results and assigning them to categories like Environment Issues, Automation Problems, App Defects, etc. An outstanding example of this category is Report Portal.
  • Perform visual validations on web apps using OCR or image-recognition techniques. There are plenty of tools out there such as TestArchitectAppliTools and SikuliX.

AI is just one trend in the industry. Read more about other trends and predictions here: 2019 Test Automation State-of-the-Practice and Trends | LogiGear Magazine

Leave a comment or contact me if you have anything to add on regarding to applying Artificial Intelligence in Automation Testing of Web Apps.

How to Choose the Best Testing Tool for Automated Testing

I’ve noticed that the process of choosing the best automation testing tool for a project is not always clear. So I’d like to shed some more light on it. The process (Choose the Best Testing Tool) should look like this.

1. Define a list of criteria

Define a list of criteria that your ideal Test Automation framework should meet such as supported automation platforms (desktop, web or mobile?), usability, maintainability, stability, ease of debugging test failures, ease of integrating test automation into a CI/CD pipeline, first-class Docker support, budget/pricing, etc.

2. Weigh the criteria you selected

Which criteria are more desirable and which annoying problems you can live with in the long run? Look around (especially for tool listing articles), and pick at least 3 good Test Automation frameworks that closely match your criteria considering your current situation.

3. Try

Try building a workable framework if you picked open-source solutions like Selenium, Protractor, Cypress, etc. or ask for a demo if you‘re evaluating commercial solutions like UFT, TestComplete, TestArchitect, Ranorex, etc. When you’re at it, pay special attention to the criteria you already listed out. Some call this stage a POC (Proof of Concept). Rate each tool on the scale from 1 to 10 for every criterion you selected. Then compute the score of each tool considering the weight of each criterion. Make sure you’re being fair and your rankings are transitive (if toolX > toolY and toolY > toolZ, toolX must be superior to toolZ).

4. Pick the best tool

Voila! Pick the best tool that got the highest score. No brainer!

Hopefully, this weighted-decision method can help you pick the best software testing tool for automated testing. Leave a comment or contact me if you have anything to add on on how to Choose the Best Testing Tool.

Where is Software Testing Heading?

Just like other vibrant industries, software testing is changing every day. As a tester, what should you learn to stay on top of your game? Below are some trends you might want to take a look at in 2018. Sharpen the saw!

  • Blockchain app testing: Unless you’ve been living under a rock for the past few years, you’d probably have heard of buzzwords like Bitcoin, Ethereum and Blockchain. Blockchain is taking the world by storm. More and more investments are made on developing Blockchain-based applications, translated: more software testing needed.
    Tip: “Mastering Bitcoin: Unlocking Digital Cryptocurrencies” of Andreas Antonopoulos is a very good start. This book provides basic understanding about Bitcoin and Blockchain through very good examples.
  • Smart product testing: Devices equipped with sensors (think smart toys like Anki Overdrive), voice-based & AI-powered devices (think Amazon Alexa) are taking the center stage. Millions of Amazon Echo and Google Home devices have been sold. Gartner predicts that the market for voice-based wireless speakers will reach $2 billion by 2020. Testers nowadays face a wildly different problem that requires a wildly different skill set.
    Tip: Learn how to program an Alexa “skill” (another name for “apps” on Alexa Marketplace). You just need to sign up for an AWS account (free) and start writing a Lambda function (guide).
  • More test automation: Manual testers have found themselves in a shrinking job market. The industry demands more technical skills like the ability to churn out automated tests, preferably platform-agnostic.
    Tip: If you’re a manual tester, it’s still not too late to learn test automation. Here’s a general guideline: http://qr.ae/TbSswT
  • Wiring automated tests into the pipelines: In today’s DevOps world, bug fixes & new product increments are directly pushed out to end-users, in the continuous fashion. Shit hits the fan if those pieces of code are continuously untested. But the path to Continuous Testing is not always straightforward. Unexpected plumbing landmines are always lurking around the corner.
    Tip: You or your colleagues won’t need to be coding experts to wire automated tests to your pipelines. Almost all ALM tools provide a command-line interface.
  • Service-oriented testing: This is absolutely not a brand new trend in 2018. The trend started long ago: the number of API-level tests keeps growing while UI-level tests keep shrinking. API tests obviously dig deeper, run faster & more reliably (think Fowler pyramid).
    Tip: Learn how to use REST endpoints via tools like POSTMAN and curl first. Then you can write API tests using RestAssured.
  • Involvement of non-engineering testers: More and more non-engineering testers (or some might call “test analysts” or “domain experts”) are taking part in software testing although they don’t possess a strong technical background. This trend calls for an effective scripting language that is readable & writable for non-technical testers, yet executable for the test runners.
    Tip: Focus on the business flows and business logics of your app instead of code. Leave the coding / test implementation to automation engineers. Besides, it’s really useful to get acquainted to the Keyword-Driven method, a solution for non-technical testers to collaborate effectively with technical ones.

Obviously these trends will pivot as the industry progresses. To stay up to date, I’d recommend keeping up-to-date via several information channels below. It’s only my personal ranking / preferences so don’t shoot the messenger.

  1. Automation Awesomeness
  2. Software Development & Testing Insights | TechWell
  3. LogiGear Magazine
  4. StickyMinds
  5. Community Articles | SoftwareTestPro
  6. SD Times – Software Development News
  7. DevOps.com
  8. James Bach – Satisfice, Inc.
  9. Asktester Blog
  10. Testing Excellence

Some testable smart products:
testable smart products

Demystifying 3 Common Misconceptions About Xpath In Web Automation

Problem

Element identification lies at the core of automating web tests because without it, your test automation tool has no clue about how to locate and interact with the correct web elements on your application under test. As recommended by W3CXPath is today’s solution of choice widely adopted by many web automation solutions including the famous Selenium framework. However, just like mastering a katana, making use of XPath up to the proficient level requires quite some time and deliberate effort. This article aims to help you take full advantage of XPath by unveiling the 3 common yet deadly misconceptions which novice testers might pick up while learning this powerful “secret weapon”.

Miconceptions Unveilved

[1] XPath is an inherent property of the web element

Unraveling XPath this way is natural when you are learning the ropes of XPath but later on, it reveals itself as a dangerous analogy. Let’s say you’re copying the XPath of a button (<input>) on Chrome by right clicking on the element and choosing Copy > Copy XPath:

Copy XPath

 

This is what you get:

//*[@id=”Login”]

Meanwhile, if you use Firebug (an add-on of Firefox) on the same particular button, you will receive an utterly different result:

/html/body/div/table/tbody/tr[5]/td/input

How on earth the XPath “property” of the same element is not even remotely similar provided that the element has not changed a wee bit?

It’s because XPath is indeed not a control’s “property” after all. By definition, XPath is a language to tell a browser, a test tool or any other pieces of software how to navigate an XML document and find the specified element. Hence, it’s very normal that different tools return different XPaths for the same exact element.

This implicates that it’s up to you to determine which XPath is the most readable yet reliable to identify an element.

Some test automation frameworks such as LogiGear’s TestArchitect™ offer a built-in feature to purposefully construct an elegant XPath to efficiently locate your web element. For instance, the screenshot below shows TestArchitect’s suggested XPath for the aforementioned button.

Interface Viewer

 

This path (//input[@id=’Login’]) is more elegant because:

  • It’s more specific than Chrome’s XPath (//*[@id=”Login”]) which accepts all tags as long as it has the predefined ID. Chrome’s suggestion would return the wrong element in case there happens to be a <td> with the identical ID (very likely).
  • It’s less fragile than Firebug’s XPath (/html/body/div/table/tbody/tr[5]/td/input) which could be easily broken (returns null) if let’s say, the <input> tag is somehow moved to another table row (e.g. tr[8])

No matter which suggestion is better than which, at the end of the day, you are still in charge of being the final gatekeeper.

[2] XPath is not reliable

This impression comes from the fact that sometimes your test run fails on a different browser (say Firefox) because the element captured using XPath on the original browser (say Chrome) cannot be found. A closer look explains that the button’s XPath on Chrome is actually:

//input[@id=’stdinput-00001′]

While on Firefox, that same button’s XPath is:

//input[@id=’stdinput-0000A’]

This could be the case of dynamically generated elements which is quite popular in web development nowadays. Somewhere along the way, a developer has tied the creation of these web elements to a specific web browser. Although this practice doesn’t promote testability, it still exists out there in the wild.

If you didn’t pay attention, you could blame XPath for doing a lousy job. But in fact, the culprit is not XPath. Instead, it is the application’s peculiar method of generating different IDs on different browsers. While waiting for the developers to fix this inconsistency and improve the app’s testability, you can continue testing by adding a little tweak to your XPath as follows:

//input[contains(@id, ‘stdinput-00001‘) or contains(@id, ‘stdinput-0000A‘)]

Bravo! Your test can now run on both Chrome and Firefox.

[3] XPath is the silver bullet

Relying on XPath too much without carefully dissecting your web application would result in a big frustration when you receive unexpected failures. It’s worth emphasizing that one single XPath can fetch different elements in different contexts. For instance, let’s examine the following XPath:

//a[.=’Mac’]

What you want to interact with is the below <a> element:

Open Store

 

But to your surprise, it’s not the case in reality. Instead, this element is the one being clicked:

Open Store

 

Therefore, to be absolutely sure about what you get, you should invest in a good insight of the page’s hierarchy. In this particular example, you can easily distinguish the two elements based on their different ancestors.

XPath for the product’s name of the search result (case #1):

//div[@id=’content’]//a[.=’Mac’]

XPath for the breadcrumb button (case #2):

//div/ul/li/a[.=’Mac’]

Conclusion

Hopefully unveiling these deadly misconceptions could help you understand XPath on a deeper level so that your tests will become more robust and automating web apps is as effortless as possible. Stay tuned for the upcoming article dedicated to XPath’s best practices.