Stabilize Automation with Selenium Webdriver Waits in Python

This tutorial is driving you to the concept of Selenium Webdriver waits (Implicit and Explicit Waits) and how to apply them in Python.

If you’re familiar with automation, placing waits in efficient places is very important to stabilize automation. It can deal with issues of unstable network, data slow loading, and generally situations we can’t predict how much time we need. It’s very important, so we have to learn it.

In general, we need to wait in a specific timeout to ensure web elements loaded completely or ready, before interacting with them. Webdriver API provides two types of wait mechanisms to handle such conditions: implicit wait and explicit wait.

Implicit Wait

An implicit wait instructs the WebDriver to re-load the DOM in a defined amount of time to try to locate an element if it does not exist. The implicit wait is set by the function implicitly_wait with an amount of seconds. Executing the below code, you could see an exception raises since it can’t find an element with id ‘id-not-on-site’ in 5 seconds.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome(executable_path="C:/files/chromedriver.exe")
driver.implicitly_wait(5)
driver.get("http://google.com")
inputElement = driver.find_element_by_id("id-not-on-site")
inputElement.send_keys("goodevops.com")
inputElement.submit()

driver.close()

Explicit Wait

Almost scenarios of waits are uncertain. Explicit wait works more efficient than implicit wait, but it definitely costs you more efforts. You have to figure out signals to wait in a specific amount of time. Don’t worry! Your efforts are rewarded with more quality or stable scripts.

The following are the two Selenium Python classes for explicit waits.
• WebDriverWait
• Expected Conditions

Example 1: the below code snippet is to wait for an alert dialog appears in 10 seconds

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoAlertPresentException
from selenium.common.exceptions import TimeoutException

try:
    WebDriverWait(driver,10).until(EC.alert_is_present())
    alert_dialog = driver.switch_to.alert
    print ("Alert message: ", alert_dialog.text)
    alert_dialog.accept()
except (NoAlertPresentException, TimeoutException) as ex:
    print("Alert not present")
finally:
    driver.quit()

Example 2: this code snippet is to wait for an input with innertext ‘close’ appears in 20. If it appears, click on it will be performed.

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[text()='close']"))).click()

Again, applying waits is very important to stabilize automation scripts. Hope these practices can help you in some ways. Success.

Deploy Django in Sub-Directory behind uWSGI and NGINX on CentOS 7

It took me awhile to deploy Django in sub-directory. Actually, I move it from sub-domain to sub-directory. I hope this post will save you some hours searching the Internet.

Prerequisite or situation

You are running Django in sub-domain, another domain and you want to move it to a sub-directory. You’re properly having issue with urlpatterns which may lead to code revise. However you still want to keep your code same.

You may also want to check out how to Setup Django behind uWSGI and NGINX on CentOS 7. FYI I am currently using Linode and DigitalOcean to host my sites.

Deploy Django in sub-directory

If you are running Django, along with WordPress, on a VPS like me, you only need to update NginX & uWSGI configuration files.

uWSGI ini

NginX configuration file

Wrap it up

Here you go demo: https://www.vndeveloper.com/django-demo/

That’s it. There are only few things from configuration files. You may also want to check out Setup Django behind uWSGI and NGINX on CentOS 7 to understand how to configure at server side.

References:
[1] https://uwsgi-docs.readthedocs.io/en/latest/Changelog-2.0.11.html#fixpathinfo-routing-action

How Can Artificial Intelligence be Applied in Automation Testing of Web Apps?

There are several interesting web app automation scenarios that we can improve using Artificial Intelligence (AI):

  • Increase test execution stability (self-healing automation) by letting AI to automatically locate web elements when the primary locators fail. This feature already appears in some cutting-edge automation tools like Mabl.
  • Increase automation productivity by using Natural Language Processing (NLP) to automatically translate manual test cases into automated test cases. In theory, this capability should be feasible but I haven’t seen any open-source or commercial solutions available on the market yet.
  • Increase test coverage by auto-generate input parameters that could exhaustively test the API under test using advanced algorithms such as Pairwise.
  • Reduce test execution time by only running the tests that are impacted by a certain code commit instead of exhaustively running all of them after every code commit. For instance, check out the Test Impact Analysis feature of Microsoft Azure DevOps.
  • Reduce failure debugging time by auto analyzing test results and assigning them to categories like Environment Issues, Automation Problems, App Defects, etc. An outstanding example of this category is Report Portal.
  • Perform visual validations on web apps using OCR or image-recognition techniques. There are plenty of tools out there such as TestArchitectAppliTools and SikuliX.

AI is just one trend in the industry. Read more about other trends and predictions here: 2019 Test Automation State-of-the-Practice and Trends | LogiGear Magazine

Leave a comment or contact me if you have anything to add on regarding to applying Artificial Intelligence in Automation Testing of Web Apps.

How to Choose the Best Testing Tool for Automated Testing

I’ve noticed that the process of choosing the best automation testing tool for a project is not always clear. So I’d like to shed some more light on it. The process (Choose the Best Testing Tool) should look like this.

1. Define a list of criteria

Define a list of criteria that your ideal Test Automation framework should meet such as supported automation platforms (desktop, web or mobile?), usability, maintainability, stability, ease of debugging test failures, ease of integrating test automation into a CI/CD pipeline, first-class Docker support, budget/pricing, etc.

2. Weigh the criteria you selected

Which criteria are more desirable and which annoying problems you can live with in the long run? Look around (especially for tool listing articles), and pick at least 3 good Test Automation frameworks that closely match your criteria considering your current situation.

3. Try

Try building a workable framework if you picked open-source solutions like Selenium, Protractor, Cypress, etc. or ask for a demo if you‘re evaluating commercial solutions like UFT, TestComplete, TestArchitect, Ranorex, etc. When you’re at it, pay special attention to the criteria you already listed out. Some call this stage a POC (Proof of Concept). Rate each tool on the scale from 1 to 10 for every criterion you selected. Then compute the score of each tool considering the weight of each criterion. Make sure you’re being fair and your rankings are transitive (if toolX > toolY and toolY > toolZ, toolX must be superior to toolZ).

4. Pick the best tool

Voila! Pick the best tool that got the highest score. No brainer!

Hopefully, this weighted-decision method can help you pick the best software testing tool for automated testing. Leave a comment or contact me if you have anything to add on on how to Choose the Best Testing Tool.

Web Server (VPS) Optimization Checklist for WordPress

I have been using VPS as web server since 2005 to host my sites. Before that, I switched among shared web hosting for many reasons. One of them is cost efficiency because I have more than one websites (using WordPress). I also want to try some thing else like Django, Flask which I cannot do with share web hosting.

If you are about to start a WordPress blog for the first time, you may want to start with shared web hosting. Below are list of shared web hosting suggested by WordPress.org

Spin up a VPS web server

I am currently using both Linode and DigitalOcean for my sites. They are all good and I want to experience with both so I keep using both till now. You can pickup one of them or one at your own choice. There are long list of VPS provider so please do some researches.

I wrote a tutorial on how to setup LEMP stack on CentOS 7. So please go ahead to check it out for basic web server setup.

VPS optimization for WordPress

Below is checklist or to-do list that we need to work on our VPS

  • Configure Nginx with HTTP 2 support
  • PHP extensions, Zend Opcache
  • Nginx GZIP Compression configuration
  • Install and configure Nginx module Brotli Compression
  • Install and configure Nginx module  Google PageSpeed
  • Use W3-Total-Cache: this is actually not server side directly related. However you cannot or hard to use W3-Total-Cache without full control web server.
  • Install and configure Memcached so you won’t get MySQL crash sometime or “Error establishing a database connection”. This one can also speed up your WordPress site since required data is already in memory & smaller PHP script needs to be executed meaning faster web site.
  • If your site already has some content with images, you may want to optimize them using optipng, jpegoptim, pngquant, etc.

web server test result

Final thought

Fast web site needs not only strong web server but also well configured. You really don’t want to pay for a super strong VPS and still see loading time of your site more than 3 seconds. All VPSs’ I used so far is $5/month (1vCPU, 1GB of RAM, 25GB SSD).

Keeping an active desktop session

Credit: this post heavily uses/reuses the content from Arnon Axelrod’s post. For some reasons, I couldn’t leave him a comment because of reCAPTCHA issue. So I decided to create this post and cover part he didn’t. I strongly recommend you to visit his post for full story.

Some test automation tools require an active desktop session to operate normally. I have been struggling with other approaches: autologin, relogin, unlock locked screen, etc. All of them led to same dead end. I already have a post for “How To Unlock Desktop Screen After Remote Access Disconnected” however it requires running cmd command as administrator which is not so convenience.

What locks the desktop?
1. System locked (Windows Key + L) – So far, I don’t see any better solution than using VNC.
2. Remote Desktop session disconnected – See solution below

In my opinion, Arnon Axelrod’s approach or solution is so elegent for this very common test automation problem. What we need to do is keeping an active desktop session by getting cmd command triggered automatically upon RDC disconnected.

In brief, the solution is composed of few pieces
1. A Task Scheduler’s task that is triggered upon disconnecting form Remote Desktop
2. The query session command
3. The TSCon command
4. A windows script that combines the 2 commands

for /f “skip=1 tokens=2” %s in (‘query user %USERNAME%’) do (tscon.exe %s /dest:console)