Home / Automating Selenium Tests in Azure Pipelines

Overview

Selenium is a portable open source software-testing framework for web applications. It has the capability to operate on almost every operating system. It supports all modern browsers and multiple languages including .NET (C#), Java.

In this lab, you will learn how to execute selenium testcases on a C# web application, as part of the Azure DevOps Release pipeline.

Before you begin

  1. Refer the Getting Started before you follow the below exercises.

  2. Click on the deploy to Azure button below, to provision a Windows Server 2016 virtual machine along with SQL Express 2017 and browsers - Chrome and FireFox.

    Deploy to Azure

    It should take approximately 20-25 minutes to provision the resources. Once the deployment is successful, you will see the resources as shown.

    azure_resources

  3. Use the Azure DevOps Demo Generator to provision the project on your Azure DevOps Organization.

Exercise 1: Create a Deployment Group

We will use Deployment Groups feature in Azure DevOps to deploy the application on a VM which was provisioned earlier to execute the Selenium test cases. Deployment Groups in Azure DevOps makes it easier to organize the servers that you use to host your app. A deployment group is a collection of machines with a Azure DevOps agent on each of them. Each machine interacts with Azure DevOps to coordinate deployment of your applications.

We will also deploy the SQL database in the VM using Deployment Groups.

  1. Go to Deployment Groups under Pipelines tab. Click on Add a deployment group .

    add_deploymentgroup

  2. Provide deployment group name and click on Create.

    create_deploymentgroup

    This will generate a PowerShell script to associate the VM to this deployment group.

  3. Select Use a personal access token in the script for authentication check the box, so that we will not have to provide password every time the script is executed. Click on Copy script to clipboard to copy the script which will be used in the next exercise to associate the VM to the deployment group.

    create_deploymentgroup2

Exercise 2: Associate the VM to Deployment Group

In this exercise, we will execute the registration script on the VM to associate with the deployment group.

  1. Login to the VM using RDP with the following credentials

    • Username: vmadmin
    • Password: P2ssw0rd@123
  2. Open Windows PowerShell in administrator mode, paste the copied Registration script and hit Enter.

    configure_deploymentgroup-2

  3. When the Enter deployment group tags for agent? (Y/N) (press enter for N) > message is prompted in the PowerShell window, type Y and hit enter.

    deploygroup_agent

  4. Enter web, db for the tags

    configure_deploymentgroup

  5. When prompt -Enter User account to use for the service (press enter for NT AUTHORITY\SYSTEM) > is displayed, hit Enter to configure the service to run under NT AUTHORITY\SYSTEM account.

    userserviceaccount-dg

  6. Refresh the Azure DevOps Deployment Groups page you will notice that your Deployment group i online.

    configure_deploymentgroup2

Exercise 3: Configure agent on the VM

Let us configure a private agent on this VM, since Selenium requires the agent to be run in interactive mode to execute the UI tests.

  1. Go to the VM and open the folder C:\VSTSwinAgent.

  2. Open a command prompt in administrator mode. Change the path to C:\VSTSWinAgent and type Config.cmd and hit Enter.

  3. Provide the following details:

    • Enter server URL: Your Azure DevOps Organization URL
    • Authentication type: Type PAT and press the enter key for PAT as the authentication type and paste the PAT in the next prompt.
    • Let us use the default options for rest of the configuration. Press Enter for all prompts until the command execution completes.
    • Once the agent is registered, type run.cmd and hit Enter to start the agent.

    Click here for more information on how to configure the agent.

    configure_windowsagent

Exercise 4: Configure Release Pipeline

The target machine is available in the deployment group to deploy the applications and run selenium testcases. The release definition uses Phases to deploy to target servers.

  1. Go to Releases under Pipelines tab. Select Selenium release definition and click on Edit.

    setuprelease

  2. Open Dev environment to see the three deployment phases.

    setuprelease2

    releasephases

    • IIS Deployment phase: In this phase, we deploy application to the VM using following tasks-

      • IIS Web App Manage: This task runs on the target machine registered with the Deployment Group. It creates a website and an Application Pool locally with the name PartsUnlimited running under the port 82 http://localhost:82

      • IIS Web App Deploy: This task deploys the application to the IIS server using Web Deploy.

    • Database deploy phase: In this phase, we use SQL Server Database Deploy task to deploy dacpac file to the DB server.

    • Selenium tests execution: Executing UI testing as part of the release process is a great way of detecting unexpected changes, and need not be difficult. In this phase, we will execute Selenium tests on the deployed web application. The below tasks describes using Selenium to test the websites in the release pipeline.

      • Visual Studio Test Platform Installer: The Visual Studio Test Platform Installer task will acquire the Microsoft test platform from nuget.org or a specified feed, and add it to the tools cache. It satisfies the ‘vstest’ demand and a subsequent Visual Studio Test task in a build or release pipeline can run without needing a full Visual Studio install on the agent machine.
      • Run Selenium UI tests: This task uses vstest.console.exe to execute the selenium testcases on the agent machines.
  3. Click on IIS Deployment phase and select the Deployment Group which we have created in Exercise 2.

    setuprelease_IIS

  4. Repeat the above step for SQL Deployment phase

    setuprelease_db

  5. Click on Selenium tests execution phase and set Agent queue to Default then Save the changes.

    setuprelease_selenium

Exercise 5: Trigger Build and Release

In this exercise, we will trigger the Build to compile Selenium C# scripts along with the Web application. The resulting binaries are copied to Azure VM and finally the selenium scripts are executed as part of the automated Release.

  1. Navigate to Builds under Pipelines tab and queue the build.

    buildqueue

  2. This build will publish the test artifacts to Azure DevOps, which will be used in release.

    buildqueue2

  3. Once the build is successful, release will be triggered. Navigate to Releases tab to see the deployment in-progress.

    releasequeue

  4. When Selenium test execution phase starts, connect back to the VM provisioned earlier to see UI tests execution.

    Releaseprogress

  5. In this lab, we are executing four UI test scenarios configured to run on Chrome and Firefox browsers.

Tests running in Chrome

seleniumtest

Tests running in Firefox

seleniumtestfirefox

Once the release succeeds, click on the Tests tab to analyze the test results. Select the required filters from the dropdown in Outcome section to view the tests and their status.

analyzetests

Summary

In this lab, you have learnt how to automate and execute Selenium UI test cases on different browsers on an Azure VM with Azure Pipelines.