In the quest to write scalable, reliable, and fast test automation, new tools are routinely introduced into the market. These tools aim to enhance the efficiency and effectiveness of the testing process, enabling developers to deliver high-quality software with confidence. In 2020, a game-changer emerged in the form of Playwright, developed by the Microsoft team. With its arrival, many have been left pondering: Is Playwright ready to be used, or should our test automation remain reliant on the “tried and trusted” WebDriver automation tools?
To answer this question, it is crucial to gain a comprehensive understanding of what Playwright is and how it differs from WebDriver. Playwright is an open-source automation library designed specifically for browser automation. It allows developers to write automated tests and perform browser interactions seamlessly across multiple platforms and browsers. By providing a unified API, Playwright simplifies the process of building robust and reliable tests, all while maintaining excellent cross-browser compatibility.
On the other hand, WebDriver has long been the go-to choice for browser automation. It is an industry-standard framework that enables developers to control browsers and simulate user interactions. WebDriver has proven its reliability and effectiveness over time, with a vast user base and extensive community support. Many organizations have relied on WebDriver for their test automation needs, forming a solid foundation of trust.
The question of whether to adopt Playwright or stick with WebDriver depends on various factors. In this Playwright VS webdriverIO guide, we will explore the similarities and differences between Playwright and WebDriver, dissecting their features, performance, and overall suitability for different automation scenarios.
Features
| Feature | WebDriverIO | Playwright |
| Programming Language | JavaScript | JavaScript, TypeScript, Python, .NET, Java |
| Category | Browser Automation, Unit Testing, Functional Testing | End-to-End Testing |
| Test Runner | Built-in test runner, supports Jasmine, Cucumber, Mocha | xUnit (No parallel test support with xUnit) |
| Client-side | Yes | Yes |
| Server-side | Yes | Yes (with APIRequestContext methods) |
| Fixtures | Yes | Yes (Test Fixtures for isolated test environments) |
| Group Fixtures | Yes | Yes (Group tests based on meaning) |
| Generators | N/A | Yes (Generate tests, emulate devices, etc.) |
| Mocks | Yes (wdio-wiremock-service) | Yes (Context-wide network interception for request mocking) |
| Grouping | Yes (Test suites and parallel execution with ‘maxInstances’) | Yes (Group tests for logical names and scoping hooks) |
| Other | – | Cross-platform testing, headless/headed modes, mobile testing |
| License | MIT License | Apache License 2.0 |
Playwright VS WebdriverIO Comparison
1. Auto-waits
Auto-waits, a crucial aspect of test automation, are implemented differently in Playwright and WebDriverIO. Let’s explore how each framework handles auto-waits and the implications for test reliability and maintenance.
Playwright’s Auto-Wait:
Playwright has built-in auto-wait functionality, where it automatically performs actionability checks on elements before executing actions. When a test script interacts with an element, Playwright internally waits for the element to become ready and actionable. It checks conditions such as visibility, interactivity, and the absence of overlays or obstructions. Once these conditions are met, Playwright proceeds with the action, ensuring reliable and stable test execution.
Playwright test that automatically waits for links to be clickable:
WebDriverIO’s Approach:
WebDriverIO, being a WebDriver-based framework, follows a traditional approach where developers manually control the waiting time through code in their test scripts. WebDriverIO provides explicit wait functionality, allowing developers to specify waiting conditions and timeouts for elements before performing actions. This approach requires developers to add explicit wait statements, such as `waitUntil` or `waitForClickable`, to ensure elements are ready and actionable.
WebDriverIO test that must wait for links to be clickable
In the above examples, we have two test scripts written in WebDriverIO and Playwright, respectively. Both tests aim to wait for links on a webpage to be clickable and verify that they have text content.
In the WebDriverIO test, we use the wait until function to explicitly wait for each link to become clickable before performing any action on it. This ensures that the link is fully loaded and ready to receive user interactions. The timeout parameter specifies the maximum time to wait for the link to become clickable before throwing an error.
On the other hand, in the Playwright test, Playwright’s auto-wait feature automatically waits for the element to become actionable before proceeding with any action. This means that Playwright internally checks if the element is ready to receive user interactions, removing the need for explicit waiting statements. Hence, we can directly retrieve the link’s text content without any additional waiting code.
Playwright’s auto-wait functionality enhances test reliability and simplifies test maintenance. By automatically waiting for elements to be actionable, Playwright reduces the chances of encountering race conditions and flaky tests. This leads to more stable and reliable test automation workflows.
With Playwright, the burden of managing explicit waits in code is significantly reduced, allowing developers to focus more on the actual test logic. This makes test scripts cleaner, easier to read, and less prone to errors.
2. Network Control
Network Control with Playwright:
In the Playwright code above, we demonstrate different aspects of network control. We simulate a file upload by navigating to a webpage and using setInputFiles to provide a file path for an input element of type “file”. We handle authentication by creating a new context with HTTP credentials and navigating to a protected page. We intercept network requests by creating a new context and using the route method to intercept all requests and log the intercepted URLs. Lastly, we mock out a request-response by creating a new context, defining a route for a specific URL, and fulfilling the route with a mocked response.
Network Control with WebDriverIO:
In the WebDriverIO code snippet, we showcase network control capabilities. We simulate a file upload by navigating to a webpage and using chooseFile to select a file for an input element. We handle authentication by navigating to a login page, filling in the username and password fields, and clicking the login button. We intercept network requests by retrieving performance logs and analyzing the logged messages. Finally, we mock out a request-response by using the mock method to intercept requests to a specific URL and provide a mocked response.
Important Note: Please note that the code snippets provided are simplified examples and may need to be adapted to your specific test scenarios and environments.
3. Browser contexts
– In Playwright, a browser context represents an isolated environment within a browser instance. Each context has its own set of cookies, local storage, and other browser state. It allows you to simulate multi-page scenarios and manage multiple sessions simultaneously.
– When you create a new context in Playwright using `browser.newContext()`, you can perform actions within that context, such as navigating to different URLs, interacting with elements, and maintaining a persistent session.
– Playwright also provides the option to create an incognito context by passing `{ incognito: true }` when creating a new context. This allows you to test scenarios where you need to ensure the website functions correctly in incognito mode without any persistent user data from previous sessions.
Browser Contexts with Playwright:
WebDriverIO:
– In WebDriverIO, the concept of browser contexts is handled differently compared to Playwright. WebDriverIO focuses more on managing browser windows or tabs rather than explicit browser contexts.
– By default, WebDriverIO operates in a single browser context, where actions are performed within the main browser window or tab.
– To simulate multiple browser windows or tabs in WebDriverIO, you can use the `newWindow()` function. This allows you to open new windows or tabs within the same browser session. However, WebDriverIO does not provide explicit built-in support for isolated browser contexts with separate cookies or local storage.
Browser Contexts with WebDriverIO:
Run Parallel Tests on Your CI Server and Cut Down Execution Time
Running tests in parallel on a CI (Continuous Integration) server can significantly reduce the overall execution time of a test suite, allowing developers to get faster feedback and optimize their workflow. By distributing the tests across multiple parallel jobs, the bottleneck of a single job waiting for completion is avoided, leading to a substantial reduction in the total test execution time. LambdaTest, a cloud-based automation testing platform, offers features that enable running parallel tests on CI servers, ensuring faster CI build times. Let’s explore the process in detail:
1. Slow tests and wasted developer time:
Slow tests can be a significant hindrance to developer productivity, especially when waiting for long test execution times. Developers often have to wait for tests to complete before getting feedback on their changes, leading to wasted time and reduced efficiency. By running tests in parallel, the total execution time can be significantly reduced, allowing developers to receive feedback much faster.
2. Splitting tests across parallel CI jobs:
To achieve parallel test execution, the test suite is split into smaller subsets that can be executed concurrently on separate CI jobs. The exact approach for splitting tests depends on the testing framework and CI server being used. Some common strategies include:
Test file-based splitting: Dividing the tests based on the test files or test suites, ensuring that each parallel job executes a separate subset of tests.
Test class/module-based splitting: If the tests are organized into classes or modules, splitting them based on these units to distribute the workload across parallel jobs.
Tag-based splitting: Assigning tags or labels to tests and using these tags to split the tests across parallel jobs. For example, executing tests tagged as “smoke tests” in one job and “regression tests” in another.
3. Running parallel jobs with LambdaTest on CI:
LambdaTest offers seamless integration with popular CI servers like Jenkins, CircleCI, Travis CI, and more. Here’s how you can use LambdaTest to execute parallel tests on your CI server:
- Sign up and create an account on LambdaTest (if you haven’t already).
- Obtain your access key and set it up as an environment variable in your CI server to authenticate your test runs.
- Configure your CI server to split your test suite into smaller subsets and execute them as parallel jobs.
- In each parallel job, configure your test scripts to run on LambdaTest using their provided test automation frameworks (e.g., Selenium, WebDriverIO, etc.). The specific configuration depends on the framework and language you are using.
- Set the desired parallelism level (the number of parallel jobs you want to run) based on the resources available and the desired execution time.
- Ensure that each parallel job uses a different set of test configurations, such as browser/OS combinations, to achieve comprehensive test coverage.
- Trigger your CI server to start the parallel jobs, and LambdaTest will distribute the tests across multiple parallel instances, enabling faster execution.
- Monitor the progress and results of the parallel test runs in the LambdaTest dashboard or integrate the test results into your CI server’s reporting mechanism.
By leveraging the parallel execution capabilities of LambdaTest on your CI server, you can significantly reduce the total execution time of your test suite, allowing you to get feedback in a matter of minutes rather than waiting for hours. This optimizes the CI/CD pipeline, improves developer productivity, and helps deliver high-quality software faster.
Wrap-up
Choosing the right framework depends on your specific requirements and the nature of your project. Playwright’s advanced features and cross-browser support make it a compelling choice for complex and modern web applications. On the other hand, WebDriverIO’s compatibility with Selenium and wide community adoption make it a reliable option for traditional web testing scenarios. In conclusion, both Playwright and WebDriverIO offer robust features for automating web applications. It is important to evaluate your project’s needs and consider factors like cross-browser support, network control, ease of use, and community support when making a decision.