Automated Test Framework Atf

Add or import test data to specify parameter runtime values.

Add a variable to a test step to hold a particular type of data when the test runs.

Automate the creation of test suites by using a filter to dynamically add tests to a test suite when they match the filter conditions. Reduce the time that your test designers spend manually creating and maintaining test suites.

View the client test runners table filtered to show only those runners available to run manually-started tests.

View the client test runners table filtered to show only those runners available to run tests to be started by a schedule.

Add to a multi-level test suite by including a child test suite within a parent test suite.

Add a test to a test suite that already exists.

Enable or disable the Automated Test Framework, modify retention policies, move tests between instances, control user access to the Automated Test Framework, and create custom test step configurations and step environments.

On the Properties form, you can set parameters that control how the system executes automated tests and test suites.

Set request and response payload sizes, filter request and response headers, and create basic auth profiles.

On the Auto Flush form, you specify a retention policy for a set of records on a given test results table.

Auto-generate ATF tests by selecting the auto-generate option either from the Auto-generate Tests module or Tests/Suites modules.

This use case illustrates testing a business rule with the Automated Test Framework.

Pause a test to troubleshoot failures or unexpected behavior by adding a breakpoint for a particular test step.

Breakpoints allow you to pause your test at any step of a test run in order to troubleshoot and test authoring.

Basic tasks in the Automated Test Framework.

You can cancel automated tests and automated test suites that are running or are queued to run.

The Client Test Runner opens a browser window or tab for running manually-started client automated tests.

If you are running a test or a test suite, select the Cloud Runner browser option to run your tests in a cloud browser.

Metadata code coverage tracks which lines of code in your custom scripts are executed during ATF test runs. It helps identify untested code, validate test quality, and assess deployment risk in ReleaseOps.

You can compare execution times for different runs of an automated test or automated test suite. You can also compare results over time for a single automated test suite.

To grasp how to interact with configurable workspace components, review these examples.

Learn about the components and its associated actions in the configurable workspace.

Simplify test creation by directly interacting with components on most Configurable Workspace pages via the Page Inspector.

In a step configuration record, the step description generation script field determines the step description that the system generates when a step of this type is added to a test.

In a step configuration record, the step execution script field determines what a step with this configuration does when it runs.

Reduce time when creating tests by copying an entire test suite. Rename and modify the test suite after copying. The Copy Test Suite button on the Test Suite form copies all the nested tests and child test suites within the text suite.

Copy an existing test, which you can then re-name and modify.

Create basic auth profiles to specify basic authentication credentials for Send Request - Inbound test steps.

Create a custom step config category.

Create a custom step configuration that can form the basis of new steps that run on the server.

Create a reusable test to avoid redundancy, ensuring better test maintenance and reliable test execution across the instance.

Create a series of steps for an automated test to run in a specified order.

Group automated tests into a suite you can execute as a batch.

Reduce testing time by creating a template containing a list of steps to add all at once to an automated test.

Create a named automated test containing a series of steps to execute.

Leverage the Page Inspector to create tests by directly interacting with components on most Configurable Workspace pages.

Step configuration records (or step configs) define how each step type behaves. You can create new step configurations that define custom steps that run on the server.

This use case illustrates testing a data policy with the Automated Test Framework.

By default, steps execute in the order in which you created them. You can change this order by editing the Execution Order field.

Modify the Auto Flush data retention policy, which designates how long the system retains data, and referencing data, for test and test suite results. You can change the frequency of flushing for the sys_atf_test_result or sys_atf_test_suite_result base tables. This setting controls how far back in time test result data is available.

Edit an existing test template.

If necessary, edit a test step after you create it.

Allow or prevent tests and test suites from executing on this instance.

The Automated Test Framework tracks data created by running tests and rolls back changes after testing. The system excludes certain tables from being tracked during testing.

You can add a list of REST request and response headers that are not to be saved in step-result records. You can filter headers that might contain authentication credentials or other sensitive information. The phrase "Header redacted for security" is saved instead.

Below is a table of the properties you must have as you set up the ServiceNow Headless Browser for Automated Test Framework.

Improve your UI testing by automating the creation of browsers to process Automated Test Framework (ATF) User Interface (UI) tests. This feature is known as a headless browser, and helps you test UI functionality without having to manually open a browser to the Client Test Runner, which is what processes the UI tests in your local browser.

Look up all 14 ATF Health Check scans, understand what each check detects, and configure the modified-records threshold to fit your environment.

Execute an ATF Health Check scan to detect configuration issues on your instance. You can run a full suite of all 14 checks, run only core checks, run checks for ATF Test Generator and Cloud Runner, or run a single check.

ATF Health Check detects ATF misconfiguration that may lead to support cases by scanning your instance for 14 known issues and providing findings you can resolve proactively.

Inspect and troubleshoot the functionality of different page types like UI Pages, Service Portal, Standard UI, and Custom URL using the Page Inspector.

If you are new to the Automated Test Framework, read this overview to learn what the framework can do. Next, follow the tutorial to create and run a test that uses the most basic of ATF features. After you feel comfortable with the basics, explore more advanced features provided by the ATF.

The Automated Test Framework (ATF) enables you to create and run automated tests to confirm that your instance works after making a change. For example, after an upgrade, during application development, or when deploying instance configurations with update sets. Review failed test results to identify the changes that caused the failure and the changes that you should review.

Select a UI action from a list to perform different actions on a list or a related list.

The following list of tables are not supported by the ATF tests failure resolution feature.

Modify how often active client test runners report in to the system and how long the system retains records for inactive client test runners.

Move automated tests from one instance to another using the normal process for update sets.

After you feel comfortable creating and running simple tests, explore the more advanced features of the Automated Test Framework.

You can troubleshoot automatic test performance by inspecting system transaction log records and potentially shorten execution time by adjusting how often automatic tests capture screenshots.

Identify the HTML and JavaScript page components in your user interfaces that are available for custom UI testing. Enable automated testing by ensuring that your user interfaces only contain testable page components.

Some automated test steps create data that you can use as an input to a subsequent step.

Performance profiling allows you to do performance testing on your instances.

If the test or test suite you are running contains steps that work with a form (any step involving a UI), or any other UI test step element (such as Automated Service Catalog test steps), work with the Pick a Browser dialog.

Reference information for the Automated Test Framework.

Re-run failed tests within a test suite without rerunning the entire suite.

These properties are installed with ATF REST.

Assign a form field the value of an output variable returned from a previous step.

Leverage reusable tests to simplify test maintenance and streamline the management of large tests and test suites. Reusable tests reduce redundancy, ensuring consistent and reliable test execution across your system.

Automated Test Framework is installed with these roles.

Examples of relationship terms and how aggregated results roll up for test suites.

After creating an automated test suite, run it in a non-production instance.

After creating an automated test, run it on a non-production instance.

Schedule one or more test suites to run at a specific date and time.

You can schedule a test suite to run at a specified date and time.

The Scheduled Client Test Run opens a browser window for running scheduled client automated tests.

Capturing many screenshots can impair test performance. You can control which types of screenshots the system captures to minimize this effect.

Use the Select2 component to search and select your option from a drop-down menu easily.

This use case illustrates testing a service catalog request with the Automated Test Framework.

On the Test Step Config Category form, you specify a retention policy for a set of records on a given test results table.

The step config record controls how a test step of this type behaves.

Examples of where the system displays values assigned to some of the step config fields.

The Step Results record contains information about one step in a test result. You access specific step results from the Step Results related list in the Test Results record.

A Scheduled Suite Run record associates a Suite Schedule record with a Test Suite.

The Suite Results record displays information about one test suite schedule.

Run a group of tests in a specific order to test an application or a group of related features.

Test step templates contain a list of steps to be added all at once to an automated test.

The Administration module contains forms for configuring and managing the automated test framework.

Build and execute the ATF tests and test suites using the information in this section.

The Test Results Item (test log) record contains console logging and test execution information.

In the Test record form, you view and edit values of fields for the test record.

Each time you run a test, the automated test framework creates a record of the test results. Use the Test Results module to view details about the results of individual tests and individual steps within a test. If the test generated screenshots, they appear as attachments on the test results record.

A Test Results record contains detailed results information about one test execution. Client Error Details and Failure Details sections appear when a test fails due to error conditions, and include detailed error information you can use for troubleshooting purposes.

If an automated test includes steps that involve a form or any other user-interface (UI) element, it runs those steps in a browser tab or window called a test runner or client test runner.

The Test Suite form contains information about one test suite.

The Test Suite Results record displays information about the results of one execution of one test suite.

The Test Template record contains information about one test template.

Resolve ATF test failures faster using the actionable support provided by the new ATF failure insights feature. You can achieve it by accessing the Find changes since last successful run related link.

Experience the different types of ATF tests and testing techniques.

Follow these step-by-step instructions to create and run your first automated test. This test creates a new user record.

This use case illustrates assigning a form field the value of an output variable from a previous step.

This use case illustrates testing basic form operations with the Automated Test Framework.

Use cases can help you construct tests for common scenarios.

The Get Newly Created Resource via REST API Test test is provided with the Automated Test Framework, and uses the REST - Inbound and assert steps.

This use case illustrates testing a script include with the Automated Test Framework.

With test templates you can add a predefined list of steps to a test. Any list of steps that follows a set pattern makes a good candidate for a template.

Manually create allowed client error entries as needed in the Allowed Client Errors table.

When an automated test is running, view its progress in the Run Test progress dialog.

View test results from completed test and test suite runs. Carefully consider the results of automated test runs and perform any corrective actions required to resolve any revealed issues.

To help troubleshoot performance issues with automatic tests, you can inspect related records from the transactions log entry [syslog_transaction] table.

Test an attachment-dependent business rule by uploading an attachment either from a form or from a server-side API call. For example, you can have a business rule that doesn't let you close an incident without an attachment such as a screenshot.

Create reliable, scalable, and efficient tests by following these design considerations.

The Automated Test Framework helps you ensure the integrity of your instance by enabling the creation and execution of automated tests following any changes.

Validate that your instance still works after you make any configuration change such as apply an upgrade or develop an application. Copy and customize the ServiceNow -provided quick start tests to pass when using your instance-specific data.

Configure client test runner browsers to run automated tests and avoid performance degradations.

Test components in custom UI pages.

Build a test that uses variables to store test data.

Test customized user interfaces such as UI pages and UI macros by retrieving their HTML and JavaScript page components and identifying the test actions they support.

Change the testing properties of a particular page component using HTML attributes that are specific to Automated Test Framework.

Domain separation is supported in the Automated Test Framework. Domain separation enables you to separate data, processes, and administrative tasks into logical groupings called domains. You can control several aspects of this separation, including which users can see and access data.

Enable a developer setting to inspect UI pages that open within the platform. Use the Manual Page Inspector to inspect pages that open in a new tab, such as Service Portal pages.

Create a Docker secret that stores the password of the ServiceNow user who will log into the instance to execute the tests. Docker Secrets is a feature of ServiceNow for securely storing the passwords that will be used in containers.

Create a Docker secret, which stores the password of the ServiceNow user who will log into the instance to execute the tests. Docker Secrets is a feature of ServiceNow for securely storing the passwords that will be used in containers.

Generate TLS/SSL certificates to secure the Docker REST API and authenticate HTTP requests.

Generate TLS/SSL certificates to secure the Docker REST API and authenticate HTTP requests.

Step 7 in the Microsoft Windows setup for the ServiceNow Headless Browser for ATF: Configure ATF with properties.

Step 6 in the Linux setup for the ServiceNow Headless Browser for ATF: Configure ATF with properties.

Configure Docker Server to authenticate all requests.

Complete Step 2 in the Linux setup for the ServiceNow Headless Browser for ATF: Configure Docker Server to authenticate all requests.

Pull the Docker image from the Public Registry.

Pull the Docker image from the Public Registry.

Step 1 in the Windows setup for the ServiceNow Headless Browser for Automated Test Framework: Install Docker.

Step 6 in the Microsoft Windows setup for the ServiceNow Headless Browser for ATF: Set up your instance so it can support the Headless Browser.

Step 5 in the Linux setup for the ServiceNow Headless Browser for ATF: Set up your instance so it can support the Headless Browser.

The ServiceNow Headless Browser for Automated Test Framework provides automation so you can skip having to manually open a browser during testing. The Headless Browser setup is available in both Linux and Microsoft Windows. This topic covers the setup for Linux.

The ServiceNow Headless Browser for Automated Test Framework (ATF) provides automation so you can skip having to manually open a browser during testing. The Headless Browser setup is available in both Linux and Microsoft Windows. This topic covers the setup for Windows.

These tips can help you troubleshoot your Linux or Microsoft Windows setup of the ServiceNow Headless Browser for Automated Test Framework.

Verify that your Headless Browser setup procedures have been successful.

Step 7, the final step in the Linux setup for the ServiceNow Headless Browser for ATF: Verify that your Headless Browser setup procedures are successful.

Identify client errors and resolve them in client-side scripts.

Prevent conflicting tests from running in parallel by marking them as mutually exclusive. For example, when the system identifies tests that modify the same record, the system makes these tests mutually exclusive. You can also manually mark tests as mutually exclusive.

Use the sn-atf-data-type and sn-atf-data-type-params attributes to override the type of field displayed in a Set Component Value test step.

Reduce test design time by running multiple tests and test suites in parallel. Design tests to run in parallel by avoiding resource conflicts and data dependencies.

Run a test multiple times with different test data for each run. Create parameters to store test data for each test run.

Copy and customize quick start tests provided by the ServiceNow AI Platform to validate that your instance works after you make any configuration changes. For example, if you apply an upgrade or develop an application.

The Reported Client Error module lists test logs across all tests that are client errors and have failed. You can review individual test log records, and allowed client errors as needed to prevent them from adversely impacting future test executions.

Test custom inbound web services and backwards compatibility by making REST calls.

Start a client test runner and view information about test runners and test runs.

Execute a scheduled UI test suite immediately using a script without having to wait for the scheduled time. You can use this method while trying to automate the process of running a test.

Execute the following steps to add additional outputs in Run Server Side Script and Custom Scripted StepConfig test steps.

Test business logic and background processes by performing operations on the server.

The Step Configuration Categories module opens a list of records specifying the step categories on the Add Step dialog. From this module, you can add, delete, and edit these categories.

Step configuration records define how each type of step behaves.

A test step environment specifies where the step executes (for example, server versus browser). In this release, custom step configs can use only the Server-Independent environment.

The Suites Results module opens the Suites Results table.

Open the Suites Schedules table. You can drill down to see details about the results of individual schedules or create a new schedule.

The Suites module opens the Test Suites table. You can create, edit, and run test suites from this table.

The Table Cleanup module opens a list of records specifying the retention policies for test result and test suite result tables and the records within them. From this module, you can view and modify these policies.

Find test steps for a particular user interface or ServiceNow AI Platform feature.

Verify the functionality of menus and modules in the application navigator.

Validate catalog item transactions and requester flows from Service Portal.

Interact directly with the configurable workspace components for simpler test creation.

Validate the behavior of page components on custom user interfaces.

Use Automated Test Framework (ATF) to test email notifications, outbound email flows, and inbound email responses.

Validate the functionality of fields and UI actions on a form.

Validate the functionality of fields and UI actions in Service Portal form widgets.

Validate the functionality and visibility of records and UI actions in lists and related lists.

Verify the functionality of REST calls.

Create reusable test components that can be incorporated into various other tests (regular or reusable test), minimizing redundant test code and enhancing test maintainability.

Perform server-side operations. For example, query and update a record, impersonate a user, or run a server-side script.

Validate single catalog item transactions as well as requester and fulfiller flows in Service Catalog.

Validate the functionality of the UI actions.

The Test Templates module opens a list of available templates. From this module, you create, view, and edit test templates.

The Tests module opens the Test table. From here, you can add, edit, and run tests. By opening an individual test record, you can view and edit the steps comprising that test.

Test user interfaces by mimicking user actions and interacting with the visible components of a page.

The Waiting/Running Suite Runs module opens a list of records showing the test suites waiting to be run.

The Waiting/Running Test Run module opens a list of records showing the tests waiting to be run.

Allow client errors as you review step results.

Allow client errors as you review test logs.

Allow client errors as you review test results.

Review the list of existing Allowed Client Error [sys_atf_whitelist] records to see which client errors produce warnings and which are ignored. Modify existing Allowed Client Error records as needed or create new ones.

Add known client errors to the allowed client errors list to allow tests and steps to continue running when a specific error occurs. Set the report level to specify what the Automated Test Framework does when the error occurs in future tests.