Functional Testing

In this article

Overview

Purpose

Scope

Environment

Prerequisites

Functional Tests

Covered areas

How to run the Functional Test

Selenium and VNC Viewer usage in MFTF

Overview

This document provides technical documentation for the functional test scripts developed for LS Modules in Magento 2. These scripts are built using Magento Functional Testing Framework (MFTF) to automate end-to-end testing and to ensure the stability of core functionalities within LS Modules.

Purpose

The primary goal of these functional test scripts is, to validate the expected behavior of LS Modules across various Magento 2 environments. By automating these tests, test coverages are enhanced, manual effort is reduced, and consistent quality in each release is ensured.

Scope

This documentation covers:

  • The structure and implementation of MFTF test scripts for LS Modules.
  • Configuration and setup required for executing the tests.
  • Key test scenarios.
  • How to Run the tests.
  • Selenium and VNC.

Environment

  • Magento Version: 2.4.7-p3
  • Pipeline: Azure DevOps CI/CD Pipeline
  • Custom Module: LS Magento Extension
  • How to Run the tests
  • Selenium and VNC viewer usage.

Prerequisites

Before running the functional tests, make sure that:

  • Magento 2 is set up with the LS Modules installed.
  • The MFTF framework is properly configured (refer to Adobe devdocs to know more on MFTF setup).
  • Required test data and credentials are available.

Functional Tests

To run functional tests provided in the LS Magento extension, certain configurations are required. It is recommended to run each group of functional tests individually to easily identify any failing or error-throwing module.

Functional tests require a dedicated database used exclusively for testing purposes. These tests are only executed in staging or development environments, or on a cloned production environment, to avoid impacting live data.

Additionally, you need to configure the environment variables under magento_root/dev/tests/acceptance/tests/.env file.

For sensitive data such as Magento admin password, payment gateway IDs, secret keys, it must be configured under magento_root/dev/tests/acceptance/tests/.credentials.

Here is a sample .env file:

Copy 
MAGENTO_BASE_URL=http://test.magento.com 
MAGENTO_BACKEND_NAME=admin 
MAGENTO_ADMIN_USERNAME=admin 
SELENIUM_CLOSE_ALL_SESSIONS=true  
SELENIUM_HOST=selenium  
BROWSER=chrome  
WINDOW_WIDTH=1920  
WINDOW_HEIGHT=1080 
MODULE_ALLOWLIST=Magento_Framework,ConfigurableProductWishlist,ConfigurableProductCatalogSearch  
WAIT_TIMEOUT=60  
MAGENTO_CLI_WAIT_TIMEOUT=60  
BROWSER_LOG_BLOCKLIST=other  
ELASTICSEARCH_VERSION=7  
CENTRAL_STORE_CODE=S0013  
HIERARCHY_CODE=FASHIONCOSMETICS  
SERVICE_BASE_URL= http://test.com/commerceservice 
SERVICE_BASE_URL_DISABLED= http://test.com/commerceservice-1 
CS_VERSION=2024.10.0 CENTRAL_VERSION=""26.0.0.0 (26.0.0.0 [25900] CL:True EL:True)""  
INDUSTRY=retail  
EXISTING_CUSTOMER_1_EMAIL=test_user@lsretail.com  
EXISTING_CUSTOMER_1_PASSWORD=Password123@  
EXISTING_CUSTOMER_1_LSR_USERNAME=mc_57745  
EXISTING_CUSTOMER_1_LSR_ID=MSO000012  
EXISTING_CUSTOMER_1_LSR_CARDID=10051  
SIMPLE_PRODUCT_1_ITEM_ID=40430  
SIMPLE_PRODUCT_2_ITEM_ID=40440
SIMPLE_PRODUCT_3_ITEM_ID=40180  
CONFIGURABLE_PRODUCT_1_ITEM_ID=40015  
CONFIGURABLE_VARIANT_PRODUCT_1_VARIANT_ID=000  
CONFIGURABLE_VARIANT_PRODUCT_2_VARIANT_ID=001  
CONFIGURABLE_VARIANT_PRODUCT_1_SKU=40015-000  
CONFIGURABLE_VARIANT_PRODUCT_2_SKU=40015-001  
GIFT_CARD_NUMBER_1=10000000  
ORDER_PREFIX=MFTF-  
RESTRICTED_ORDER_STATUSES_DEFAULT=canceled,fraud,payment_review  
RESTRICTED_ORDER_STATUSES_UPDATED=canceled,fraud,payment_review,processing,pending 

Here is a sample .credentials file:

Copy 
magento/MAGENTO_ADMIN_PASSWORD=<magento_admin_password> 
magento/payflow_pro_partner=<Payflow_Pro_Partner> 
magento/payflow_pro_user=<Payflow_Pro_User>  
magento/payflow_pro_pwd=<Payflow_Pro_Pwd> 
magento/payflow_pro_vendor=<Payflow_Pro_Vendor> 

Covered areas

  • Customer Login & Registration
  • Customer Add Address
  • Reset Customer Password
  • Admin Customer Sync
  • Checkout Login
  • Checkout with Click and Collect Dropdown selection
  • Checkout with Click and Collect Google Map selection
  • Coupon Code on Checkout
  • Coupon Code on Shopping Cart
  • Gift Card Balance Check
  • Gift Card on Checkout
  • Gift Card on Shopping Cart
  • Guest user on Configurable PDP
  • Guest user on Simple PDP
  • Logged In user on Configurable PDP
  • Logged In user on Simple PDP
  • Guest Checkout on Click and Collect and Pay at Store
  • Guest Checkout on Click and Collect and Paypal Payflow
  • Guest Checkout on Flat Rate and Check/Money order
  • Guest Checkout on Flat Rate and Paypal Payflow
  • Store front order cancellation
  • Store front store pages
  • Logged In user on Success page.
  • Admin Order Sync to Central
  • Admin Order Edit and Sync to Central
  • Admin Reset Ls Tables
  • Admin Reset Customer data
  • Admin Reset Product data
  • Admin Reset Tax rules
  • Admin Reset Order data

How to run the Functional Test

Navigate to Magento root/dev/tests/acceptance folder and run the commands as seen below.

  • Using Group Name

    Copy 
    php ../../../vendor/bin/mftf run:group lsCustomer --verbose

  • Using Individual Test Name

    Copy 
    php ../../../vendor/bin/mftf run:test LsAdminStoreConfigurationTest --verbose

Note:- --verbose generates detailed log of execution that helps for any troubleshooting. The mftf.log file is found under Magento root/dev/tests/acceptance folder.

Selenium and VNC Viewer usage in MFTF

  • Selenium in MFTF:

    Magento Functional Testing Framework (MFTF) relies on Selenium to automate browser interactions for test execution. Selenium WebDriver controls the browser and executes test steps as defined in the MFTF XML test scripts. MFTF requires a running Selenium server to communicate with the browser.

  • VNC Viewer for MFTF Test Debugging:

    • Since Selenium runs in headless mode by default (without displaying the browser), VNC Viewer allows users to visually monitor tests in real-time. For example: - Real VNC.
    • In some instances, VNC viewer application is not working unless the "--headless=old" from the chromeOptions in dev/tests/acceptance/tests/functional.suite.yml is removed.

    • Using Selenium for automated testing and VNC Viewer for real-time debugging helps streamline MFTF test execution and troubleshooting.