Блог

Я пытаюсь запустить свой набор тестов на ферме устройств AWS, используя пользовательскую среду. Набор тестов работает нормально локально, но когда я запускаю его на ферме устройств, некоторые тесты случайно завершаются неудачно, а некоторые другие действуют так, как ожидалось. Иногда кажется, что он пропускает перехваты cucumber или просто не выполняет шаги.

Вот моя пользовательская конфигурация среды:

 
# Phases are collection of commands that get executed on Device Farm.
phases:
  # The install phase includes commands that install dependencies that your tests use.
  # Default dependencies for testing frameworks supported on Device Farm are already installed.
  install:
    commands:
      # By default the ruby version installed is 2.5.1
      - mkdir /tmp/tempdir
      - export TMPDIR="/tmp/tempdir"
      - export TMP="/tmp/tempdir"
      - export TEMP="/tmp/tempdir"
      - rvm install "ruby-2.6.5"
      - rvm use 2.6.5
      # you can switch to an alternate ruby version using below command.
      #- rvm install 2.3.1 --autolibs=0

      # Unpackage and install the gems
      - echo "Navigate to test package directory"
      - cd $DEVICEFARM_TEST_PACKAGE_PATH

      # Use a pre-configured ruby environment to run your tests.
      # This environment has the following gems pre-installed (appium_lib (9.16.1), test-unit (3.2.9)) along with their dependencies.
      # If you are using this env, please make sure you do not upload the Gemfile.lock while packaging your tests.
      # If the Gemfile.lock contains different versions for the already installed packages, it will ignore the pre-installed packages.
      # Using this env can help you speed up your test set up phase as you wont have to install all the gems.
      # This default env is only available for ruby 2.5.3.
      - rvm gemset use default-ruby-gemset-env-version-1 --create
      # Alternatively, you can create a new virtual ruby env using the command:
      #- rvm gemset use env --create

      # Install the gems from the local vendor/cache directory
      - gem install bundler --no-document
      - bundle config set path 'vendor/cache'
      - gem update --system 
      - bundle install

      # This test execution environment uses Appium version 1.9.1 by default, however we enable you to change it using the Appium version manager (avm). An
      # example "avm" command below changes the version to 1.14.2.
      # For your convenience, we have preinstalled the following versions: 1.9.1, 1.10.1, 1.11.1, 1.12.1, 1.13.0, 1.14.1, 1.14.2, 1.15.1 or 1.16.0.
      # To use one of these Appium versions, change the version number in the "avm" command below to your desired version:
      - ln -s /usr/local/avm/versions/1.9.1/node_modules/.bin/appium  /usr/local/avm/versions/1.9.1/node_modules/appium/bin/appium.js

  # The pre-test phase includes commands that setup your test environment.
  pre_test:
    commands:
      # We recommend starting appium server process in the background using the command below.
      # Appium server log will go to $DEVICEFARM_LOG_DIR directory.
      # The environment variables below will be auto-populated during run time.
      - echo "Start appium server"
      - >-
        appium --log-timestamp
        --default-capabilities "{"deviceName": "$DEVICEFARM_DEVICE_NAME", "platformName":"$DEVICEFARM_DEVICE_PLATFORM_NAME",
        "app":"$DEVICEFARM_APP_PATH", "udid":"$DEVICEFARM_DEVICE_UDID", "platformVersion":"$DEVICEFARM_DEVICE_OS_VERSION",
        "chromedriverExecutable":"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE"}"
        >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>amp;1 amp;

      - >-
        start_appium_timeout=0;
        while [ true ];
        do
            if [ $start_appium_timeout -gt 60 ];
            then
                echo "appium server never started in 60 seconds. Exiting";
                exit 1;
            fi;
            grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>amp;1;
            if [ $? -eq 0 ];
            then
                echo "Appium REST http interface listener started on 0.0.0.0:4723";
                break;
            else
                echo "Waiting for appium server to start. Sleeping for 1 second";
                sleep 1;
                start_appium_timeout=$((start_appium_timeout 1));
            fi;
        done;

  # The test phase includes commands that start your test suite execution.
  test:
    commands:
      # Your test package is downloaded in $DEVICEFARM_TEST_PACKAGE_PATH so we first change directory to that path.
      - echo "Navigate to test source code"
      - cd $DEVICEFARM_TEST_PACKAGE_PATH

      - echo "Start Appium Ruby test"
      # Modify/Enter the command below to start the tests. The comamnd should be similar to what you use to run the tests locally.
      # "bundle exec" is a Bundle command to execute a script in the context of the current bundle.
      # For e.g. assuming you run your tests locally using command "ruby YOUR_TEST_FILENAME.rb.", to run your ruby tests using bundle exec command you can use:
      - bundle exec rake set_environment[amazon]
      - bundle exec rake test

  # The post test phase includes are commands that are run after your tests are executed.
  post_test:
    commands:

# The artifacts phase lets you specify the location where your tests logs, device logs will be stored.
# And also let you specify the location of your test logs and artifacts which you want to be collected by Device Farm.
# These logs and artifacts will be available through ListArtifacts API in Device Farm.
artifacts:
  # By default, Device Farm will collect your artifacts from following directories
  - $DEVICEFARM_LOG_DIR
  

Вот журналы cucumber (я пытался использовать backtrace для получения дополнительной информации, но по какой-то причине не работает в AWS)

 Start Appium Ruby test
[DeviceFarm] bundle exec rake set_environment[amazon]
[DeviceFarm] bundle exec rake test
Using the test, no_bugs and pretty_progress profiles...
F------

Failing Scenarios:
cucumber -p test -p no_bugs -p pretty_progress features/regression/android/games/lightning.feature:45 # Scenario: Top up modal appears for a user without funds on lightning flow purchase attempts on all or nothing

1 scenario (1 failed)
6 steps (6 skipped)
1m1.742s
Cucumbers failed
  

Я обнаружил, что мои возможности для appium были неправильными, и им не хватало некоторых возможностей, без них appium и cucumber ведут себя странно, вот возможности, которые я использовал:

 caps = {
          "automationName": "UiAutomator2",
          "platformName": ENV['DEVICEFARM_DEVICE_PLATFORM_NAME'],
          "deviceName": ENV['DEVICEFARM_DEVICE_NAME'],
          "app": ENV['DEVICEFARM_APP_PATH'],
          "uuid": ENV['DEVICEFARM_DEVICE_UDID'],
          "appPackage": "appPackageOfYourChoice",
          "appActivity": "appActivityOfYourChoice",
          "autoGrantPermissions": true
      }