Блог

Я пытаюсь запустить конвейер данных в Azure Devops со следующим определением YAML

Это requirements.txt файл:

 trigger: none

schedules:
  - cron: "0 0 * * *"
    displayName: "Daily midnight data pipeline run"
    branches:
      include:
      - master
    always: true

name: 'data_pipeline'
jobs:
  - job: 'data_pipeline_job'
    pool:
      vmImage: 'ubuntu-16.04'
    variables:
      - group: KeyVault
      - group: ProductionEnvVars
    steps:
      - task: UsePythonVersion@0
        inputs:
          versionSpec: '3.9.0'
          architecture: 'x64'
    
      - script: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
        displayName: 'Install requirements'
    
      - script: |
          python src/my_custom_package/create_data.py
        displayName: 'Create and Register Data'
        env:
          STORAGE_ACCT_NAME: $(StorageAccountName)
          STORAGE_ACCT_KEY: $(StorageAccountKey)
          TENANT_ID: $(TenantID)
          SPN_ID: $(SpnID)
          SPN_PASSWORD: $(SpnPassword)
          AML_WORKSPACE_NAME: $(AmlWorkspaceName)
          RESOURCE_GROUP: $(ResourceGroup)
          SUBSCRIPTION_ID: $(SubscriptionID)
  

Однако после некоторых успешных предварительных условий я начинаю получать эти ошибки

  Using legacy 'setup.py install' for numpy, since package 'wheel' is not installed.
  Installing collected packagesGenerating script.
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/1afa3c52-aff1-4a34-ac26-f17b4946deec.sh
Collecting pip
  Downloading pip-20.2.4-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
Successfully installed pip-20.2.4
Obtaining file:///home/vsts/work/1/s/src (from -r requirements.txt (line 8))
Collecting azureml-sdk==1.2.0
  Downloading azureml_sdk-1.2.0-py3-none-any.whl (4.6 kB)
Collecting azure-storage-blob==12.3.0
  Downloading azure_storage_blob-12.3.0-py2.py3-none-any.whl (279 kB)
Collecting joblib==0.14.1
  Downloading joblib-0.14.1-py2.py3-none-any.whl (294 kB)
Collecting pandas==1.0.3
  Downloading pandas-1.0.3.tar.gz (5.0 MB)
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /opt/hostedtoolcache/Python/3.9.0/x64/bin/python /opt/hostedtoolcache/Python/3.9.0/x64/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_yuvh7a4/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.29.13' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version>='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version>='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"''
       cwd: None
  Complete output (4485 lines):
  Ignoring numpy: markers 'python_version == "3.6" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.6" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.7" and platform_system == "AIX"' don't match your environment
  Collecting setuptools
    Downloading setuptools-50.3.2-py3-none-any.whl (785 kB)
  Collecting wheel
  

Azure по-прежнему несовместим с версией 3.9. Смотрите также на https://github.com/numpy/numpy/issues/17482