Using scikit-learn/numpy on Microsoft Azure WebApp

When you want to use scikit-learn on Azure WebApp there is a high chance that you will see this kind of error:

StdErr:  
2017-03-04 01:07:31.809521: Unhandled exception in wfastcgi.py: Traceback (most recent call last):  
  File "D:\Python34\Scripts\wfastcgi.py", line 711, in main
    env, handler = read_wsgi_handler(response.physical_path)
  File "D:\Python34\Scripts\wfastcgi.py", line 568, in read_wsgi_handler
    return env, get_wsgi_handler(handler_name)
  File "D:\Python34\Scripts\wfastcgi.py", line 541, in get_wsgi_handler
    handler = handler()
  File ".\ptvs_virtualenv_proxy.py", line 102, in get_virtualenv_handler
    handler = get_wsgi_handler(os.getenv('WSGI_ALT_VIRTUALENV_HANDLER'))
  File ".\ptvs_virtualenv_proxy.py", line 89, in get_wsgi_handler
    raise ValueError('"%s" could not be imported%s' % (handler_name, last_tb))
ValueError: "web.api.app" could not be imported: Traceback (most recent call last):  
  File ".\ptvs_virtualenv_proxy.py", line 73, in get_wsgi_handler
    handler = __import__(module_name, fromlist=[name_list[0][0]])
  File ".\web\api.py", line 7, in <module>
    from analysis.textclassification.ClassifierFactory import ClassifierFactory
  File ".\analysis\textclassification\ClassifierFactory.py", line 2, in <module>
    from sklearn.ensemble import RandomForestClassifier
  File "D:\home\site\wwwroot\env\Lib\site-packages\sklearn\__init__.py", line 56, in <module>
    from . import __check_build
ImportError: cannot import name '__check_build'  

This exception could make you think that there is an issue with scikit-learn package installation, however it is actually hiding the real problem. Try to comment the line __check_build from script D:\home\site\wwwroot\env\Lib\site-packages\sklearn\__init__.py and you will see the next exception:

StdErr:  
2017-03-04 01:28:20.824329: Unhandled exception in wfastcgi.py: Traceback (most recent call last):  
  File "D:\Python34\Scripts\wfastcgi.py", line 711, in main
    env, handler = read_wsgi_handler(response.physical_path)
  File "D:\Python34\Scripts\wfastcgi.py", line 568, in read_wsgi_handler
    return env, get_wsgi_handler(handler_name)
  File "D:\Python34\Scripts\wfastcgi.py", line 541, in get_wsgi_handler
    handler = handler()
  File ".\ptvs_virtualenv_proxy.py", line 102, in get_virtualenv_handler
    handler = get_wsgi_handler(os.getenv('WSGI_ALT_VIRTUALENV_HANDLER'))
  File ".\ptvs_virtualenv_proxy.py", line 89, in get_wsgi_handler
    raise ValueError('"%s" could not be imported%s' % (handler_name, last_tb))
ValueError: "web.api.app" could not be imported: Traceback (most recent call last):  
  File ".\ptvs_virtualenv_proxy.py", line 73, in get_wsgi_handler
    handler = __import__(module_name, fromlist=[name_list[0][0]])
  File ".\web\api.py", line 7, in <module>
    from analysis.textclassification.ClassifierFactory import ClassifierFactory
  File ".\analysis\textclassification\ClassifierFactory.py", line 2, in <module>
    from sklearn.ensemble import RandomForestClassifier
  File "D:\home\site\wwwroot\env\Lib\site-packages\sklearn\ensemble\__init__.py", line 6, in <module>
    from .base import BaseEnsemble
  File "D:\home\site\wwwroot\env\Lib\site-packages\sklearn\ensemble\base.py", line 11, in <module>
    from ..base import clone
  File "D:\home\site\wwwroot\env\Lib\site-packages\sklearn\base.py", line 10, in <module>
    from scipy import sparse
  File "D:\home\site\wwwroot\env\Lib\site-packages\scipy\__init__.py", line 61, in <module>
    from numpy._distributor_init import NUMPY_MKL  # requires numpy+mkl
ImportError: cannot import name 'NUMPY_MKL'  

This one tells a little bit more. Apparently there is an issue with numpy package, which can cause problems when installed on Windows machines. Here is my solution to make scikit-learn/numpy work correctly.

  1. Open Kudu. It's your best friend for debugging issues with Python packages. All of the below command have to be copied into Kudu CMD console.
  2. Make your you are using the correct version of Python in your virtual environment. In my case it's 3.4. To check this type env\scripts\python -m pip -V in the root folder of your webpage.
D:\home\site\wwwroot>env\scripts\python -m pip -V  
pip 9.0.1 from D:\home\site\wwwroot\env\lib\site-packages (python 3.4)

D:\home\site\wwwroot>  
  1. Upgrade pip with command env\scripts\python -m pip install --upgrade pip
  2. Download numply+MKL wheel from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy. Make a change in the filename from cp34m to none. Next upload this file to your website. I have copied it into wheels directory and the final path to my wheel is wheels\numpy-1.12.0+mkl-cp34-none-win32.whl
  3. Install numpy from this wheel. If you haven't installed it previously just type env\scripts\python -m pip install wheels\numpy-1.12.0+mkl-cp34-none-win32.whl. In my case I had to reinstall it, so it goes like this env\scripts\python -m pip install --upgrade --force-reinstall wheels\numpy-1.12.0+mkl-cp34-none-win32.whl.
  4. Run your application. In my case the errors with scikit-learn/numpy dissappered. Happy machine learning on Azure!

Share this post

comments powered by Disqus