I am trying to get started running python inside of Sql Server 2017 with Python Machine Learning Services.
Out of the box it works and I can run simple scripts like listing the installed packages:
EXEC sp_execute_external_script
@language = N'Python',
@script = N'import pkg_resources
print([p.project_name for p in pkg_resources.working_set])'
When I want to add a new package I follow the Microsoft documentation for 2017: https://learn.microsoft.com/en-us/sql/machine-learning/package-management/install-python-packages-standard-tools?view=sql-server-2017 which says to basically just pip install the new package. So I go to the sql server python install:
C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES
and run:
scripts\pip.exe install pyarrow
This installs an upgraded numpy version as a dependency and when I then try to call even simple python scripts like above I get the following error:
Msg 39012, Level 16, State 1, Line 0
Unable to communicate with the runtime for 'Python' script. Please check the requirements of 'Python' runtime.
Nothing will run while numpy is in this newer version. I can fix the issue by repairing the Sql Server install from the installation media, but that gets me back to my initial setup without the required pyarrow package.
After trying to troubleshoot (incl. installing specific versions of pandas and numpy) I now can't get back to a working state with repairs - will try uninstall and reinstalling the feature.
I can see that the procedure has changed for installing python packages in Sql Server 2019 (https://learn.microsoft.com/en-us/sql/machine-learning/package-management/install-additional-python-packages-on-sql-server?view=sql-server-ver15) so there might be some flaws in 2017 but I can't find it documented anywhere.
I run the latest Sql Server 2017 (cu21) developer edition.
Some suspects:
- I'm not too familiar but it says in multiple places that Sql Server 2017 Machine Learning services comes with Conda and I'm not sure if Conda and pip are compatible?
- The error suggests a dependency from microsofts revoscalepy on pandas and again from pandas on numpy.
Current full error:
Msg 39012, Level 16, State 1, Line 0
Unable to communicate with the runtime for 'Python' script. Please check the requirements of 'Python' runtime.
STDERR message(s) from external script:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\__init__.py", line 99, in <module>
from .RxSerializable import RxMissingValues
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 10, in <module>
from pandas import DataFrame
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\__init__.py", line 39, in <module>
from pandas.core.api import *
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\api.py", line 10, in <module>
from pandas.core.groupby import Grouper
STDERR message(s) from external script:
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
from pandas.core.groupby.groupby import GroupBy # noqa: F401
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\groupby\groupby.py", line 25, in <module>
from pandas.util._validators import validate_kwargs
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\util\_validators.py", line 7, in <module>
from pandas.core.dtypes.common import is_bool
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\dtypes\common.py", line 10, in <module>
from pandas.core.dtypes.dtypes import (
STDERR message(s) from external script:
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\dtypes\dtypes.py", line 16, in <module>
from .inference import is_list_like
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\pandas\core\dtypes\inference.py", line 9, in <module>
from pandas.compat import (
ImportError: cannot import name 'Set'
Completion time: 2020-08-25T09:43:23.9780283+02:00