Allow overriding the python install on windows

* Also document the new requirement for os.add_dll_directory as of python 3.8

docs/python_api/examples/renderdoc_intro.rst

@@ -14,7 +14,7 @@ For this section we assume you have built a copy of RenderDoc and have the modul
 
 .. note::
 
-  You must use exactly the same version of python to load the module as was used to build it. On Windows the version of python that comes with the repository and is used by default is python 3.6.
+  You must use exactly the same version of python to load the module as was used to build it. On Windows the version of python that comes with the repository and is used by default is python 3.6. This can be overridden at build time by setting the environment variable `RENDERDOC_PYTHON_PREFIX32` or `RENDERDOC_PYTHON_PREFIX64` to the path of a compatible python install. NOTE that the embedded zip distribution doesn't come with python38.lib which is necessary, so you must regenerate it or get it from the installed python. The installed python folder doesn't come with a python38.zip standard library bundle which you'd need to create or get from the embedded zip distribution.
 
 Once you have the module, either place the module within your python's default library search path, or else insert the location of the python module into the path in your script. You can either set the ``PYTHONPATH`` environment variable or do it at the start of your script:
 
@@ -25,14 +25,16 @@ Once you have the module, either place the module within your python's default l
 
     sys.path.append('/path/to/renderdoc/module')
 
-Additionally, the renderdoc python module needs to be able to load the main renderdoc library - the module library it self just contains stubs and python wrappers for the C++ interfaces. You can either place the renderdoc library in the system library paths, or solve it in a platform specific way. For example on windows you can either place ``renderdoc.dll`` in the same directory as the python module, or append to ``PATH``:
+Additionally, the renderdoc python module needs to be able to load the main renderdoc library - the module library it self just contains stubs and python wrappers for the C++ interfaces. You can either place the renderdoc library in the system library paths, or solve it in a platform specific way. For example on windows you can either place ``renderdoc.dll`` in the same directory as the python module, or append to ``PATH``. On Python 3.8 and above ``PATH`` is no longer searched by default so you need to explicitly add the DLL folder:
 
 .. highlight:: python
 .. code:: python
 
-    import os
+    import os, sys
 
     os.environ["PATH"] += os.pathsep + os.path.abspath('/path/to/renderdoc/native/library')
+    if sys.platform == 'win32' and sys.version_info[1] >= 8:
+        os.add_dll_directory("/path/to/renderdoc/native/library")
 
 On linux you'd perform a similar modification to ``LD_LIBRARY_PATH``.
 

docs/python_api/index.rst

@@ -8,9 +8,9 @@ RenderDoc exposes APIs to python at two different levels:
 
 Within RenderDoc - when either running scripts on the command line, or via the :doc:`../window/python_shell` - both modules are pre-imported and available automatically.
 
-It is also possible to build the ``renderdoc`` module standalone which can be loaded into python and used for scripting directly without the UI program. Due to the inherent difficulty of distributing C python modules this isn't included by default in distributed builds at the time of writing, but is generated by default in source builds - ``renderdoc.pyd`` on windows or ``renderdoc.so`` elsewhere.
+It is also possible to build the ``renderdoc`` module standalone which can be loaded into python and used for scripting directly without the UI program. Due to the inherent difficulty of distributing C python modules this isn't included by default in distributed builds at the time of writing, but is generated by default in source builds - ``renderdoc.pyd`` on windows or ``renderdoc.so`` elsewhere. Use of this module is strictly a convenience and is not supported.
 
-You must use exactly the same version of python to load the module as was used to build it. On Windows the version of python that comes with the repository and is used by default is python 3.6.
+You must use exactly the same version of python to load the module as was used to build it. On Windows the version of python that comes with the repository and is used by default is python 3.6. This can be overridden at build time by setting the environment variable `RENDERDOC_PYTHON_PREFIX32` or `RENDERDOC_PYTHON_PREFIX64` to the path of a compatible python install. NOTE that the embedded zip distribution doesn't come with python38.lib which is necessary, so you must regenerate it or get it from the installed python. The installed python folder doesn't come with a python38.zip standard library bundle which you'd need to create or get from the embedded zip distribution.
 
 .. note::
 

qrenderdoc/Code/pyrenderdoc/pyrenderdoc_module.vcxproj

@@ -26,6 +26,7 @@
   </PropertyGroup>
   <Import Project="$(SolutionDir)\util\WindowsSDKTarget.props" />
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
+  <Import Project="$(SolutionDir)\qrenderdoc\Code\pyrenderdoc\python.props" />
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
     <ConfigurationType>DynamicLibrary</ConfigurationType>
     <UseDebugLibraries>false</UseDebugLibraries>
@@ -95,13 +96,12 @@
       <TreatWarningAsError>true</TreatWarningAsError>
       <MultiProcessorCompilation>true</MultiProcessorCompilation>
       <AdditionalOptions>/wd4100 /wd4512</AdditionalOptions>
-      <AdditionalIncludeDirectories>$(ProjectDir)..\..\;..\..\3rdparty\python\include;$(SolutionDir)\renderdoc\api\replay</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>$(ProjectDir)..\..\;$(PythonIncludeDir);$(SolutionDir)\renderdoc\api\replay</AdditionalIncludeDirectories>
     </ClCompile>
     <Link>
       <SubSystem>Windows</SubSystem>
       <GenerateDebugInformation>true</GenerateDebugInformation>
-      <AdditionalLibraryDirectories>..\..\3rdparty\python\$(Platform);%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
-      <AdditionalDependencies>python36.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>$(PythonImportLib);%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
     <ProjectReference>
       <UseLibraryDependencyInputs>true</UseLibraryDependencyInputs>

qrenderdoc/Code/pyrenderdoc/python.props

@@ -0,0 +1,81 @@
+<?xml version="1.0" encoding="utf-8"?>
+<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
+
+<PropertyGroup>
+  <PythonBase>$(SolutionDir)\qrenderdoc\3rdparty\python</PythonBase>
+
+  <CustomPythonUsed>0</CustomPythonUsed>
+
+  <!-- output file of python36.dll, python36.zip, python36.lib etc -->
+	<PythonMajorMinor>36</PythonMajorMinor>
+
+	<PythonIncludeDir>$(PythonBase)\include</PythonIncludeDir>
+	<PythonImportLib>$(PythonBase)\$(Platform)\python$(PythonMajorMinor).lib</PythonImportLib>
+	<PythonStandardLibraryZip>$(PythonBase)\python$(PythonMajorMinor).zip</PythonStandardLibraryZip>
+	<PythonDLLsDir>$(PythonBase)\$(Platform)</PythonDLLsDir>
+	<PythonInterpDLL>$(PythonBase)\$(Platform)\python$(PythonMajorMinor).dll</PythonInterpDLL>
+</PropertyGroup>
+
+<!-- either we have the 'embeddable zip' which has everything in the same folder,
+     or we have an install which has things under libs/, include/ and DLLs/
+     We also just naively check every PythonMajorMinor.
+     NOTE: pythonXY.lib isn't included in the embed zip currently, so you'd have
+     to regenerate it from the pythonXY.dll. You also need to include the include/
+     folder manually from the python source distribution.
+     ALSO: pythonXY.zip isn't included in the installed distribution (it instead
+     uses the Lib/ folder with all the loose uncompiled library source).
+     It will need to be generated manually or obtained from the embeddable zip.
+     As a result we use the existance of the .lib, .h and .zip as a key to
+     ensure we only use -->
+
+<!-- MSBuild doesn't implement a simple loop so just do this by hand by
+     taking advantage of MSBuild evaluating these in-order.
+     First we set the number we're testing, then see if we find the files. If
+     we find the files we set CustomPythonUsed which prevents all subsequent checks
+     and then at the end we'll pick it up to set all the derived properties.
+     To add a new version to check for, copy paste both lines and update the number.
+     -->
+
+<!-- first define the override prefix we're searching against -->
+<PropertyGroup>
+	<PythonOverride Condition="'$(Platform)'=='Win32'">$(RENDERDOC_PYTHON_PREFIX32)</PythonOverride>
+	<PythonOverride Condition="'$(Platform)'=='x64'">$(RENDERDOC_PYTHON_PREFIX64)</PythonOverride>
+</PropertyGroup>
+
+<PropertyGroup><PythonMajorMinorTest>38</PythonMajorMinorTest></PropertyGroup>
+<PropertyGroup Condition="'$(CustomPythonUsed)'=='0' AND Exists('$(PythonOverride)\include\Python.h') AND Exists('$(PythonOverride)\python$(PythonMajorMinorTest).zip') AND (Exists('$(PythonOverride)\python$(PythonMajorMinorTest).lib') OR Exists('$(PythonOverride)\libs\python$(PythonMajorMinorTest).lib'))"><CustomPythonUsed>$(PythonMajorMinorTest)</CustomPythonUsed></PropertyGroup>
+
+<PropertyGroup><PythonMajorMinorTest>37</PythonMajorMinorTest></PropertyGroup>
+<PropertyGroup Condition="'$(CustomPythonUsed)'=='0' AND Exists('$(PythonOverride)\include\Python.h') AND Exists('$(PythonOverride)\python$(PythonMajorMinorTest).zip') AND (Exists('$(PythonOverride)\python$(PythonMajorMinorTest).lib') OR Exists('$(PythonOverride)\libs\python$(PythonMajorMinorTest).lib'))"><CustomPythonUsed>$(PythonMajorMinorTest)</CustomPythonUsed></PropertyGroup>
+
+<PropertyGroup><PythonMajorMinorTest>36</PythonMajorMinorTest></PropertyGroup>
+<PropertyGroup Condition="'$(CustomPythonUsed)'=='0' AND Exists('$(PythonOverride)\include\Python.h') AND Exists('$(PythonOverride)\python$(PythonMajorMinorTest).zip') AND (Exists('$(PythonOverride)\python$(PythonMajorMinorTest).lib') OR Exists('$(PythonOverride)\libs\python$(PythonMajorMinorTest).lib'))"><CustomPythonUsed>$(PythonMajorMinorTest)</CustomPythonUsed></PropertyGroup>
+
+<PropertyGroup><PythonMajorMinorTest>35</PythonMajorMinorTest></PropertyGroup>
+<PropertyGroup Condition="'$(CustomPythonUsed)'=='0' AND Exists('$(PythonOverride)\include\Python.h') AND Exists('$(PythonOverride)\python$(PythonMajorMinorTest).zip') AND (Exists('$(PythonOverride)\python$(PythonMajorMinorTest).lib') OR Exists('$(PythonOverride)\libs\python$(PythonMajorMinorTest).lib'))"><CustomPythonUsed>$(PythonMajorMinorTest)</CustomPythonUsed></PropertyGroup>
+
+<PropertyGroup><PythonMajorMinorTest>34</PythonMajorMinorTest></PropertyGroup>
+<PropertyGroup Condition="'$(CustomPythonUsed)'=='0' AND Exists('$(PythonOverride)\include\Python.h') AND Exists('$(PythonOverride)\python$(PythonMajorMinorTest).zip') AND (Exists('$(PythonOverride)\python$(PythonMajorMinorTest).lib') OR Exists('$(PythonOverride)\libs\python$(PythonMajorMinorTest).lib'))"><CustomPythonUsed>$(PythonMajorMinorTest)</CustomPythonUsed></PropertyGroup>
+
+<PropertyGroup Condition="'$(CustomPythonUsed)'!='0'">
+  <PythonBase>$(PythonOverride)</PythonBase>
+
+	<PythonMajorMinor>$(CustomPythonUsed)</PythonMajorMinor>
+
+  <!-- these are always in the root, regardless of the installation type -->
+	<PythonIncludeDir>$(PythonBase)\include</PythonIncludeDir>
+	<PythonStandardLibraryZip>$(PythonBase)\python$(PythonMajorMinor).zip</PythonStandardLibraryZip>
+	<PythonInterpDLL>$(PythonBase)\python$(PythonMajorMinor).dll</PythonInterpDLL>
+
+  <!-- for embeddable zip, find these in the root. Otherwise find these in subfolders -->
+	<PythonDLLsDir Condition="Exists('$(PythonOverride)\_ctypes.pyd')">$(PythonBase)</PythonDLLsDir>
+	<PythonDLLsDir Condition="Exists('$(PythonOverride)\DLLs\_ctypes.pyd')">$(PythonBase)\DLLs</PythonDLLsDir>
+	<PythonImportLib Condition="Exists('$(PythonOverride)\python$(PythonMajorMinor).lib')">$(PythonBase)\python$(PythonMajorMinor).lib</PythonImportLib>
+	<PythonImportLib Condition="Exists('$(PythonOverride)\libs\python$(PythonMajorMinor).lib')">$(PythonBase)\libs\python$(PythonMajorMinor).lib</PythonImportLib>
+</PropertyGroup>
+
+<Target Name="PythonOverridden" AfterTargets="Build" Condition="'$(CustomPythonUsed)'!='0'">
+	<Message Importance="high" Text="Built against python from $(PythonOverride)" />
+</Target>
+
+</Project>

qrenderdoc/Code/pyrenderdoc/qrenderdoc_module.vcxproj

@@ -26,6 +26,7 @@
   </PropertyGroup>
   <Import Project="$(SolutionDir)\util\WindowsSDKTarget.props" />
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
+  <Import Project="$(SolutionDir)\qrenderdoc\Code\pyrenderdoc\python.props" />
   <PropertyGroup Label="Configuration">
     <ConfigurationType>DynamicLibrary</ConfigurationType>
     <UseDebugLibraries>false</UseDebugLibraries>
@@ -75,15 +76,14 @@
       <TreatWarningAsError>true</TreatWarningAsError>
       <MultiProcessorCompilation>true</MultiProcessorCompilation>
       <AdditionalOptions>/wd4100 /wd4512 /wd4127</AdditionalOptions>
-      <AdditionalIncludeDirectories>$(ProjectDir)..\..\;..\..\3rdparty\python\include;$(SolutionDir)\renderdoc\api\replay</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>$(ProjectDir)..\..\;$(PythonIncludeDir);$(SolutionDir)\renderdoc\api\replay</AdditionalIncludeDirectories>
       <DisableSpecificWarnings>4714;%(DisableSpecificWarnings)</DisableSpecificWarnings>
     </ClCompile>
     <Link>
       <SubSystem>Windows</SubSystem>
       <GenerateDebugInformation>true</GenerateDebugInformation>
       <LinkTimeCodeGeneration>Default</LinkTimeCodeGeneration>
-      <AdditionalLibraryDirectories>..\..\3rdparty\python\$(Platform);%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
-      <AdditionalDependencies>python36.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <AdditionalDependencies>$(PythonImportLib);%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
     <ProjectReference>
       <UseLibraryDependencyInputs>true</UseLibraryDependencyInputs>

qrenderdoc/qrenderdoc_local.vcxproj

@@ -34,6 +34,7 @@
   </PropertyGroup>
   <Import Project="$(SolutionDir)\util\WindowsSDKTarget.props" />
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
+  <Import Project="$(SolutionDir)\qrenderdoc\Code\pyrenderdoc\python.props" />
   <PropertyGroup>
     <PlatformToolSet>v140</PlatformToolSet>
     <OutputDirectory>$(SolutionDir)$(Platform)\$(Configuration)\</OutputDirectory>
@@ -103,7 +104,7 @@
       <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
       <SuppressStartupBanner>true</SuppressStartupBanner>
       <TreatWChar_tAsBuiltInType>true</TreatWChar_tAsBuiltInType>
-      <AdditionalIncludeDirectories>$(ProjectDir);3rdparty;$(IntDir)generated\;$(SolutionDir)\renderdoc\api\replay;3rdparty\python\include;3rdparty\pyside\include\PySide2;3rdparty\pyside\include\shiboken2;$(QtIncludeDir);$(QtIncludeDir)\QtWidgets;$(QtIncludeDir)\QtGui;$(QtIncludeDir)\QtCore;$(QtIncludeDir)\QtSvg;$(QtIncludeDir)\QtNetwork;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>$(ProjectDir);3rdparty;$(IntDir)generated\;$(SolutionDir)\renderdoc\api\replay;$(PythonIncludeDir);3rdparty\pyside\include\PySide2;3rdparty\pyside\include\shiboken2;$(QtIncludeDir);$(QtIncludeDir)\QtWidgets;$(QtIncludeDir)\QtGui;$(QtIncludeDir)\QtCore;$(QtIncludeDir)\QtSvg;$(QtIncludeDir)\QtNetwork;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
       <AdditionalOptions>/Zc:strictStrings /Zc:throwingNew %(AdditionalOptions)</AdditionalOptions>
       <PreprocessorDefinitions>_WINDOWS;UNICODE;WIN32;WIN64;RENDERDOC_PLATFORM_WIN32;SCINTILLA_QT=1;MAKING_LIBRARY=1;SCI_LEXER=1;QT_NO_CAST_FROM_ASCII;QT_NO_CAST_TO_ASCII;QT_WIDGETS_LIB;QT_GUI_LIB;QT_CORE_LIB;QT_SVG_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <WarningLevel>Level4</WarningLevel>
@@ -116,8 +117,8 @@
     </ClCompile>
     <Link>
       <SubSystem>Windows</SubSystem>
-      <AdditionalDependencies>user32.lib;python36.lib;qtmain.lib;Qt5Widgets.lib;Qt5Gui.lib;Qt5Core.lib;Qt5Svg.lib;Qt5Network.lib;shell32.lib</AdditionalDependencies>
-      <AdditionalLibraryDirectories>3rdparty\pyside\$(Platform);3rdparty\python\$(Platform);$(QtLibDir);%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
+      <AdditionalDependencies>user32.lib;$(PythonImportLib);qtmain.lib;Qt5Widgets.lib;Qt5Gui.lib;Qt5Core.lib;Qt5Svg.lib;Qt5Network.lib;shell32.lib</AdditionalDependencies>
+      <AdditionalLibraryDirectories>3rdparty\pyside\$(Platform);$(QtLibDir);%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
       <DataExecutionPrevention>true</DataExecutionPrevention>
       <IgnoreImportLibrary>true</IgnoreImportLibrary>
       <GenerateDebugInformation>true</GenerateDebugInformation>
@@ -146,15 +147,15 @@
       <AdditionalDependencies>user32.lib;python36.lib;qtmaind.lib;Qt5Widgetsd.lib;Qt5Guid.lib;Qt5Cored.lib;Qt5Svgd.lib;Qt5Networkd.lib;shell32.lib</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
-  <ItemDefinitionGroup Condition="Exists('$(SolutionDir)\qrenderdoc\3rdparty\pyside\include\PySide2\pyside.h')">
+  <ItemDefinitionGroup Condition="Exists('$(SolutionDir)\qrenderdoc\3rdparty\pyside\include\PySide2\pyside.h') AND '$(CustomPythonUsed)'=='0'">
     <ClCompile>
       <PreprocessorDefinitions>PYSIDE2_ENABLED=1;%(PreprocessorDefinitions)</PreprocessorDefinitions>
     </ClCompile>
     <Link>
       <AdditionalDependencies>shiboken2.lib;%(AdditionalDependencies)</AdditionalDependencies>
     </Link>
   </ItemDefinitionGroup>
-  <ItemDefinitionGroup Condition="!Exists('$(SolutionDir)\qrenderdoc\3rdparty\pyside\include\PySide2\pyside.h')">
+  <ItemDefinitionGroup Condition="!Exists('$(SolutionDir)\qrenderdoc\3rdparty\pyside\include\PySide2\pyside.h') OR '$(CustomPythonUsed)'!='0'">
     <ClCompile>
       <PreprocessorDefinitions>PYSIDE2_ENABLED=0;%(PreprocessorDefinitions)</PreprocessorDefinitions>
     </ClCompile>
@@ -1702,18 +1703,18 @@ IF %ERRORLEVEL% NEQ 0 (echo ====================================================
     <None Include="Resources\README.md" />
   </ItemGroup>
   <ItemGroup>
-    <Content Include="3rdparty\python\python36.zip">
-      <Link>python36.zip</Link>
+    <Content Include="$(PythonStandardLibraryZip)">
+      <Link>python$(PythonMajorMinor).zip</Link>
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
       <Visible>false</Visible>
     </Content>
-    <Content Include="3rdparty\python\$(Platform)\_ctypes.pyd">
+    <Content Include="$(PythonDLLsDir)\_ctypes.pyd">
       <Link>_ctypes.pyd</Link>
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
       <Visible>false</Visible>
     </Content>
-    <Content Include="3rdparty\python\$(Platform)\python36.dll">
-      <Link>python36.dll</Link>
+    <Content Include="$(PythonInterpDLL)">
+      <Link>python$(PythonMajorMinor).dll</Link>
       <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
       <Visible>false</Visible>
     </Content>

util/test/run_tests.py

@@ -44,11 +44,15 @@
 
 if args.renderdoc is not None:
     if os.path.isfile(args.renderdoc):
-        os.environ["PATH"] += os.pathsep + os.path.abspath(os.path.dirname(args.renderdoc))
+        renderdoc_dirpath = os.path.abspath(os.path.dirname(args.renderdoc))
     elif os.path.isdir(args.renderdoc):
-        os.environ["PATH"] += os.pathsep + os.path.abspath(args.renderdoc)
+        renderdoc_dirpath = os.path.abspath(args.renderdoc)
     else:
         raise RuntimeError("'{}' is not a valid path to the renderdoc library".format(args.renderdoc))
+    os.environ["PATH"] += os.pathsep + renderdoc_dirpath
+    # Python 3.8 doesn't search PATH so add it to the DLL search path
+    if sys.platform == 'win32' and sys.version_info[1] >= 8:
+        os.add_dll_directory(renderdoc_dirpath)
 
 custom_pyrenderdoc = None