python: Replace deprecated setup.py with pypa/build+pip

[Rtoax]

When setuptools>58.2.0 setup.py installation is a deprecated method, a warning message will be prompted [1]:

$ sudo make install
....
running install
/usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66:
EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()

Use the recommended solution [2] to replace setup.py with pypa/build and pip-install to completely solve the problem. It is worth noting that under Debian, you need to add the --break-system-packages parameter to pip-uninstall, as shown in the following prompt:

Debian12:~$ sudo pip3 uninstall bcc
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at the
risk of breaking your Python installation or OS, by passing
--break-system-packages.
hint: See PEP 668 for the detailed specification.

# Successfully uninstalled
Debian12:~$ sudo pip3 uninstall bcc --break-system-packages

Fix: #4586 [1]
Link: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html [2]

[Rtoax]

Rebase to master (commit 5043759)

[Rtoax]

rebase to master.

[yonghong-song]

Applying this patch on top of current master, I used the following script to build bcc:

$ cat build_bcc.sh
cd bcc/build
cmake3 ..  -DPYTHON_CMD=python3.12 -DCMAKE_INSTALL_PREFIX=/usr
make -j && sudo make install

And I see the following error:

[ 35%] Generating ext4slower.8.gz                                                                                                    
/usr/bin/python3.12: No module named build                                                                                           
[ 36%] Building CXX object src/cc/CMakeFiles/bpf-shared.dir/common.cc.o                                                              
[ 36%] Generating ext4dist.8.gz                                                                                                      
[ 36%] Building CXX object src/cc/CMakeFiles/bpf-shared.dir/usdt/usdt_args.cc.o                                                      
[ 36%] Building CXX object src/cc/CMakeFiles/bpf-static.dir/usdt/usdt_args.cc.o                                                      
make[2]: *** [src/python/CMakeFiles/bcc_py_python3.12.dir/build.make:84: src/python/bcc-python3.12/dist/bcc-0.34.0+7739f369-py3-none-
any.whl] Error 1                                                                                                                     
make[1]: *** [CMakeFiles/Makefile2:1128: src/python/CMakeFiles/bcc_py_python3.12.dir/all] Error 2                                    
make[1]: *** Waiting for unfinished jobs....                                                                                         
[ 37%] Generating filelife.8.gz

With out patch, the build is successful. Could you take a look?

[Rtoax]

Applying this patch on top of current master, I used the following script to build bcc:

Thanks for you reply

[ 35%] Generating ext4slower.8.gz
/usr/bin/python3.12: No module named build

On my fedora42 python3.13, i just dnf install python3-build, it's works fine, would you please try?(and i try debian 12 python3.11.2 too, sudo apt install python3-build)

[Rtoax]

Rebase to master.

compile log of this patch

cmake \
	-DCMAKE_INSTALL_PREFIX=/usr \
	-DENABLE_LLVM_SHARED=1 \
	-DCMAKE_BUILD_TYPE=Debug \
	..
....
creating build/bdist.linux-x86_64/wheel/bcc-0.34.0+07ed4752.dist-info/WHEEL
creating '/home/rongtao/Git/bcc/build/src/python/bcc-python3.13/dist/.tmp-8nzx18qf/bcc-0.34.0+07ed4752-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'bcc/__init__.py'
adding 'bcc/containers.py'
adding 'bcc/disassembler.py'
adding 'bcc/libbcc.py'
adding 'bcc/perf.py'
adding 'bcc/syscall.py'
adding 'bcc/table.py'
adding 'bcc/tcp.py'
adding 'bcc/usdt.py'
adding 'bcc/utils.py'
adding 'bcc/version.py'
adding 'bcc-0.34.0+07ed4752.dist-info/METADATA'
adding 'bcc-0.34.0+07ed4752.dist-info/WHEEL'
adding 'bcc-0.34.0+07ed4752.dist-info/top_level.txt'
adding 'bcc-0.34.0+07ed4752.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built bcc-0.34.0+07ed4752-py3-none-any.whl
....

sudo make install
....
Processing /home/rongtao/Git/bcc/build/src/python/bcc-python3.13/dist/bcc-0.34.0+07ed4752-py3-none-any.whl
Installing collected packages: bcc
Successfully installed bcc-0.34.0+c97a9ce1


sudo make uninstall
....
-- Uninstall python-bcc for python3.13.
Found existing installation: bcc 0.34.0+c97a9ce1
Uninstalling bcc-0.34.0+c97a9ce1:
  Successfully uninstalled bcc-0.34.0+c97a9ce1

[yonghong-song]

Somehow, it still does not work for me. Using python3.12 (my command line cmake3 .. -DPYTHON_CMD=python3.9 -DCMAKE_INSTALL_PREFIX=/usr), I got the above failure:

...
[ 40%] Generating javacalls.8.gz                                                                                                 
/usr/bin/python3.12: No module named build                                                                                       
make[2]: *** [src/python/CMakeFiles/bcc_py_python3.12.dir/build.make:84: src/python/bcc-python3.12/dist/bcc-0.34.0+2cadbaed-py3-n
one-any.whl] Error 1                                                                                                             
make[1]: *** [CMakeFiles/Makefile2:1128: src/python/CMakeFiles/bcc_py_python3.12.dir/all] Error 2                                
make[1]: *** Waiting for unfinished jobs....                                                                                     
[ 40%] Linking CXX shared library libdebuginfo_test_lib.so
...

Using python3.9, I got further, like below

[ 69%] Built target debuginfo_test_lib
* Creating venv isolated environment...
[ 70%] Generating archive.zip
[ 71%] Generating debuginfo.so
  adding: libdebuginfo_test_lib.so (stored 0%)
  adding: zip_subdir/file.txt (stored 0%)
[ 71%] Built target debuginfo
[ 71%] Built target zip_archive
[ 71%] Generating with_gnu_debuglink.so
[ 72%] Generating debuginfo.so.xz
[ 72%] Built target with_gnu_debuglink
[ 72%] Built target debuginfo_xz 
[ 72%] Generating with_gnu_debugdata.so
[ 72%] Built target with_gnu_debugdata
[ 72%] Linking CXX static library libbcc-loader-static.a
[ 72%] Built target bcc-loader-static
* Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
[ 72%] Built target api-objects
[ 72%] Linking CXX static library libapi-static.a
[ 72%] Built target api-static
[ 72%] Linking CXX static library libbcc_bpf.a
[ 72%] Linking CXX shared library libbcc_bpf.so
[ 72%] Built target bpf-static
[ 73%] Building C object introspection/CMakeFiles/bps.dir/bps.c.o
...
[ 73%] Built target usdt-static                                                                                                  
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnection
Error('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f93bef45be0>: Failed to establish a new connection: [Errno -2
] Name or service not known')': /simple/wheel/                                                                                   
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnection
Error('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f93bef45df0>: Failed to establish a new connection: [Errno -2
] Name or service not known')': /simple/wheel/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnection
Error('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f93bef45fa0>: Failed to establish a new connection: [Errno -2
] Name or service not known')': /simple/wheel/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnection
Error('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f93bef6d190>: Failed to establish a new connection: [Errno -2
] Name or service not known')': /simple/wheel/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnection
Error('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f93bef6d340>: Failed to establish a new connection: [Errno -2
] Name or service not known')': /simple/wheel/
ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
ERROR: No matching distribution found for wheel

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/build/__main__.py", line 376, in main
    built = build_call(
  File "/usr/lib/python3.9/site-packages/build/__main__.py", line 207, in build_package
    out = _build(isolation, builder, outdir, distribution, config_settings, skip_dependency_check)
  File "/usr/lib/python3.9/site-packages/build/__main__.py", line 144, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/usr/lib/python3.9/site-packages/build/__main__.py", line 112, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/usr/lib/python3.9/site-packages/build/env.py", line 214, in install
    _subprocess(cmd)
  File "/usr/lib/python3.9/site-packages/build/env.py", line 79, in _subprocess
    raise e
  File "/usr/lib/python3.9/site-packages/build/env.py", line 76, in _subprocess
    subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-hhafvnmn/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-war
n-script-location', '-r', '/tmp/build-reqs-kh40gkoh.txt']' returned non-zero exit status 1.

ERROR Command '['/tmp/build-env-hhafvnmn/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r',
 '/tmp/build-reqs-kh40gkoh.txt']' returned non-zero exit status 1.
make[2]: *** [src/python/CMakeFiles/bcc_py_python3.9.dir/build.make:84: src/python/bcc-python3.9/dist/bcc-0.34.0+2cadbaed-py3-non
e-any.whl] Error 1
make[1]: *** [CMakeFiles/Makefile2:1128: src/python/CMakeFiles/bcc_py_python3.9.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 73%] Built target clang_frontend-objects
make: *** [Makefile:146: all] Error 2

[Rtoax]

ERROR Command '['/tmp/build-env-hhafvnmn/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r',
'/tmp/build-reqs-kh40gkoh.txt']' returned non-zero exit status 1.
make[2]: *** [src/python/CMakeFiles/bcc_py_python3.9.dir/build.make:84: src/python/bcc-python3.9/dist/bcc-0.34.0+2cadbaed-py3-non
e-any.whl] Error 1

@yonghong-song Thanks for your reply, I think some dependencies may not be installed. I checked my local compilation script build/./src/python/CMakeFiles/bcc_py_python3.dir/build.make and found that this line of error is when making a whl package.

cd /home/rongtao/Git/bcc/build/src/python/bcc-python3 && python3 -m build --wheel /home/rongtao/Git/bcc/build/src/python/bcc-python3

Can you confirm whether the wheel package of python3.9 is installed in your environment?

# python3.9 -m pip install build wheel  #maybe need more, i'm not sure

[Rtoax]

Rebase to master.

[Rtoax]

rebase to master