Date: 23-10-20  Time: 04:46 AM

Author Topic: Python and wheels continuing story  (Read 2213 times)

0 Members and 1 Guest are viewing this topic.

Michael

  • Administrator
  • Hero Member
  • *****
  • Posts: 1267
  • Karma: +0/-0
Re: Python and wheels continuing story
« Reply #10 on: July 04, 2020, 10:08:11 AM »
Do you have a additional test machine? Or create a wpar that "copies" your /opt (rather than a namefs mount).
On that wpar - install my package of Python. I have packaged it in a way that it should co-exist with the AIX Toolbox version - just need to prefix your PATH with /opt/bin.
I am surprised about pip "breaking" as it is pure-Python.
It does not find "PackageFinder", which it seems is an internal package of setuptools and perhaps of pip internal code:
root@x070:[/opt/lib]find . -name \*.py | xargs grep -i packagefind
./python3.7/site-packages/distlib/locators.py:    as pip's PackageFinder, which works in an analogous fashion.
./python3.7/site-packages/pip/_internal/build_env.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/build_env.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:needing download / PackageFinder capability don't unnecessarily import the
./python3.7/site-packages/pip/_internal/cli/req_command.py:PackageFinder machinery and all its vendored dependencies, etc.
./python3.7/site-packages/pip/_internal/cli/req_command.py:from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:        finder,                   # type: PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:        finder,                              # type: PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:        finder,           # type: PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:        # type: (PackageFinder) -> None
./python3.7/site-packages/pip/_internal/cli/req_command.py:        # type: (...) -> PackageFinder
./python3.7/site-packages/pip/_internal/cli/req_command.py:        return PackageFinder.create(
./python3.7/site-packages/pip/_internal/commands/list.py:from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/commands/list.py:        return PackageFinder.create(
./python3.7/site-packages/pip/_internal/distributions/base.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/distributions/base.py:        # type: (PackageFinder, bool) -> None
./python3.7/site-packages/pip/_internal/distributions/installed.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/distributions/installed.py:        # type: (PackageFinder, bool) -> None
./python3.7/site-packages/pip/_internal/distributions/sdist.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/distributions/sdist.py:        # type: (PackageFinder, bool) -> None
./python3.7/site-packages/pip/_internal/distributions/sdist.py:        # type: (PackageFinder) -> None
./python3.7/site-packages/pip/_internal/distributions/wheel.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/distributions/wheel.py:        # type: (PackageFinder, bool) -> None
./python3.7/site-packages/pip/_internal/index/package_finder.py:__all__ = ['FormatControl', 'BestCandidateResult', 'PackageFinder']
./python3.7/site-packages/pip/_internal/index/package_finder.py:    """A collection of candidates, returned by `PackageFinder.find_best_candidate`.
./python3.7/site-packages/pip/_internal/index/package_finder.py:class PackageFinder(object):
./python3.7/site-packages/pip/_internal/index/package_finder.py:        # type: (...) -> PackageFinder
./python3.7/site-packages/pip/_internal/index/package_finder.py:        """Create a PackageFinder.
./python3.7/site-packages/pip/_internal/network/session.py:        # configured on this PackageFinder instance.
./python3.7/site-packages/pip/_internal/operations/prepare.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/operations/prepare.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/operations/prepare.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/req/req_file.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/req/req_file.py:    finder=None,  # type: Optional[PackageFinder]
./python3.7/site-packages/pip/_internal/req/req_file.py:    :param finder:      Instance of pip.index.PackageFinder.
./python3.7/site-packages/pip/_internal/req/req_file.py:    finder=None,  # type: Optional[PackageFinder]
./python3.7/site-packages/pip/_internal/req/req_file.py:    finder=None,  # type: Optional[PackageFinder]
./python3.7/site-packages/pip/_internal/req/req_file.py:    # type: (Optional[PackageFinder]) -> LineParser
./python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/requirements.py:        # We can safely always allow prereleases here since PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/requirements.py:        # We can safely always allow prereleases here since PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py:    from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py:        finder,  # type: PackageFinder
./python3.7/site-packages/pip/_internal/self_outdated_check.py:from pip._internal.index.package_finder import PackageFinder
./python3.7/site-packages/pip/_internal/self_outdated_check.py:            # Lets use PackageFinder to see what the latest pip version is
./python3.7/site-packages/pip/_internal/self_outdated_check.py:            finder = PackageFinder.create(
./python3.7/site-packages/pip/_vendor/distlib/locators.py:    as pip's PackageFinder, which works in an analogous fashion.
./python3.7/site-packages/setuptools/__init__.py:class PackageFinder:
./python3.7/site-packages/setuptools/__init__.py:class PEP420PackageFinder(PackageFinder):
./python3.7/site-packages/setuptools/__init__.py:find_packages = PackageFinder.find
./python3.7/site-packages/setuptools/__init__.py:  find_namespace_packages = PEP420PackageFinder.find

Again, using the WPAR approach - you could try to force re-install python3 from the AIX Toolbox (which will downgrade your installed pip, and then try "pip3 download pip", and then use the downloaded pip to upgrade your system.
Or, try downloading pip, and from the unpacked directory, use -
Code: [Select]
python(3) setup.py install

Michael

  • Administrator
  • Hero Member
  • *****
  • Posts: 1267
  • Karma: +0/-0
Re: Python and wheels continuing story
« Reply #11 on: July 04, 2020, 03:52:49 PM »
Ran into an issue with pandas-1.0.5 and cython integration.
Testing 0.24.2 and 0.25.3 to see if the issue was also there - or more recent.

raghugr

  • Jr. Member
  • **
  • Posts: 7
  • Karma: +0/-0
Re: Python and wheels continuing story
« Reply #12 on: July 05, 2020, 06:16:01 PM »
Thank you very much Michael for supporting me through this. It was so encouraging to get a backing :).

I was able to install the pandas finally after at least 4-5 days. I believe these are the AIX toolbox packages I had sorted, may be it will help you to setup one for aixtools as well,

libstdc++-8.3.0-2.ppc
libstdc++-devel-8.3.0-2.ppc
libgomp-8.3.0-2.ppc
libgfortran-8.3.0-2.ppc
gcc-8.3.0-2.ppc
gcc-gfortran-8.3.0-2.ppc
libgcc-8.3.0-2.ppc
gcc-cpp-8.3.0-2.ppc
gcc-c++-8.3.0-2.ppc
openblas-0.3.6-1.ppc
lapack-3.8.0-1.ppc
blas-3.8.0-1.ppc
lapack-devel-3.8.0-1.ppc
python3-devel-3.7.6-1.ppc

Thank you again!

Michael

  • Administrator
  • Hero Member
  • *****
  • Posts: 1267
  • Karma: +0/-0
Re: Python and wheels continuing story
« Reply #13 on: July 06, 2020, 06:38:11 AM »
Thank you for the feedback.
I am nearly there.
Sadly, xlc does not support a ALL of syntax (that is quite common these days, C++11 standard).
As I went through my records I was surprised to see the last time I built pandas was Oct. 2016 (v0.19.0).
I am glad to hear you have it installed. Is it "portable" to another system? I ask, because one of the reasons I started "aixtools" was because I did not like (read did not condone) needing to run "make install" aka "python setup.py install" to get something extra installed - and then there is version management and/or uninstall.
FYI: I am going to continue looking into this. I'll post when I have a real "wheel" based solution.

raghugr

  • Jr. Member
  • **
  • Posts: 7
  • Karma: +0/-0
Re: Python and wheels continuing story
« Reply #14 on: July 06, 2020, 10:58:47 PM »
Thanks Michael. I am really not sure about the porting stuff since I am still new to this AIX. May be if there are few steps then I can try as long as I have some root access haha