Pyinstaller3.6 issue in aix7.1

[jessding2503]

Hi,Our server is AIX 7.1 and aixtools.python3.6 and xlc are already installed.
According to this article(http://forums.rootvg.net/aixtools/pyinstaller-issue-with-python3-6-deployed-via-aixtools/msg2839/#msg2839) I installed pyinstaller successfully
but it have other issue, Please help to fix the issue.



root@aix7:/$lslpp -l|grep -i xlc
  xlC.adt.include           16.1.0.0  COMMITTED  C Set ++ Application
  xlC.aix61.rte             16.1.0.0  COMMITTED  IBM XL C++ Runtime for AIX 6.1
  xlC.cpp                    9.0.0.0  COMMITTED  C for AIX Preprocessor
  xlC.msg.en_US.cpp          9.0.0.0  COMMITTED  C for AIX Preprocessor
  xlC.msg.en_US.rte         16.1.0.0  COMMITTED  IBM XL C++ Runtime
  xlC.rte                   16.1.0.0  COMMITTED  IBM XL C++ Runtime for AIX
  xlCcmp.13.1.3             13.1.3.0  COMMITTED  XL C++ compiler
  xlCcmp.13.1.3.bundle      13.1.3.0  COMMITTED  XL C++ media defined bundles
  xlCcmp.13.1.3.lib         13.1.3.0  COMMITTED  XL C++ libraries
  xlCcmp.13.1.3.license     13.1.3.0  COMMITTED  XL C++ license files
  xlCcmp.13.1.3.ndi         13.1.3.0  COMMITTED  XL C++ non-default
  xlCcmp.13.1.3.tools       13.1.3.0  COMMITTED  XL C++ tools
  xlccmp.13.1.3             13.1.3.0  COMMITTED  XL C compiler
  xlccmp.13.1.3.bundle      13.1.3.0  COMMITTED  XL C media defined bundles
  xlccmp.13.1.3.lib         13.1.3.0  COMMITTED  XL C libraries
  xlccmp.13.1.3.license     13.1.3.0  COMMITTED  XL C license files
  xlccmp.13.1.3.ndi         13.1.3.0  COMMITTED  XL C non-default installation
 
root@aix7:/$lslpp -l|grep -i python3
   aixtools.python3.adt       3.6.9.0  COMMITTED  python python3 ADT files
   aixtools.python3.man.en_US
                      3.6.9.0  COMMITTED  python python3 man pages
   aixtools.python3.rte       3.6.9.0  COMMITTED  python python3 26-Oct-2019
   
   

root@aix7:/data1/python36/pyinstaller3.6/pyinstaller-3.6$pip3 list
Package      Version
------------ --------------
altgraph     0.17
APScheduler  3.6.3
func-timeout 4.3.5
future       0.18.2
macholib     1.14
pefile       2019.4.18pwd
pexpect      4.8.0
pika         1.1.0
pip          20.0.2
ptyprocess   0.6.0
PyInstaller  3.6+6d4cce1f8e
PyMySQL      0.9.3
pytz         2019.3
setuptools   40.6.2
six          1.14.0
some-package 0.1
SQLAlchemy   1.3.11
tzlocal      2.0.0
wheel        0.34.2


root@aix7:/data1/python36$pyinstaller -F  test.py
122 INFO: PyInstaller: 3.6+6d4cce1f8e
123 INFO: Python: 3.6.9
156 INFO: Platform: AIX-1-000374CBD600-powerpc-64bit-COFF
161 INFO: wrote /data1/python36/test.spec
192 INFO: UPX is not available.
195 INFO: Extending PYTHONPATH with paths
['/data1/python36', '/data1/python36']
196 INFO: checking Analysis
215 INFO: checking PYZ
224 INFO: checking PKG
224 INFO: Building PKG because PKG-00.toc is non existent
224 INFO: Building PKG (CArchive) PKG-00.pkg
Traceback (most recent call last):
  File "/opt/bin/pyinstaller", line 11, in <module>
    load_entry_point('PyInstaller==3.6+6d4cce1f8e', 'console_scripts', 'pyinstaller')()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/build_main.py", line 734, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/build_main.py", line 681, in build
    exec(code, spec_namespace)
  File "/data1/python36/test.spec", line 33, in <module>
    console=True )
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 436, in __init__
    upx_exclude=self.upx_exclude
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 199, in __init__
    self.__postinit__()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 276, in assemble
    pylib_name = os.path.basename(bindepend.get_python_library_path())
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/depend/bindepend.py", line 945, in get_python_library_path
    raise IOError(msg)
OSError: Python library not found: libpython3.6.a, libpython3.6.so
    This would mean your Python installation doesn't come with proper library files.
    This usually happens by missing development package, or unsuitable build parameters of Python installation.

    * On Debian/Ubuntu, you would need to install Python development packages
      * apt-get install python3-dev
      * apt-get install python-dev
    * If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)

[Michael]

a) This is, largely, historical changes in the way Python has managed, and named the shared libraries on AIX.
b) There are some additional issues (within pyinstaller) that I have been working on, and waiting for them to merge into pyinstaller.
See, or visit: https://github.com/pyinstaller/pyinstaller/issues?q=is%3Aissue+is%3Aopen+AIX
re: a) I'll work on a repackaing of Python-3.6 (now v3.6.10). I backport a patch in Python3.9 to support PEP425 tags, and I'll add some glue so that pyInstaller should find the libraries. One of the fixes (see b)) is also concerned with that searching. - What I'll do is add some symbolic links from where PyInstaller (and python!) look to where the library actually is.
re: the repackaging - I am stopping with the name aixtools.python3.* and now using aixtools.python.pyXY - where X is the major number, e.g., 3, and Y the minor (e.g., 6) to enable having multiple installations in parallel.
More later...

[Michael]

I have rebuilt Python-3.6.10, and I have downloaded and patched pyinstaller.
In the patched pyinstaller directory I run:

Code: [Select]

python ./setup.py bdist_wheelAnd now I see

Code: [Select]

(py36) root@x065:[/data/prj/python/virtual/py36/PyInstaller-3.6]ls -l dist
total 5920
-rw-r--r--   1 root     1000        3026207 Apr 01 15:58 PyInstaller-3.6-py3-none-any.whl
(py36) root@x065:[/data/prj/python/virtual/py36/PyInstaller-3.6]python ./setup.py bdist_wheel
Note: there is a bdist error here the file name should be:
PyInstaller-3.6-cp36-cp36-aix_5307_0747_64.whl -- BUT that does not matter for now.
Now you can install the PyInstaller wheel you just built:

Code: [Select]

(py36) root@x065:[/data/prj/python/virtual/py36]ln */dist/*whl .
(py36) root@x065:[/data/prj/python/virtual/py36]pip install *.whl
Processing ./PyInstaller-3.6-py3-none-any.whl
Processing ./altgraph-0.17-py2.py3-none-any.whl
Requirement already satisfied: setuptools==46.1.3 from file:///data/prj/python/virtual/py36/setuptools-46.1.3-py3-none-any.whl in ./lib/python3.6/site-packages (46.1.3)
Installing collected packages: altgraph, PyInstaller
Successfully installed PyInstaller-3.6 altgraph-0.17


Code: [Select]

(py36) root@x065:[/data/prj/python/virtual/py36]

"test.py" 3 lines, 37 characters
(py36) root@x065:[/data/prj/python/virtual/py36]./test.py
hello
(py36) root@x065:[/data/prj/python/virtual/py36]pyinstaller -F test.py
188 INFO: PyInstaller: 3.6
188 INFO: Python: 3.6.10
232 INFO: Platform: AIX-3-00C291F54C00-powerpc-64bit-COFF
237 INFO: wrote /data/prj/python/virtual/py36/test.spec
275 INFO: UPX is not available.
285 INFO: Extending PYTHONPATH with paths
['/data/prj/python/virtual/py36', '/data/prj/python/virtual/py36']
285 INFO: checking Analysis
286 INFO: Building Analysis because Analysis-00.toc is non existent
286 INFO: Initializing module dependency graph...
291 INFO: Caching module graph hooks...
329 INFO: Analyzing base_library.zip ...
11821 INFO: Caching module dependency graph...
12174 INFO: running Analysis Analysis-00.toc
13234 INFO: Analyzing /data/prj/python/virtual/py36/test.py
13243 INFO: Processing module hooks...
13244 INFO: Loading module hook "hook-xml.py"...
14071 INFO: Loading module hook "hook-encodings.py"...
14301 INFO: Loading module hook "hook-pydoc.py"...
14337 INFO: Looking for ctypes DLLs
14337 INFO: Analyzing run-time hooks ...
14361 INFO: Looking for dynamic libraries
ldd: /usr/lib/libcrypto.a: File is an archive.
ldd: /usr/lib/libssl.a: File is an archive.
25419 INFO: Looking for eggs
25420 INFO: Python library not in binary dependencies. Doing additional searching...
25803 INFO: Using Python library /lib/libpython3.6.so
25817 INFO: Warnings written to /data/prj/python/virtual/py36/build/test/warn-test.txt
25908 INFO: Graph cross-reference written to /data/prj/python/virtual/py36/build/test/xref-test.html
25969 INFO: checking PYZ
25969 INFO: Building PYZ because PYZ-00.toc is non existent
25969 INFO: Building PYZ (ZlibArchive) /data/prj/python/virtual/py36/build/test/PYZ-00.pyz
27532 INFO: Building PYZ (ZlibArchive) /data/prj/python/virtual/py36/build/test/PYZ-00.pyz completed successfully.
27544 INFO: checking PKG
27545 INFO: Building PKG because PKG-00.toc is non existent
27545 INFO: Building PKG (CArchive) PKG-00.pkg
123450 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
123459 INFO: Bootloader /data/prj/python/virtual/py36/lib/python3.6/site-packages/PyInstaller/bootloader/AIX-64bit/run
123459 INFO: checking EXE
123460 INFO: Building EXE because EXE-00.toc is non existent
123460 INFO: Building EXE from EXE-00.toc
123466 INFO: Appending archive to EXE /data/prj/python/virtual/py36/dist/test
123718 INFO: Building EXE from EXE-00.toc completed successfully.
(py36) root@x065:[/data/prj/python/virtual/py36]

The new package is at: http://download.aixtools.net/tools/aixtools.python.py36.3.6.10.0.I

[jessding2503]

Thanks. It worked

[jessding2503]

hello,Michael:
   I have another question
   I can install psutil success,use aixtools.python.py36.3.6.10.0,and it also work.
   but when i package my program with pyinstaller, it  failed to execute my script。
   
  ImportRrror: 0509-130 Symbol resolution failed for /data/manager_aix/libperfstat.a[shr_64.o] because"
      0509-136 Symbol pty_get_24x7_catalogpage_ext(number 190) is not exported from dependent module /unix
      0509-136 Symbol ptx_get_24x7_data_ext (number 191) is not exported from dependent module /unix
      0509-022 Cannot load module /data/manager_aix/psutil_aix.so.
     0509-192 Examine .loader section symbols with the 'dump -Tv' command.

[Michael]

I know it is saying from /unix - but I need to check ond see if that is actually supposed to come from the kernel or not.
Also, try - $ dump -X64 -H /data/manager_aix/psutil_aix.so
and also
$ ldd /data/manager_aix/psutil_aix.so
And is /data/manager_aix/psutil_aix.so opensource, or your own code?
atm: I do not think it is the Python executable, or pyinstaller - there is the implicit assumption that a shared library is able to find it's dependencies.

[Michael]

@jessding2503 - any change?