Pyinstaller issue with Python3.6 deployed via AIXTOOLs

[Michael]

I have a new version of Python3-3.6.8 packaged - for you to test. http://download.aixtools.net/test/aixtools.python3.3.6.8.0.I

You will need to manually add the symbolic link for /usr/lib/libpython3.6.so (#ls -s /opt/lib/libpython3.6m.so /usr/lib/libpython3.6.so).

Please note - this is the 32-bit build. If you require a 64-bit build, I'll do that as well.

I have filed an issue with pyinstaller re: it not finding recognizing the library3.6m.so (it finds it, but does not recognize it under that name).

Separately, I am going to open an issue with cpython re: that the shared library file is a seperate file, not an archive containing a shareable file (aka member).

All for today!

Michael

[martind]

I`d rather wait for the 64 bit version. My assumption is that the libpython3.6.so library will be rolled with Pyinstaller into the executable.

Thanks for your assistance.

[Michael]

I have a new version - 64-bit - you should be able to install it on top of what you have and the installed modules should still be there., e.g.,

+-----------------------------------------------------------------------------+
                                Summaries:
+-----------------------------------------------------------------------------+

Installation Summary
--------------------
Name                        Level           Part        Event       Result
-------------------------------------------------------------------------------
aixtools.python3.rte        3.6.8.0         USR         APPLY       SUCCESS
aixtools.python3.adt        3.6.8.0         USR         APPLY       SUCCESS
root@x067:[/home/root]pip list
Package     Version
----------- ---------
altgraph    0.16.1
future      0.17.1
macholib    1.11
pefile      2019.4.18
pip         19.1
PyInstaller 3.4
setuptools  40.6.2
You are using pip version 19.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


And

root@x067:[/home/root/python/hello]pyinstaller --onefile hello.py
119 INFO: PyInstaller: 3.4
119 INFO: Python: 3.6.8
166 INFO: Platform: AIX-1-00C291F54C00-powerpc-64bit-COFF
168 INFO: wrote /home/root/python/hello/hello.spec
208 INFO: UPX is not available.
212 INFO: Extending PYTHONPATH with paths
['/home/root/python/hello', '/home/root/python/hello']
213 INFO: checking Analysis
220 INFO: Building because _python_version changed
220 INFO: Initializing module dependency graph...
223 INFO: Initializing module graph hooks...
227 INFO: Analyzing base_library.zip ...
13260 INFO: running Analysis Analysis-00.toc
15014 INFO: Caching module hooks...
15030 INFO: Analyzing /home/root/python/hello/hello.py
15045 INFO: Loading module hooks...
15047 INFO: Loading module hook "hook-encodings.py"...
15268 INFO: Loading module hook "hook-pydoc.py"...
15270 INFO: Loading module hook "hook-xml.py"...
16102 INFO: Looking for ctypes DLLs
16102 INFO: Analyzing run-time hooks ...
16126 INFO: Looking for dynamic libraries
ldd: /usr/lib/libcrypto.a: File is an archive.
ldd: /usr/lib/libssl.a: File is an archive.
39262 INFO: Looking for eggs
39263 INFO: Python library not in binary dependencies. Doing additional searching...
39812 INFO: Using Python library /lib/libpython3.6.a
39827 INFO: Warnings written to /home/root/python/hello/build/hello/warn-hello.txt
39919 INFO: Graph cross-reference written to /home/root/python/hello/build/hello/xref-hello.html
39994 INFO: checking PYZ
39998 INFO: Building because toc changed
39998 INFO: Building PYZ (ZlibArchive) /home/root/python/hello/build/hello/PYZ-00.pyz
40958 INFO: Building PYZ (ZlibArchive) /home/root/python/hello/build/hello/PYZ-00.pyz completed successfully.
40967 INFO: checking PKG
40969 INFO: Building because toc changed
40969 INFO: Building PKG (CArchive) PKG-00.pkg
77487 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
78656 INFO: Bootloader /opt/lib/python3.6/site-packages/PyInstaller-3.4-py3.6.egg/PyInstaller/bootloader/AIX-64bit/run
78657 INFO: checking EXE
78659 INFO: Building because toc changed
78660 INFO: Building EXE from EXE-00.toc
78673 INFO: Appending archive to EXE /home/root/python/hello/dist/hello
81131 INFO: Building EXE from EXE-00.toc completed successfully.

[martind]

where is the 64bit aixtools python package?

[martind]

the 32 bit version install fails thus:


+-----------------------------------------------------------------------------+
                         Installing Software...
+-----------------------------------------------------------------------------+

installp:  APPLYING software for:
        aixtools.python3.rte 3.6.8.0
        aixtools.python3.man.en_US 3.6.8.0
        aixtools.python3.adt 3.6.8.0

Restoring files, please wait.
1552 files restored.
2942 files restored.
4006 files restored.
5141 files restored.
Mount volume 2 on /home/dyer/aixtools.python3.3.6.8.0.I.
        Press the Enter key to continue.
0503-700 inurest:  Error in restoring files
0503-037 inurest:  Failure on system call to execute command /usr/sbin/restbyname -S  -xYAqf/home/dyer/aixtools.python3.3.6.8.0.I -Z /tmp/inutmp8MwMaa/sorted.al.

0503-464 installp:  The installation has FAILED for the "usr" part
        of the following filesets:
        aixtools.python3.rte 3.6.8.0
        aixtools.python3.man.en_US 3.6.8.0
        aixtools.python3.adt 3.6.8.0

installp:  Cleaning up software for:
        aixtools.python3.rte 3.6.8.0
        aixtools.python3.man.en_US 3.6.8.0
        aixtools.python3.adt 3.6.8.0

any ideas why?

[Michael]

My apologies: at http://download.aixtools.net/test/aixtools.python3.3.6.8.0-64bit.I

Do not have the 32-bit and 64-bit in the same directory when you install - unclear which one will install.

btw - the installp flag -d USUALLY points at a directory, and then it reads/creates .toc in the directory. If you give a filename as the value to -d, it only installs from that file.

Thus: after download you could try installp -d aixtools.python3.3.6.8.0-64bit.I -aF aixtools.python3.rte aixtools.python3.adt

Why it is asking for a second volume - no idea - unless you did not get the entire file.

[martind]

pyinstaller works fine for helloworld:

# pyinstaller --onefile helloworld.py
72 INFO: PyInstaller: 3.4
73 INFO: Python: 3.6.8
111 INFO: Platform: AIX-1-00F9B3C44C00-powerpc-64bit-COFF
119 INFO: wrote /home/dyer/helloworld.spec
153 INFO: UPX is not available.
172 INFO: Extending PYTHONPATH with paths
['/home/dyer', '/home/dyer']
172 INFO: checking Analysis
172 INFO: Building Analysis because Analysis-00.toc is non existent
172 INFO: Initializing module dependency graph...
174 INFO: Initializing module graph hooks...
177 INFO: Analyzing base_library.zip ...
7007 INFO: running Analysis Analysis-00.toc
7794 INFO: Caching module hooks...
7805 INFO: Analyzing /home/dyer/helloworld.py
7814 INFO: Loading module hooks...
7815 INFO: Loading module hook "hook-encodings.py"...
7973 INFO: Loading module hook "hook-pydoc.py"...
7975 INFO: Loading module hook "hook-xml.py"...
8523 INFO: Looking for ctypes DLLs
8523 INFO: Analyzing run-time hooks ...
8537 INFO: Looking for dynamic libraries
ldd: /usr/lib/libssl.a: File is an archive.
ldd: /usr/lib/libcrypto.a: File is an archive.
17099 INFO: Looking for eggs
17100 INFO: Python library not in binary dependencies. Doing additional searching...
17477 INFO: Using Python library /opt/lib/libpython3.6.so
17494 INFO: Warnings written to /home/dyer/build/helloworld/warn-helloworld.txt
17553 INFO: Graph cross-reference written to /home/dyer/build/helloworld/xref-helloworld.html
17624 INFO: checking PYZ
17624 INFO: Building PYZ because PYZ-00.toc is non existent
17624 INFO: Building PYZ (ZlibArchive) /home/dyer/build/helloworld/PYZ-00.pyz
18306 INFO: Building PYZ (ZlibArchive) /home/dyer/build/helloworld/PYZ-00.pyz completed successfully.
18319 INFO: checking PKG
18319 INFO: Building PKG because PKG-00.toc is non existent
18319 INFO: Building PKG (CArchive) PKG-00.pkg
42557 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
42575 INFO: Bootloader /opt/lib/python3.6/site-packages/PyInstaller-3.4-py3.6.egg/PyInstaller/bootloader/AIX-64bit/run
42576 INFO: checking EXE
42576 INFO: Building EXE because EXE-00.toc is non existent
42576 INFO: Building EXE from EXE-00.toc
42576 INFO: Appending archive to EXE /home/dyer/dist/helloworld
42651 INFO: Building EXE from EXE-00.toc completed successfully.

then cd dist:

# cd dist
# ls -l
total 38536
-rwxr-xr-x    1 root     system     19723463 May 09 09:04 helloworld
# ./helloworld
hello world

perfect!!

however, ipp_testing.py:

pyinstaller --onefile IPP_Testing.py
72 INFO: PyInstaller: 3.4
72 INFO: Python: 3.6.8
111 INFO: Platform: AIX-1-00F9B3C44C00-powerpc-64bit-COFF
119 INFO: wrote /home/dyer/IPP_Testing.spec
153 INFO: UPX is not available.
164 INFO: Extending PYTHONPATH with paths
['/home/dyer', '/home/dyer']
164 INFO: checking Analysis
164 INFO: Building Analysis because Analysis-00.toc is non existent
164 INFO: Initializing module dependency graph...
166 INFO: Initializing module graph hooks...
169 INFO: Analyzing base_library.zip ...
6960 INFO: running Analysis Analysis-00.toc
7698 INFO: Caching module hooks...
7709 INFO: Analyzing /home/dyer/IPP_Testing.py
8308 INFO: Loading module hooks...
8309 INFO: Loading module hook "hook-encodings.py"...
8470 INFO: Loading module hook "hook-pydoc.py"...
8472 INFO: Loading module hook "hook-xml.py"...
8598 INFO: Looking for ctypes DLLs
8598 INFO: Analyzing run-time hooks ...
8613 INFO: Looking for dynamic libraries
ldd: /usr/lib/libssl.a: File is an archive.
ldd: /usr/lib/libcrypto.a: File is an archive.
16951 INFO: Looking for eggs
16952 INFO: Python library not in binary dependencies. Doing additional searching...
17310 INFO: Using Python library /opt/lib/libpython3.6.a
17323 INFO: Warnings written to /home/dyer/build/IPP_Testing/warn-IPP_Testing.txt
17381 INFO: Graph cross-reference written to /home/dyer/build/IPP_Testing/xref-IPP_Testing.html
17405 INFO: checking PYZ
17405 INFO: Building PYZ because PYZ-00.toc is non existent
17405 INFO: Building PYZ (ZlibArchive) /home/dyer/build/IPP_Testing/PYZ-00.pyz
18084 INFO: Building PYZ (ZlibArchive) /home/dyer/build/IPP_Testing/PYZ-00.pyz completed successfully.
18100 INFO: checking PKG
18100 INFO: Building PKG because PKG-00.toc is non existent
18100 INFO: Building PKG (CArchive) PKG-00.pkg
46750 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
46778 INFO: Bootloader /opt/lib/python3.6/site-packages/PyInstaller-3.4-py3.6.egg/PyInstaller/bootloader/AIX-64bit/run
46779 INFO: checking EXE
46779 INFO: Building EXE because EXE-00.toc is non existent
46779 INFO: Building EXE from EXE-00.toc
46779 INFO: Appending archive to EXE /home/dyer/dist/IPP_Testing
46858 INFO: Building EXE from EXE-00.toc completed successfully.

seems to build fine. however, when attempting to execute i see:

# cd dist
# ls -l
total 83144
-rwxr-xr-x    1 root     system     22841941 May 09 09:06 IPP_Testing
-rwxr-xr-x    1 root     system     19723463 May 09 09:04 helloworld
# ./IPP_Testing
[5767290] DLL name length exceeds buffer

hmmh.....

[Michael]

Good News - and frustrating news.

Have you opened an issue with PyInstaller?

I have two opened: PyInstaller cannot build bootloader for 32-bit or 64-bit  and Pyinstaller3.4 does not recognize libpython3.6m.so as valid library.

Guess I need to also check for a maillist. "Check".

An example of an issue you could make is: PyInstaller generated executable crashes with GStreamer on Windows 10

And maybe you know this already - but there are some suggestions - such as:

Package your program in --onedir mode. If this does work, but --onefile mode does not, this most times is an indicator for a problem within your application or one of the libraries you are using.

Package without UPX, say: use the option --noupx or set upx=False in your .spec-file. Some libraries are known to not work with this. And upx is known to have issues when compressing .dlls.

Important: Never use --onefile for debugging. Try solving the problem in --onedir mode. This is much easier since you can easily see what gets collected and digging into problems is much easier, too.

Note: The tool pyi-archive_viewer can be used to look into the packed application (the .exe file) and verify all required modules and data-files are there. You'll have to look into outPYZ*.pyz, too. Please see the the manual for how to use pyi-archive_viewer.

plus more at PyInstaller:How to Report Bugs