Date: 28-11-20  Time: 00:02 AM

Recent Posts

Pages: 1 ... 8 9 [10]
91
AIXTOOLS / Re: Pyinstaller3.6 issue in aix7.1
« Last post by jessding2503 on April 03, 2020, 07:09:08 AM »
hello,Michael:
   I have another question
   I can install psutil success,use aixtools.python.py36.3.6.10.0,and it also work.
   but when i package my program with pyinstaller, it  failed to execute my script。
   
  ImportRrror: 0509-130 Symbol resolution failed for /data/manager_aix/libperfstat.a[shr_64.o] because"
      0509-136 Symbol pty_get_24x7_catalogpage_ext(number 190) is not exported from dependent module /unix
      0509-136 Symbol ptx_get_24x7_data_ext (number 191) is not exported from dependent module /unix
      0509-022 Cannot load module /data/manager_aix/psutil_aix.so.
     0509-192 Examine .loader section symbols with the 'dump -Tv' command.

92
AIXTOOLS / Re: Python 3.8, ModuleNotFoundError: No module named '_sqlite3'
« Last post by Michael on April 02, 2020, 10:08:50 AM »
Here is a sample script - does not DO anything. You could pipe the output into (k)sh to automate renaming wheels made on other (older) systems so that you can load them on the running system.
Code: [Select]
#!/usr/bin/env python3
"""
Examine arguments for aix pep425 tags
and propose ln (link) commands to give a new name
to an existing wheel for AIX.

Copyright 2020 by AIXTOOLS.NET, Michael Felt
"""

import sys, glob, sysconfig

# save runttime-tag and reverse a copy for comparision
rt_tag = sysconfig.get_platform().split("-")
_tag = list(rt_tag)
_tag.reverse()

# loop through all arguments and look for a AIX tags
for idx in range(1,len(sys.argv)):
  for arg in glob.glob(sys.argv[idx]):
    wheel=arg.split("-")
    if not wheel[-1].startswith("aix_"):
        continue
    # have a potential tag, reverse tag for comparision
    file_tag=wheel[-1].split(".")[0].split("_")
    file_tag.reverse()

    # if the tags are equal, or they are not 32-bit|64-bit - continue
    if (_tag == file_tag) or _tag[0] not in ["32", "64"]:
      continue

    # if the running tag is greater AND same bit-size link names
    elif (_tag > file_tag) and _tag[0] == file_tag[0]:
      wheel[-1] = "_".join(rt_tag) + ".whl"
      print("ln {} {}".format(arg, "-".join(wheel)))
93
AIXTOOLS / Re: Python 3.8, ModuleNotFoundError: No module named '_sqlite3'
« Last post by Michael on April 02, 2020, 06:06:40 AM »
Thanks for the update. And also the verification that renaming the files works.
I need to get that to be automated, but pypa is very slow to incorporate the changes - even after the base was added to to Python3.9.
Anyway, I'll start with a program to "rename" files that would be binary compatible .Three checks of running system compared with "package" system:
  • both are 64 or 32 bit
  • running system builddate >= than the filename
  • running system AIX-TL >= than AIX-TL in the filename.
94
AIXTOOLS / Re: Pyinstaller3.6 issue in aix7.1
« Last post by jessding2503 on April 02, 2020, 02:41:37 AM »
Thanks. It worked
95
AIXTOOLS / Re: Python 3.8, ModuleNotFoundError: No module named '_sqlite3'
« Last post by hammertime on April 01, 2020, 11:14:41 PM »
Thanks @Michael .  I have tested the commands you listed and all work perfectly.  For information here are the wheel files that were created -

-rw-r--r--    1 root     system       156030 01 Apr 22:03 certifi-2019.11.28-py2.py3-none-any.whl
-rw-r--r--    1 root     system       133356 01 Apr 22:03 chardet-3.0.4-py2.py3-none-any.whl
-rw-r--r--    1 root     system       255894 01 Apr 22:22 cx_Oracle-7.3.0-cp38-cp38-aix_7105_1913_64.whl
-rw-r--r--    1 root     system        58388 01 Apr 22:03 idna-2.9-py2.py3-none-any.whl
-rw-r--r--    1 root     system      1449902 01 Apr 22:22 nltk-3.4.5-py3-none-any.whl
-rw-r--r--    1 root     system         8712 01 Apr 22:22 phonetics-1.0.5-py2.py3-none-any.whl
-rw-r--r--    1 root     system        18928 01 Apr 22:03 PyJWT-1.7.1-py2.py3-none-any.whl
-rw-r--r--    1 root     system       509158 01 Apr 22:03 pytz-2019.3-py2.py3-none-any.whl
-rw-r--r--    1 root     system        58406 01 Apr 22:03 requests-2.23.0-py2.py3-none-any.whl
-rw-r--r--    1 root     system        10938 01 Apr 22:03 six-1.14.0-py2.py3-none-any.whl
-rw-r--r--    1 root     system       636507 01 Apr 22:03 textblob-0.15.3-py2.py3-none-any.whl
-rw-r--r--    1 root     system      1105877 01 Apr 22:22 twilio-6.38.0-py2.py3-none-any.whl
-rw-r--r--    1 root     system       125642 01 Apr 22:03 urllib3-1.25.8-py2.py3-none-any.whl


I did then copy the cx_Oracle whl file to another server with a slight different patch level so had to rename the _1913_ part of the filename.  So thanks again as I picked this up from reading your other threads.

As to mileage...  at the moment I only have one developer asking to use Python on AIX.  So hard to say.  It is good to see what can be done so please keep up your excellent work!
96
AIXTOOLS / Re: Pyinstaller3.6 issue in aix7.1
« Last post by Michael on April 01, 2020, 04:16:24 PM »
I have rebuilt Python-3.6.10, and I have downloaded and patched pyinstaller.
In the patched pyinstaller directory I run:
Code: [Select]
python ./setup.py bdist_wheelAnd now I see
Code: [Select]
(py36) root@x065:[/data/prj/python/virtual/py36/PyInstaller-3.6]ls -l dist
total 5920
-rw-r--r--   1 root     1000        3026207 Apr 01 15:58 PyInstaller-3.6-py3-none-any.whl
(py36) root@x065:[/data/prj/python/virtual/py36/PyInstaller-3.6]python ./setup.py bdist_wheel
Note: there is a bdist error here the file name should be:
PyInstaller-3.6-cp36-cp36-aix_5307_0747_64.whl -- BUT that does not matter for now.
Now you can install the PyInstaller wheel you just built:
Code: [Select]
(py36) root@x065:[/data/prj/python/virtual/py36]ln */dist/*whl .
(py36) root@x065:[/data/prj/python/virtual/py36]pip install *.whl
Processing ./PyInstaller-3.6-py3-none-any.whl
Processing ./altgraph-0.17-py2.py3-none-any.whl
Requirement already satisfied: setuptools==46.1.3 from file:///data/prj/python/virtual/py36/setuptools-46.1.3-py3-none-any.whl in ./lib/python3.6/site-packages (46.1.3)
Installing collected packages: altgraph, PyInstaller
Successfully installed PyInstaller-3.6 altgraph-0.17
Code: [Select]
(py36) root@x065:[/data/prj/python/virtual/py36]

"test.py" 3 lines, 37 characters
(py36) root@x065:[/data/prj/python/virtual/py36]./test.py
hello
(py36) root@x065:[/data/prj/python/virtual/py36]pyinstaller -F test.py
188 INFO: PyInstaller: 3.6
188 INFO: Python: 3.6.10
232 INFO: Platform: AIX-3-00C291F54C00-powerpc-64bit-COFF
237 INFO: wrote /data/prj/python/virtual/py36/test.spec
275 INFO: UPX is not available.
285 INFO: Extending PYTHONPATH with paths
['/data/prj/python/virtual/py36', '/data/prj/python/virtual/py36']
285 INFO: checking Analysis
286 INFO: Building Analysis because Analysis-00.toc is non existent
286 INFO: Initializing module dependency graph...
291 INFO: Caching module graph hooks...
329 INFO: Analyzing base_library.zip ...
11821 INFO: Caching module dependency graph...
12174 INFO: running Analysis Analysis-00.toc
13234 INFO: Analyzing /data/prj/python/virtual/py36/test.py
13243 INFO: Processing module hooks...
13244 INFO: Loading module hook "hook-xml.py"...
14071 INFO: Loading module hook "hook-encodings.py"...
14301 INFO: Loading module hook "hook-pydoc.py"...
14337 INFO: Looking for ctypes DLLs
14337 INFO: Analyzing run-time hooks ...
14361 INFO: Looking for dynamic libraries
ldd: /usr/lib/libcrypto.a: File is an archive.
ldd: /usr/lib/libssl.a: File is an archive.
25419 INFO: Looking for eggs
25420 INFO: Python library not in binary dependencies. Doing additional searching...
25803 INFO: Using Python library /lib/libpython3.6.so
25817 INFO: Warnings written to /data/prj/python/virtual/py36/build/test/warn-test.txt
25908 INFO: Graph cross-reference written to /data/prj/python/virtual/py36/build/test/xref-test.html
25969 INFO: checking PYZ
25969 INFO: Building PYZ because PYZ-00.toc is non existent
25969 INFO: Building PYZ (ZlibArchive) /data/prj/python/virtual/py36/build/test/PYZ-00.pyz
27532 INFO: Building PYZ (ZlibArchive) /data/prj/python/virtual/py36/build/test/PYZ-00.pyz completed successfully.
27544 INFO: checking PKG
27545 INFO: Building PKG because PKG-00.toc is non existent
27545 INFO: Building PKG (CArchive) PKG-00.pkg
123450 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
123459 INFO: Bootloader /data/prj/python/virtual/py36/lib/python3.6/site-packages/PyInstaller/bootloader/AIX-64bit/run
123459 INFO: checking EXE
123460 INFO: Building EXE because EXE-00.toc is non existent
123460 INFO: Building EXE from EXE-00.toc
123466 INFO: Appending archive to EXE /data/prj/python/virtual/py36/dist/test
123718 INFO: Building EXE from EXE-00.toc completed successfully.
(py36) root@x065:[/data/prj/python/virtual/py36]

The new package is at: http://download.aixtools.net/tools/aixtools.python.py36.3.6.10.0.I
97
AIXTOOLS / Re: Pyinstaller3.6 issue in aix7.1
« Last post by Michael on April 01, 2020, 11:13:49 AM »
a) This is, largely, historical changes in the way Python has managed, and named the shared libraries on AIX.
b) There are some additional issues (within pyinstaller) that I have been working on, and waiting for them to merge into pyinstaller.
See, or visit: https://github.com/pyinstaller/pyinstaller/issues?q=is%3Aissue+is%3Aopen+AIX
re: a) I'll work on a repackaing of Python-3.6 (now v3.6.10). I backport a patch in Python3.9 to support PEP425 tags, and I'll add some glue so that pyInstaller should find the libraries. One of the fixes (see b)) is also concerned with that searching. - What I'll do is add some symbolic links from where PyInstaller (and python!) look to where the library actually is.
re: the repackaging - I am stopping with the name aixtools.python3.* and now using aixtools.python.pyXY - where X is the major number, e.g., 3, and Y the minor (e.g., 6) to enable having multiple installations in parallel.
More later...
98
AIXTOOLS / Pyinstaller3.6 issue in aix7.1
« Last post by jessding2503 on April 01, 2020, 09:47:00 AM »
Hi,Our server is AIX 7.1 and aixtools.python3.6 and xlc are already installed.
According to this article(http://forums.rootvg.net/aixtools/pyinstaller-issue-with-python3-6-deployed-via-aixtools/msg2839/#msg2839) I installed pyinstaller successfully
but it have other issue, Please help to fix the issue.



root@aix7:/$lslpp -l|grep -i xlc
  xlC.adt.include           16.1.0.0  COMMITTED  C Set ++ Application
  xlC.aix61.rte             16.1.0.0  COMMITTED  IBM XL C++ Runtime for AIX 6.1
  xlC.cpp                    9.0.0.0  COMMITTED  C for AIX Preprocessor
  xlC.msg.en_US.cpp          9.0.0.0  COMMITTED  C for AIX Preprocessor
  xlC.msg.en_US.rte         16.1.0.0  COMMITTED  IBM XL C++ Runtime
  xlC.rte                   16.1.0.0  COMMITTED  IBM XL C++ Runtime for AIX
  xlCcmp.13.1.3             13.1.3.0  COMMITTED  XL C++ compiler
  xlCcmp.13.1.3.bundle      13.1.3.0  COMMITTED  XL C++ media defined bundles
  xlCcmp.13.1.3.lib         13.1.3.0  COMMITTED  XL C++ libraries
  xlCcmp.13.1.3.license     13.1.3.0  COMMITTED  XL C++ license files
  xlCcmp.13.1.3.ndi         13.1.3.0  COMMITTED  XL C++ non-default
  xlCcmp.13.1.3.tools       13.1.3.0  COMMITTED  XL C++ tools
  xlccmp.13.1.3             13.1.3.0  COMMITTED  XL C compiler
  xlccmp.13.1.3.bundle      13.1.3.0  COMMITTED  XL C media defined bundles
  xlccmp.13.1.3.lib         13.1.3.0  COMMITTED  XL C libraries
  xlccmp.13.1.3.license     13.1.3.0  COMMITTED  XL C license files
  xlccmp.13.1.3.ndi         13.1.3.0  COMMITTED  XL C non-default installation
 
root@aix7:/$lslpp -l|grep -i python3
   aixtools.python3.adt       3.6.9.0  COMMITTED  python python3 ADT files
   aixtools.python3.man.en_US
                      3.6.9.0  COMMITTED  python python3 man pages
   aixtools.python3.rte       3.6.9.0  COMMITTED  python python3 26-Oct-2019
   
   

root@aix7:/data1/python36/pyinstaller3.6/pyinstaller-3.6$pip3 list
Package      Version
------------ --------------
altgraph     0.17
APScheduler  3.6.3
func-timeout 4.3.5
future       0.18.2
macholib     1.14
pefile       2019.4.18pwd
pexpect      4.8.0
pika         1.1.0
pip          20.0.2
ptyprocess   0.6.0
PyInstaller  3.6+6d4cce1f8e
PyMySQL      0.9.3
pytz         2019.3
setuptools   40.6.2
six          1.14.0
some-package 0.1
SQLAlchemy   1.3.11
tzlocal      2.0.0
wheel        0.34.2


root@aix7:/data1/python36$pyinstaller -F  test.py
122 INFO: PyInstaller: 3.6+6d4cce1f8e
123 INFO: Python: 3.6.9
156 INFO: Platform: AIX-1-000374CBD600-powerpc-64bit-COFF
161 INFO: wrote /data1/python36/test.spec
192 INFO: UPX is not available.
195 INFO: Extending PYTHONPATH with paths
['/data1/python36', '/data1/python36']
196 INFO: checking Analysis
215 INFO: checking PYZ
224 INFO: checking PKG
224 INFO: Building PKG because PKG-00.toc is non existent
224 INFO: Building PKG (CArchive) PKG-00.pkg
Traceback (most recent call last):
  File "/opt/bin/pyinstaller", line 11, in <module>
    load_entry_point('PyInstaller==3.6+6d4cce1f8e', 'console_scripts', 'pyinstaller')()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/build_main.py", line 734, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/build_main.py", line 681, in build
    exec(code, spec_namespace)
  File "/data1/python36/test.spec", line 33, in <module>
    console=True )
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 436, in __init__
    upx_exclude=self.upx_exclude
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 199, in __init__
    self.__postinit__()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/building/api.py", line 276, in assemble
    pylib_name = os.path.basename(bindepend.get_python_library_path())
  File "/opt/lib/python3.6/site-packages/PyInstaller-3.6+6d4cce1f8e-py3.6.egg/PyInstaller/depend/bindepend.py", line 945, in get_python_library_path
    raise IOError(msg)
OSError: Python library not found: libpython3.6.a, libpython3.6.so
    This would mean your Python installation doesn't come with proper library files.
    This usually happens by missing development package, or unsuitable build parameters of Python installation.

    * On Debian/Ubuntu, you would need to install Python development packages
      * apt-get install python3-dev
      * apt-get install python-dev
    * If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)

99
AIXTOOLS / Re: Python 3.8, ModuleNotFoundError: No module named '_sqlite3'
« Last post by Michael on April 01, 2020, 08:24:07 AM »
Your pep425 tag will probably be different (for cx_oracle) - but you can try using:
Code: [Select]
export OBJECT_MODE=64
pip3 wheel twilio phonetics cx_oracle textblob
This should create or download several wheels, e.g.
Code: [Select]
(py38) root@x065:[/data/prj/python/virtual]ls -l *.whl
-rw-r--r--   1 root     felt          18928 Apr 01 08:14 PyJWT-1.7.1-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         156030 Apr 01 08:14 certifi-2019.11.28-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         133356 Apr 01 08:14 chardet-3.0.4-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         239828 Apr 01 08:17 cx_Oracle-7.3.0-cp38-cp38-aix_5307_0747_64.whl
-rw-r--r--   1 root     felt          58388 Apr 01 08:14 idna-2.9-py2.py3-none-any.whl
-rw-r--r--   1 root     felt        1449905 Mar 31 17:20 nltk-3.4.5-py3-none-any.whl
-rw-r--r--   1 root     felt           8712 Apr 01 08:15 phonetics-1.0.5-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         509158 Apr 01 08:14 pytz-2019.3-py2.py3-none-any.whl
-rw-r--r--   1 root     felt          58406 Apr 01 08:14 requests-2.23.0-py2.py3-none-any.whl
-rw-r--r--   1 root     felt          10938 Mar 31 17:20 six-1.14.0-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         636507 Apr 01 08:14 textblob-0.15.3-py2.py3-none-any.whl
-rw-r--r--   1 root     felt        1091094 Apr 01 08:15 twilio-6.37.0-py2.py3-none-any.whl
-rw-r--r--   1 root     felt         125642 Apr 01 08:14 urllib3-1.25.8-py2.py3-none-any.whl
Now - if you look carefully, you will see my prompt has (py38) in it's name. That is because BEFORE I created these wheels I installed virtualenv and created a virtualenv.
So, as root - install virtualenv, and as a regular user (e.g., michael) create a virtualenv (e.g., py38) using the command
Code: [Select]
$ virtualenv $HOME/py38; . $HOME/py38/bin/activateAnd then - install these wheels - NOT as root!! - great for testing!
Code: [Select]
(py38) $ pip install *.whlCurious about your mileage!
100
AIXTOOLS / Re: Python 3.8, ModuleNotFoundError: No module named '_sqlite3'
« Last post by hammertime on March 31, 2020, 07:46:26 PM »
Many thanks @Michael for your speedy responses.

Some very quick testing from me and all looking good.

Code: [Select]
[ts01]:root:/root # python3
Python 3.8.2 (default, Mar 31 2020, 16:07:53) [C] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import nltk
>>> import twilio
>>> import phonetics
>>> import cx_Oracle
>>> import textblob
>>>
Pages: 1 ... 8 9 [10]