AIX, Linux and POWER Portal

AIX => AIXTOOLS => Topic started by: prolly_brent on July 24, 2019, 02:47:06 PM

Title: Paramiko for Python3
Post by: prolly_brent on July 24, 2019, 02:47:06 PM
I really like using Paramiko under Python3 for doing SSH operations.  Easy enough in the Linux/RHEL world, but I have had no success getting this module inside of Python3 on AIX (where I really need it).  It won't pip install under gcc with  AIX ToolBox's version of python3 (errors out) and I don't have the luxury of an IBM C compiler.  Any chance you would consider making paramiko accessible via your distribution?

Thanks,
Brent Fortman
Title: Re: Paramiko for Python3
Post by: Michael on July 25, 2019, 09:44:42 AM
My pleasure to give it a try. Which Python3 are you using? 3.5, 3.6, or 3.7?

Michael
Title: Re: Paramiko for Python3
Post by: prolly_brent on July 25, 2019, 12:46:05 PM
I am currently using 3.7.  Thanks.

Brent
Title: Re: Paramiko for Python3
Post by: Michael on July 26, 2019, 05:10:01 PM
I have all the modules built - but something is not 100% correct (the dependencies relationships) is not working correctly.

I'll try and fix that - but for testing I'll put all in http://download.aixtools.net/test - asap.
Title: Re: Paramiko for Python3
Post by: prolly_brent on July 26, 2019, 06:29:33 PM
I have downloaded, installed (on AIX 7200-03-02-1846), and done some basic testing.  No issues installing, and no errors during testing.  This is really great!  Thanks so much and I will keep an eye out for the official version when that's ready.

Sample code for doing simple remote command via SSH using paramiko:

#!/opt/bin/python3

def normal( a_str ):
    return a_str.decode('utf-8').rstrip('\n')

import paramiko

k = paramiko.RSAKey.from_private_key_file("/.ssh/id_rsa")
client = paramiko.SSHClient()
client.set_missing_host_key_policy( paramiko.AutoAddPolicy() )
client.connect(
                 hostname = "myhost",
                 username = "root",
                 pkey = k
               )

stdin, stdout, stderr = client.exec_command( "ls -al" )
print( normal( stdout.read() ) )

client.close()


Regards,
Brent
Title: Re: Paramiko for Python3
Post by: Michael on July 28, 2019, 01:17:03 PM
Thanks for the feedback.

The later packaging will be the same binaries. The only difference in /usr/lpp/* files. I need to debug why it is not taking my requisites info. Cutting too many corners I guess.

Michael
Title: Re: Paramiko for Python3
Post by: Michael on July 28, 2019, 02:52:10 PM
Actually, the packaging is okay as is. The corner I cut was to not "inutoc .", i.e., recreate the .toc file.

If you would try to "uninstall", e.g., six using
Code: [Select]
# installp -up aixtools.python3.six you should get something as:

  Dependency Failures
  (Deinstall Operation)
  ---------------------
  SELECTED FILESETS:  The following is a list of filesets that you asked to
  remove.  They cannot be removed until all of their dependent filesets
  are also removed.  See subsequent lists for details of dependents.

    aixtools.python3.six.rte 1.12.0.0         # tools six 26-Jul-2019

  INSTALLED DEPENDENTS:  The following filesets are dependents of one or more
  of the selected filesets listed above.  These must be removed before
  or with the filesets that you selected.  To remove these dependents with
  the selected filesets, specify the option to automatically remove dependent
  software (-g flag).

    aixtools.python3.asn1crypto.rte 0.24.0.0  # tools asn1crypto 26-Jul-2019
    aixtools.python3.bcrypt.rte 3.1.7.0       # tools bcrypt 26-Jul-2019
    aixtools.python3.cffi.rte 1.12.3.0        # tools cffi 26-Jul-2019
    aixtools.python3.cryptography.rte 2.7.0.0 # tools cryptography 26-Jul-2019
    aixtools.python3.paramiko.rte 2.6.0.0     # tools paramiko 26-Jul-2019
    aixtools.python3.pynacl.rte 1.3.0.0       # tools pynacl 26-Jul-2019

  << End of Failure Section >>

SUCCESSES
---------
  Filesets listed in this section passed pre-deinstall verification
  and will be removed.

  Selected Filesets
  -----------------
  aixtools.python3.six.adt 1.12.0.0           # tools six ADT files

  << End of Success Section >>

FILESET STATISTICS
------------------
    2  Selected to be deinstalled, of which:
        1  Passed pre-deinstall verification
        1  FAILED pre-deinstall verification
  ----
    1  Total to be deinstalled


******************************************************************************
End of installp PREVIEW.  No deinstall operation has actually occurred.
******************************************************************************


Now I just need to figure out how to best include these items in the wiki.
Title: Re: Paramiko for Python3
Post by: Duckabush on July 31, 2019, 05:29:42 PM
Wow this is a fantastic thread to stumble upon today. I been looking everywhere to ensure I could install Paramiko on our AIX 7.2 setup, but our IT Admin is very fearful of breaking our UniVerse install. We also do not have the luxury of owning the xlC/C++ compiler from IBM, we were attempting to install GCC, but again we don't want to run into any issues.

I hope that we can get Paramiko running as well with python 3.7.
Title: Re: Paramiko for Python3
Post by: Duckabush on July 31, 2019, 05:38:18 PM
@Michael,

Our system is generally maintained by another company, with minor modifications to our system.

Do we need to remove our Python37 install, and install new, and how do I go about installing paramiko, do i need to install files from http://download.aixtools.net/test/paramiko/?
 (Note from Michael - this directory has been removed) - See http://www.aixtools.net/index.php?title=python.wheels and http://download.aixtools.net/tools/Python3/apps/paramiko/

Thanks


edit: so based on looking around I downloaded all the files for python3 and need to put them in a folder and run
installp -d aixtools.python3.3.7.4.0.I -a all

correct?
Title: Re: Paramiko for Python3
Post by: Michael on August 02, 2019, 05:42:25 AM
The command would be “installp -d. All”

I found some extra dependencies on some gnu software I don’t want to be there, so please wait for the update to the wiki.

What is your “univers” install?

I try to make my packaging work in parallel with AIX toolbox et al.  So my hope and expectation is that nothing breaks.

Please wait for updates!!
Title: Re: Paramiko for Python3
Post by: Duckabush on August 02, 2019, 06:10:00 AM
Thanks, we will wait.

Our UniVerse install is by Rocket Software.

https://www.rocketsoftware.com/products/rocket-universe-0/rocket-universe
Title: Re: Paramiko for Python3
Post by: Michael on August 02, 2019, 12:33:30 PM
I have updated the Python3 installable - to not need gnu.gettext or gnu.libiconv, and have updated the aixtools wiki with the page python.wheels (http://www.aixtools.net/index.php/python.wheels) - which is the direction I want to go. For now, installp is still needed.

p.s. - I have contacted Rocket to get a trial version of Universe. My hope is that you could reconfigure Universe to find my Python (and just leave the version supplied by Rocket alone).

Any questions - problems encountered?
Title: Re: Paramiko for Python3
Post by: Michael on August 05, 2019, 12:04:49 PM
@Duckabush - is this working for you - or do you have a question?
Title: Re: Paramiko for Python3
Post by: Michael on August 05, 2019, 06:19:28 PM
I had a quick chat with rocket software support.

Their version of Python (probably still 3.4.1) is embedded (at p[robable /opt/uv/python/....
There should be no conflict with their Python installation and mine residing at /opt/bin and /opt/lib/python3.7

Hope this helps!
Title: Re: Paramiko for Python3
Post by: Duckabush on August 05, 2019, 06:30:18 PM
Thanks, we are checking it out today. Will report back. Thanks so much for your help.

@Michael

We ran
installp -d aixtools.python3.3.7.0.0.32bit.I -a all

It said python was already installed and didn't continue to install. When I tried to run python it is now uninstalled. Did I do something wrong?
Title: Re: Paramiko for Python3
Post by: Michael on August 06, 2019, 10:23:50 AM
The command format is:

# installp -d <directory-name> -a all ## to install everything in directory

Note: when directory-name is a file-name, it only looks at that one file.

To see what is in a directory (read, what installp sees as being offered) compare

# installd -d <file-name> -L

with


# inutoc <dir-name> # just to be sure the .toc aka table-of-contents data is current
# installp -d <dir-name> -L


Hope this helps!
Title: Re: Paramiko for Python3
Post by: Duckabush on August 09, 2019, 09:06:35 PM
Thanks, worked like a charm.
Title: Re: Paramiko for Python3
Post by: Duckabush on August 09, 2019, 09:19:54 PM
Okay, maybe it didn't nothing is showing up on /opt/bin at this point.
Title: Re: Paramiko for Python3
Post by: Michael on August 11, 2019, 10:09:18 AM
Try "# lslpp -f <fileset-name>".

There is a dependency on the 'latest' openssl.base (1.0.2.1601) for aixtools.python3.rte.

Please use: "# lslpp -L openssl.base" and let me know the result.
Title: Re: Paramiko for Python3
Post by: Duckabush on August 12, 2019, 08:03:44 PM
$ lslpp -L openssl.base
  Fileset                      Level  State  Type  Description (Uninstaller)
  ----------------------------------------------------------------------------
  openssl.base            1.0.2.1500    C     F    Open Secure Socket Layer


State codes:
 A -- Applied.
 B -- Broken.
 C -- Committed.
 E -- EFIX Locked.
 O -- Obsolete.  (partially migrated to newer version)
 ? -- Inconsistent State...Run lppchk -v.

Type codes:
 F -- Installp Fileset
 P -- Product
 C -- Component
 T -- Feature
 R -- RPM Package
E -- Interim Fix
Title: Re: Paramiko for Python3
Post by: Michael on August 13, 2019, 11:24:53 AM
Well, I had set, as mentioned before - version 1601 as the lowest level acceptable.

I'll lower the value and update the thread when it is ready.

p.s. The current latest (and recommended) level of openssl is openssl.base.1.0.2.1801. Read https://www.ibm.com/support/docview.wss?uid=isg3T1027135 for more info on how to get and install. Note: the release numbers in the article are not the latest!
Title: Re: Paramiko for Python3
Post by: Michael on August 13, 2019, 12:06:39 PM
Packaging updated to be less strict about openssl.base requirement. See wiki: http://www.aixtools.net/index.php/python3-3.7 for updated MD% checksum and package link.

I still recommend getting openssl updated (version 1.0.2.1601 was available since April, the latest - openssl.base.1.0.2.1801 is available since early this month)
Title: Re: Paramiko for Python3
Post by: Michael on August 14, 2019, 12:32:39 PM
Bother, %%^!, etc.

I re-built python using --enable-shared - which does not package perfectly. I am rebuilding, and I shall renumber the version to 3.7.4.1 to "cleanup" to ensure that any old packaging is replaced by the new packaging.

Used the wrong command from history (testing Python 3.9 builds). My apologies.

Title: Re: Paramiko for Python3
Post by: Duckabush on August 14, 2019, 06:38:24 PM
Looking at that IBM link, I don't think they would be too keen to update openssl and openssh. Too many extremely important keys are setup already and if something would go wrong. I don't believe the current admin would be willing to update it.

I will go through the process on another machine and see if I can get it done and feel comfortable. With the backup there shouldn't be much of an issue.
Title: Re: Paramiko for Python3
Post by: Michael on August 16, 2019, 11:41:10 AM
OpenSSL updates, in my experience, go nearly effortlessly. I have a BigFix sandbox, and I just apply the fixlet once it comes in, and my systems get updated. And I leave the fixlet "open" so that whenever I re/install a VM openssl is updated magically.

OpenSSH can be an issue, especially if you change release levels as the defaults, or even names (of options or option settings) change from release level to release level - which is why, I suspect, AIX was at release 6.0 of openssh for forever.

In case you had not noticed, I also package OpenSSH  - and I do so in a way that the AIX one is still installed, just not the default. And, should you decide to uninstall mine, it restores the AIX one as default. FYI: I also have the openssh-8.0 packaged (although I should check for a later 8.X version) - as long as you are testing.

***
In a different area - should actually start a new thread, so I shall, but mention it here - I am researching what is needed to have AIX wheels for Python modules. I have already found that it can work - there are just some details that need to be addressed to make it much better.