Date: 18-01-20  Time: 11:04 AM

Recent Posts

Pages: [1] 2 3 ... 10
1
AIXTOOLS / Python and wheels continuing story
« Last post by Michael on December 29, 2019, 06:58:18 PM »
While it may take some investigation to get environment variables correct - eventually this does work!
For example - one of the scientific Python modules - numpy.

root@x069:[/python]pip wheel numpy
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/31/0a/5df350c29a06835d534a6c4f5681075304da38d85f1c69e5226a635a67ce/numpy-1.18.0.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: numpy
  Building wheel for numpy (PEP 517) ... done
  Created wheel for numpy: filename=numpy-1.18.0-cp37-cp37m-AIX_7104_1845_64.whl size=7130503 sha256=b5456e8aefb80a0ff714532d697f4f115bcbe7d8bd2871593c0bb0b93cac10fd
  Stored in directory: /python
Successfully built numpy

Wheels means no compiler needed for later installs and an installation in a virtualenv is also possible (aka python -m pip --user ....)
2
Administration / Re: Christmas cleaning...
« Last post by Michael on December 20, 2019, 01:16:55 PM »
So, after the reboot I needed to reinstall aixtools.curl.adt (for the library, not the programs). Apache (rather php) would not load:

root@x071:[/root]apachectl -thttpd: Syntax error on line 116 of /var/httpd/etc/httpd.conf: Cannot load /opt/httpd/libexec/libphp5.so into server: \t0509-022 Cannot load module /opt/httpd/libexec/libphp5.so.\n\t0509-150   Dependent module libcurl.a(libcurl.so.4) could not be loaded.\n\t0509-022 Cannot load module libcurl.a(libcurl.so.4).\n\t0509-026 System error: A file or directory in the path name does not exist.\n\t0509-022 Cannot load module /opt/httpd/libexec/libphp5.so.\n\t0509-150   Dependent module /opt/httpd/libexec/libphp5.so could not be loaded.

All in all, satisfied with the spring, OOPS! Christmas - cleaning.
And, I'll move my OSS editing to a different server - away from my "services" side.
3
Administration / Re: Christmas cleaning...
« Last post by Michael on December 20, 2019, 12:47:40 PM »
Brought the list down to this. Everything seems okay. Will reboot to really test that all executable's are actually removed and out of memory
root@x071:[/root]lslpp -L | grep aixtools | grep rte  aixtools.apache.httpd.rte
  aixtools.blockip.rte       1.0.0.0    C     F    blockip 1.0.0.0 2014.Jan.20
  aixtools.buildaix.rte      2.1.2.1    C     F    buildaix 17-Dec-2019
  aixtools.expat.rte         2.2.5.0    C     F    aixtools expat 12-Nov-2017
  aixtools.freetype.rte     2.4.10.0    C     F    freetype version 2.4.10 for
  aixtools.libpng.rte       1.5.12.0    C     F    libpng version 1.5.12 for
  aixtools.lynx2.rte         8.8.0.0    C     F    lynx2 1502 0116 1533
  aixtools.openbsd.openssh.rte
  aixtools.php.rte          5.2.17.0    C     F    php 2014.Jun.20 19.37 CUT
  aixtools.python.rte       2.7.16.0    C     F    python 04-Mar-2019
  aixtools.unzip.rte         6.0.0.0    C     F    aixtools unzip 21-Jun-2017
  aixtools.xmlsoft.libxml2.rte
  aixtools.xmlsoft.libxslt.rte
  aixtools.zlib.rte         1.2.11.1    C     F    aixtools zlib 02-Feb-2017
4
Administration / Christmas cleaning...
« Last post by Michael on December 20, 2019, 12:30:36 PM »
Not really something for an article - although I should write a short BLOG to reference back to this.
I have been compiling opensource for several years - starting with - sometimes trying and failing - for 20 years. And packaging goes back 10 years ago - initial attempts - and 7 to 8 years ago with more automation.
And, at some point I have installed something on my 'core' server, that I run AIXTOOLS and ROOTVG on - and, imho - it has gotten messy. That is, I do not actually use all these tools (all the time) - and there are packages installed that I am not using - ever.
As I tell clients when doing a security assessment - remove software you are not using. And if you are using it - is it patched to latest levels?
So, my Christmas cleaning - I have some vacation time to look into this.
To keep the list somewhat short - I am only showing the "rte" run-time environment fileset. In AIXTOOLS packaging this includes the binaries. Currently I have:
root@x071:[/root]lslpp -L | grep aixtools | grep rte
  aixtools.apache.httpd.rte
  aixtools.attcp.rte         0.6.8.0    C     F    aixtools attcp 22-Feb-2018
  aixtools.bison.rte         3.0.4.0    C     F    bison 1504 0125 1445
  aixtools.blockip.rte       1.0.0.0    C     F    blockip 1.0.0.0 2014.Jan.20
  aixtools.buildaix.rte      2.1.2.1    C     F    buildaix 17-Dec-2019
  aixtools.cmake.rte         3.5.1.0    C     F    built 26-Mar-2016 1545 UTC
  aixtools.curl.rte         7.66.0.0    C     F    aixtools curl 24-Sep-2019
  aixtools.expat.rte         2.2.5.0    C     F    aixtools expat 12-Nov-2017
  aixtools.freetype.rte     2.4.10.0    C     F    freetype version 2.4.10 for
  aixtools.git.rte          2.10.1.0    C     F    aixtools git 19-Oct-2016
  aixtools.gmp.rte           5.0.5.0    C     F    gmp version 5.0.5 for powerpc
  aixtools.gnu.autoconf.rte
  aixtools.gnu.automake.rte
  aixtools.gnu.bash.rte     4.4.19.0    C     F    gnu bash 07-Feb-2018
  aixtools.gnu.diffutils.rte
  aixtools.gnu.gettext.rte  0.19.8.1    C     F    built 21-Aug-2016 1821 UTC
  aixtools.gnu.grep.rte     2.27.0.0    C     F    gnu grep 07-Dec-2016
  aixtools.gnu.help2man.rte
  aixtools.gnu.less.rte     4.81.0.0    C     F    gnu less 21-Nov-2016
  aixtools.gnu.libiconv.rte
  aixtools.gnu.m4.rte       1.4.17.0    C     F    1523 0613 1221
  aixtools.gnu.make.rte      4.1.0.0    C     F    1523 0613 1158
  aixtools.gnu.readline.rte  7.0.0.0    C     F    gnu readline 02-Nov-2016
  aixtools.gnu.wget.rte     1.19.4.0    C     F    gnu wget 17-Apr-2018
  aixtools.gnupg.rte        1.4.15.0    C     F    gnupg 1.4.15.0 2013.Oct.29
  aixtools.gperf.rte         3.0.4.0    C     F    gperf 1508 0227 0914
  aixtools.gtar.rte         1.26.0.0    C     F    gtar 1.26.0.0 2012.Dec.24
  aixtools.idsldap.rte       0.1.2.0    C     F    built 25-Mar-2016 1437 UTC
  aixtools.libee.rte         0.4.1.0    C     F    1539 1001 0632
  aixtools.libestr.rte      0.1.10.0    C     F    1539 0930 1938
  aixtools.libffi.rte        3.2.1.0    C     F    aixtools libffi 02-Jan-2017
  aixtools.libperfstat.rte   6.1.6.1    C     F    libperfstat 6.1.6.1
  aixtools.libpng.rte       1.5.12.0    C     F    libpng version 1.5.12 for
  aixtools.libsodium.rte     1.0.8.0    C     F    built 20-Mar-2016 2209 UTC
  aixtools.libtool.rte       2.4.6.0    C     F    libtool 1507 0218 0736
  aixtools.lynx2.rte         8.8.0.0    C     F    lynx2 1502 0116 1533
  aixtools.mariadb.rte      5.1.67.0    C     F    built 26-Mar-2016 1414 UTC
  aixtools.mpc.rte           1.0.2.0    C     F    mpc 2014.Oct.31 15.44 CUT
  aixtools.mpfr.rte          2.4.2.0    C     F    mpfr 2014.Oct.31 15.19 CUT
  aixtools.openbsd.openssh.rte
  aixtools.patch.rte         2.7.0.0    C     F    patch 2.7.0.0 2012.Oct.05
  aixtools.perl.rte         5.14.4.0    C     F    perl 2014.Sep.25 19.50 CUT
  aixtools.php.rte          5.2.17.0    C     F    php 2014.Jun.20 19.37 CUT
  aixtools.pkgconfig.rte    0.25.0.0    C     F    pkgconfig 0.25.0.0 2013.Oct.28
  aixtools.python.rte       2.7.16.0    C     F    python 04-Mar-2019
  aixtools.samba.rsync.rte   3.1.2.0    C     F    samba rsync 03-Nov-2016
  aixtools.sqlite3.rte      3.21.0.0    C     F    aixtools sqlite3 10-Jan-2018
  aixtools.unzip.rte         6.0.0.0    C     F    aixtools unzip 21-Jun-2017
  aixtools.xmlsoft.libxml2.rte
  aixtools.xmlsoft.libxslt.rte
  aixtools.xz.rte            5.2.2.0    C     F    built 22-Aug-2016 2005 UTC
  aixtools.zlib.rte         1.2.11.1    C     F    aixtools zlib 02-Feb-2017

The things I am going to start with removing are:
    ATTCP - AIX Test TCP
    BISON
    CMAKE
    GMP (should be gnu.gmp which makes me think it is a leftover from gcc testing)
    GNU stuff I don't use often, and probably should not use here
    * autoconf, automake, libtool (aka autotools)
    * bash
    * diffutils
    * gettext (if I can!)
    * grep (thought I needed it for git, but that seems to only be needed during packaging of git)
    * help2man (again, when I was building on this system - but no more)
    * less
    * libiconv (again, if I can)
    * readline
    GNUPG
    GPERF (whatever that is)
    GTAR (now packaged as gnu.tar, so also an ancient version to be removed)
    IDSLDAP (these are scripts I wrote to install LDAP sandboxes, not for here!)
    LIBEE (What is this?)
    LIBESTR (also, what is this!?)
    LIBFFI - should not need this - may wait just a bit, to see what uses it
    LIBPERFSTAT - not sure why I packaged my own libperfstat. Time to remove!
    LIBSODIUM - wow - back when I was trying to build salt and understand why python was not working as expected!
    MAKE - this is now gnu.make - which I would also remove
    MPC - like GMP and MPFR (below) - historical packaging!
    MPFR (see MPC)
    PATCH (would now be gnu.patch, so very old)
    PERL (one of my older projects as well, time to go)
    SAMBA.RSYNC - aka RSYNC. Needs an update, remove for now
    SQLITE3 - maybe I am using this, but I expect it is here only for a test done and forgotten
Packages that I know I need to update:
    XZ - this I think  I already have
    GIT - repackaged yesterday, and one of the reasons I am removing oldish/unused packages
    ZLIB - need to double check what the latest level is
Packages I want to get updated reasonably soon:
    apache aka httpd
    php
    The things these two need
As I move along in this - I'll post on accidents (dependencies I should have known about, e.g.). All in the life of OSS and packaging (as a hobby).
Best wishes for the holidays!

5
AIXTOOLS / Latest packaging of "stuff"
« Last post by Michael on December 18, 2019, 09:32:17 AM »
I unsticky-ed my previous "All-in-one-list" and am going to start making several new ones.
Currently I am working on "static" versions of the core GNU dependencies - GNU GETTEXT and the GNU ICONV that gettext depends on.
I'll have these packaged as static.gnu.libiconv and static.gnu.gettext. And, if this works out as I hope, I'll also make them available for anyone wishing to link against them - rather than against shared libraries.
FYI: I do try and watch twitter (as @rootvgnet), but you are likely to get my attention sooner by registering here, and posting a question or comment.
6
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 14, 2019, 04:15:50 PM »
Glad to hear others are using WPAR's!
Unlike RPM's that do not show their scripts when you unpack them (rpmcpio iirc), an installp file can be unpacked and examined using restore -Tqf <installp_file>.For GNU libiconv the file to extract is  ./usr/lpp/aixtools.gnu.libiconv/liblpp.a
The contents, in this case:
root@x072:[/tmp/coreutils]ar tv ./usr/lpp/aixtools.gnu.libiconv/liblpp.a
rw-r--r--     0/0        240 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.al
rw-r--r--     0/0       1824 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.invento ry
rw-r--r--     0/0         81 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.size
rw-r--r--     0/0        214 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.al
rw-r--r--     0/0       1611 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.inventory
rw-r--r--     0/0         54 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.size
rw-r--r--     0/0        209 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.al
rwxr-x---     0/0       3294 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.config
rw-r--r--     0/0       2113 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.inventory
rw-r--r--     0/0         79 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.size
rwxr-x---     0/0       1852 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.unconfig

And the scripts that are important, in this case, are aixtools.gnu.libiconv.rte.config and aixtools.gnu.libiconv.rte.unconfig.
In the config script:
Quote
ar -X32_64 -xv /usr/lib/libiconv.a.AIX shr.o shr4.o shr4_64.o >/dev/null
strip -X32_64 -e shr.o shr4.o shr4_64.o
slibclean
ar -X32_64 -ro /opt/lib/libiconv.a shr.o shr4.o shr4_64.o >/dev/null

[[ ! -z ${VERBOSE} ]] && \
print -- "copy new library to /usr/lib/libiconv.a"
rm -f /usr/lib/libiconv.a
cp -p /opt/lib/libiconv.a /usr/lib/libiconv.a
The strip -e command makes the loader ignore the files for NEW programs, but makes sure they are available for current programs that need them.
Hope this helps, or at least, is interesting!
7
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by sxa on December 13, 2019, 03:51:45 PM »
I'll have a look at that on a new WPAR - thanks.
So the .AIX backups/replacements are the only think you modify in /usr/lib which makes sense. I hadn't realised that you were even doing that, so good to know and I'll see what happens on my clean system :-)
8
AIXTOOLS / git-2.24.1 for AIX - Low on Dependencies
« Last post by Michael on December 13, 2019, 03:34:24 PM »
    Finished my build of git-2.24.1 for AIX. See http://www.aixtools.net/index.php/git
    I already had it done - BUT - default packaging of git has several dependencies.
    To minimize installation issues I am first built the dependencies libraries as static libraries. While the git executable is slightly larger there will be no 3rd party dependencies that can break over time.
    The dependencies and their versions embedded into this git are: libssh2-1.9, libcurl-7.67, libexpat-2.2.9, libintl (gettext-0.20.1) and libiconv-1.16.0.
    Been using it for nearly 2 weeks - I have not had any issues.
9
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 13, 2019, 03:19:53 PM »
Yes - I have that in my post-install script. Perhaps between -3 and -4 packaging they forgot to keep that in the RPM post install stuff.
I have never seen an RPM save an AIX library so it could be uninstalled later (says cannot be uninstalled instead).
Also, coreutils should not be touching /usr/lib/libintl.a or /usr/lib/libiconv.a - if that is what you meant.
rpm -qa output and I can do some additional research.
Yes, opt/bin and /opt/lib are used so I do not step on AIX or freeware. In fact, I SHOULD be using /opt/aixtools/{bin|lib}. I am considering making that change to my package scripts, and then adding symbolic links in /opt/bin, or /usr/bin. Again, as much as possible - I do not touch /usr/lib - I consider that to be "owned" by IBM-AIX. Anything I put there is liable to be removed by IBM (or overwritten).
As to getting "what I have" - you could consider looking at installing my packages from http://download.aixtools.net/gnu/aixtools.gnu.coreutils.8.31.0.0.I, http://download.aixtools.net/gnu/aixtools.gnu.libiconv.1.15.0.1.I, and http://download.aixtools.net/gnu/aixtools.gnu.gettext.0.19.8.1.I -- although, my coreutils does not need either GNU gettext or GNU libiconv
10
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by sxa on December 13, 2019, 02:49:36 PM »
Thanks - it looks like what you've got there with the modified /usr/lib/libiconv.a is what I need but don't currently have. Is the backing up to .AIX and putting the new one in place something that should be done automatically when the libiconv RPM package is installed? That sounds pretty similar to what I've done (Taken the plain AIX one and added in libiconv.so.2 into it) so I'm not entirely clear how that didn't happen on my systems unless that's a bit you've just been adding :-)

Also is /opt/lib and /opt/bin something that you do on your local machine to keep it separate from the "production" yum installed stuff in /opt/freeware/{bin/lib}

Totally agree with keeping the GCC and AIX stuff separate as far as possible. Ref the GNU C/C++ standard libraries, quite a few open source projects end up statically linking them in to avoid those problems.
Pages: [1] 2 3 ... 10