Date: 18-01-20  Time: 13:20 PM

Recent Posts

Pages: 1 [2] 3 4 ... 10
11
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 13, 2019, 02:12:38 PM »
So, I build on an AIX 5.3 TL7 system as this provides binary compatibility from that point on (this is also the builddate - 1 week, of AIX 6.1 TL0).
As libintl.a and libiconv.a regualrly give issues - when I must install them I install my version, add the IBM members to that archive (in read-only status); move the original to samename.AIX (so I restore them during an uninstall) and then claim their space. There are too many applications that, sadly, depend on these old versions.
After installing aixtools.gnu.libiconv and aixtools.gnu.gettext the libraries look like:
root@x065:[/data/prj/aixtools/git-2.24.1]lslpp -L | grep aixtools | grep rte
  aixtools.buildaix.rte      2.1.2.0    C     F    aixtools buildaix 01-May-2018
  aixtools.gnu.coreutils.rte
  aixtools.gnu.gettext.rte  0.19.8.1    C     F    built 21-Aug-2016 1821 UTC
  aixtools.gnu.libiconv.rte

root@x065:[/data/prj/aixtools/git-2.24.1]ls -l /usr/lib/libiconv.* /usr/lib/libintl.*
-rw-r--r--   1 bin      bin         2433032 Dec 13 13:53 /usr/lib/libiconv.a
-r--r--r--   1 bin      bin          379513 Mar 10 2015  /usr/lib/libiconv.a.AIX
lrwxrwxrwx   1 root     system           18 Dec 13 13:54 /usr/lib/libintl.a -> /opt/lib/libintl.a
lrwxrwxrwx   1 root     system           31 Oct 18 23:01 /usr/lib/libintl.a.AIX -> /usr/opt/freeware/lib/libintl.a

root@x065:[/data/prj/aixtools/git-2.24.1]ar tv /opt/lib/libintl.a
rwxr-xr-x     0/0      79727 Aug 21 18:17 2016 libintl.so.8
rw-r--r--     0/0       3753 Dec 13 13:54 2019 bindtextdom.o
rw-r--r--     0/0       1025 Dec 13 13:54 2019 dcgettext.o
rw-r--r--     0/0      10996 Dec 13 13:54 2019 dcigettext.o
rw-r--r--     0/0       1037 Dec 13 13:54 2019 dcngettext.o
rw-r--r--     0/0        996 Dec 13 13:54 2019 dgettext.o
rw-r--r--     0/0       1025 Dec 13 13:54 2019 dngettext.o
rw-r--r--     0/0       2926 Dec 13 13:54 2019 explodename.o
rw-r--r--     0/0       2397 Dec 13 13:54 2019 finddomain.o
rw-r--r--     0/0        992 Dec 13 13:54 2019 gettext.o
rw-r--r--     0/0       3683 Dec 13 13:54 2019 intl-compat.o
rw-r--r--     0/0       5416 Dec 13 13:54 2019 l10nflist.o
rw-r--r--     0/0       5007 Dec 13 13:54 2019 loadmsgcat.o
rw-r--r--     0/0       3219 Dec 13 13:54 2019 localcharset.o
rw-r--r--     0/0       4516 Dec 13 13:54 2019 localealias.o
rw-r--r--     0/0       1013 Dec 13 13:54 2019 ngettext.o
rw-r--r--     0/0       7400 Dec 13 13:54 2019 plural.o
rw-r--r--     0/0       1821 Dec 13 13:54 2019 textdomain.o
rwxr-xr-x     0/0      58410 Dec 13 13:54 2019 libintl.so.1
root@x065:[/data/prj/aixtools/git-2.24.1]ar tv /usr/opt/freeware/lib/libintl.a
rwxr-xr-x 10604/10000  58410 Dec 12 15:27 2005 libintl.so.1
rw-r--r-- 10604/10000   3683 Dec 12 15:27 2005 intl-compat.o
rw-r--r-- 10604/10000   3753 Dec 12 15:27 2005 bindtextdom.o
rw-r--r-- 10604/10000   1025 Dec 12 15:27 2005 dcgettext.o
rw-r--r-- 10604/10000    996 Dec 12 15:27 2005 dgettext.o
rw-r--r-- 10604/10000    992 Dec 12 15:27 2005 gettext.o
rw-r--r-- 10604/10000   2397 Dec 12 15:27 2005 finddomain.o
rw-r--r-- 10604/10000   5007 Dec 12 15:27 2005 loadmsgcat.o
rw-r--r-- 10604/10000   4516 Dec 12 15:27 2005 localealias.o
rw-r--r-- 10604/10000   1821 Dec 12 15:27 2005 textdomain.o
rw-r--r-- 10604/10000   5416 Dec 12 15:27 2005 l10nflist.o
rw-r--r-- 10604/10000   2926 Dec 12 15:27 2005 explodename.o
rw-r--r-- 10604/10000  10996 Dec 12 15:27 2005 dcigettext.o
rw-r--r-- 10604/10000   1037 Dec 12 15:27 2005 dcngettext.o
rw-r--r-- 10604/10000   1025 Dec 12 15:27 2005 dngettext.o
rw-r--r-- 10604/10000   1013 Dec 12 15:27 2005 ngettext.o
rw-r--r-- 10604/10000   7400 Dec 12 15:27 2005 plural.o
rw-r--r-- 10604/10000   3219 Dec 12 15:27 2005 localcharset.o

And since there are fewer members: also showing the 32+64 bit support:
root@x065:[/data/prj/aixtools/git-2.24.1]ar -Xany -tv /usr/lib/libiconv.a
rwxr-xr-x     0/0     1039231 Feb 11 10:34 2017 libiconv.so.2
rwxr-xr-x     0/0     1014113 Feb 11 15:34 2017 libiconv.so.2
rwxr-xr-x     0/0     112151 Dec 13 13:53 2019 shr.o
rwxr-xr-x     0/0     111901 Dec 13 13:53 2019 shr4.o
rwxr-xr-x     0/0     151987 Dec 13 13:53 2019 shr4_64.o

As to root cause - IMHO - this is because most, if not all, OSS packaged by IBM, oss4aix, Bull - use a GCC compiler - and the compiler needs GNU iconv and, iirc, gettext. And, OSS projects - if they see the GNU libraries they link against them. A compareable problem comes from GCC itself (and this may be the secret cause - a new libgc, or libcg - I forget it name - that GCC uses as it's preferred replacement for libc.a (so I am told).
In a GNU environment these are minor matters - the Linux distributor takes responsibility. With AIX, it becomes very difficult to spot which "3rd party" library is getting in the way.
In any case: I am curious to hear if my packaging would work in your environment. If not, I am willing to make it work - or come close to die trying! :p
12
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 13, 2019, 01:49:49 PM »
My version of coreutils (this is the 8.29):
root@x072:[/]ldd /opt/bin/ls
/opt/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
And the git I currently use - also self-built - and should be online: though, still quite old (2016):
Note: it does need the GNU files, but I have them in /opt/lib (try to never touch /usr/lib) - and that LIBRARY PATH is included in the executable (/opt/lib/libxxx are my packages, /usr/lib is what comes from AIX).

michael@x071:[/home/michael]ldd /opt/bin/git
/opt/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/lib/libintl.a(libintl.so.8)
         /usr/lib/libcrypto.a(libcrypto64.so)
         /opt/lib/libz.a(libz.so.1.2.8)
         /opt/lib/libiconv.a(libiconv.so.2)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthread.a(shr_xpg5_64.o)
michael@x071:[/home/michael]ls -l /opt/bin/git
-rwxr-xr-x    1 bin      bin         4005696 Oct 18 2016  /opt/bin/git
michael@x071:[/home/michael]dump -H /opt/bin/git

/opt/bin/git:
dump: /opt/bin/git: 0654-108 file is not valid in the current object file mode.
        Use the -X option to specify the desired object mode.
michael@x071:[/home/michael]dump -X64 -H /opt/bin/git

/opt/bin/git:

                        ***Loader Section***
                      Loader Header Information
VERSION#         #SYMtableENT     #RELOCent        LENidSTR
0x00000001       0x000000de       0x00001485       0x000000b8

#IMPfilID        OFFidSTR         LENstrTBL        OFFstrTBL
0x00000007       0x00015d58       0x00000a48       0x00015e10


                        ***Import File Strings***
INDEX  PATH                          BASE                MEMBER
0      /opt/lib:/usr/vac/lib:/usr/lib:/lib
1                                    libc.a              shr_64.o
2                                    libpthreads.a       shr_xpg5_64.o
3                                    libintl.a           libintl.so.8
4                                    libcrypto.a         libcrypto64.so
5                                    libz.a              libz.so.1.2.8
6                                    libiconv.a          libiconv.so.2
The new git (2.24.1) I built yesterday does have some GNU dependencies: I have to "patchup" the system to see which ones are needed for the executable, versus what was needed for the build process. e.g., I am expecting libcurl to be needed, but ldd by itself won't show that.
root@x065:[/data/prj/aixtools/git-2.24.1]ldd ./git
./git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.8)
ar: 0707-109 Member name libintl.so.8 does not exist.
dump: /tmp/tmpdir217166/extract/libintl.so.8: 0654-106 Cannot open the specified file.
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.
dump: /tmp/tmpdir217166/extract/libiconv.so.2: 0654-106 Cannot open the specified file.
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)


13
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by sxa on December 13, 2019, 01:31:04 PM »
I'll try and stick all relevant information into this one message - if I've missed anything let me know.

It looks like it might be the same /usr/lib/libiconv.a on TL5SP5 as your TL5SP4 - it's the same size and the MD5 checksum is as follows, so I would expect the issue to be reproducible there too.

$ oslevel -s
7100-05-05-1939
$ ls -l /usr/lib/libiconv.a-DIST
-r--r--r--    1 root     system       790271 Dec 12 08:49 /usr/lib/libiconv.a-DIST
$ md5sum /usr/lib/libiconv.a
827dff9b3bcbc2d062b188f2c685d4eb  /usr/lib/libiconv.a



We haven't hit issues with previous versions installed by the toolbox in the past, but it seems like newer versions of things like coreutils (for example coreutils-8.29-4.ppc vs coreutils-8.29-3.ppc but it's the case with others - I think someone told me that git 2.18 vs 2.20 might have  the same dependency update)

I wouldn't focus on the /usr/lib/libiconv.a -> /opt/freeware/lib/libiconv.a symlink as that was removed - someone else suggested that privately to me as a solution but it broken my ability to build our source code against the system libiconv.a so I backed it out.

The issue we seem to have is that if /usr/lib is in the LIBPATH first (and running from a java agent such as jenkins continuous integration which is a fairly common use case does this automatically) then it fails, and it didn't with previous installations from the AIX toolbox. It looks like there is now a dependency of libiconv.so that has changed from shr4.o in earlier versions, so it doesn't work against the system installed libiconv any more, which is presumably a change in how it was built

$ /opt/freeware/bin/git --version
git version 2.20.1
$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/freeware/lib64/libintl.a(libintl.so.8)
         /opt/freeware/lib/libiconv.a(libiconv.so.2)
         /opt/freeware/lib/libz.a(libz.so.1)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
$ export LIBPATH=/usr/lib
$ /opt/freeware/bin/git --version
exec(): 0509-036 Cannot load program /opt/freeware/bin/git because of the following errors:
        0509-150   Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded.
        0509-152   Member libiconv.so.2 is not found in archive
$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libintl.a(libintl.so.8)
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.
dump: /tmp/tmpdir13041786/extract/libiconv.so.2: 0654-106 Cannot open the specified file.
         /usr/lib/libz.a(libz.so.1)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
$


On an older systems that I installed with an older version of git (this output was a system with an oslevel of 7200-02-02-1810 but I've got an earlier 7.1 system which  also works ok with the older tools)

-bash-5.0$ /opt/freeware/bin/git --version
git version 2.8.1
-bash-5.0$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.1)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(shr4.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
-bash-5.0$ export LIBPATH=/usr/lib
-bash-5.0$ /opt/freeware/bin/git --version
git version 2.8.1
-bash-5.0$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.1)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(shr4.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)


On the working system it's linked against libiconv(shr4.o) which is in the system AIX libiconv.a and on the failing one it's linked against libiconv(libiconv.so.2). If I do the same test with /opt/freeware/bin/ls I get the same symptoms:

$ /opt/freeware/bin/ls --version
ls (GNU coreutils) 8.29
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.
$ ldd /opt/freeware/bin/ls
/opt/freeware/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /opt/freeware/lib/libintl.a(libintl.so.8)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/freeware/lib/libiconv.a(libiconv.so.2)
$ export LIBPATH=/usr/lib
$ /opt/freeware/bin/ls --version
exec(): 0509-036 Cannot load program /opt/freeware/bin/ls because of the following errors:
        0509-022 Cannot load module /opt/freeware/lib/libintl.a(libintl.so.8).
        0509-150   Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded.
        0509-152   Member libiconv.so.2 is not found in archive
        0509-022 Cannot load module ls_64.
        0509-150   Dependent module /opt/freeware/lib/libintl.a(libintl.so.8) could not be loaded.
        0509-022 Cannot load module .
$ ldd /opt/freeware/bin/ls
/opt/freeware/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /opt/freeware/lib/libintl.a(libintl.so.
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.


In the case of the ls test above, my "working" system has ls from coreutils-8.29-3, the failing one 8.29-4 - if the only difference is the change of suffix then it is presumably a change that has been made in the AIX toolbox that has changed the dependency so it now depends in libiconv(libiconv.so.2) where it didn't before (Just for completeness on the working system with 8.29-3 /opt/freeware/bin/ls does not appear to have a dependency on libiconv.a at all) I don't think that either OSS or AIX packages are overwriting each other in this instance, but we do have a situation where we have both on the system in different locations and up til recently that hasn't caused a problem. Is it possible to remove the dependency on libiconv.so.2 and switch it back to the previous dependency of shr4.o?
14
AIXTOOLS / Re: coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 13, 2019, 12:48:49 PM »
For anyone interested / I just repackaged coreuitls for AIX.
==============================
aixtools.gnu.coreutils:aixtools.gnu.coreutils.man.en_US:8.31.0.0::I:T:::::N:gnu coreutils man pages::::0::
aixtools.gnu.coreutils:aixtools.gnu.coreutils.rte:8.31.0.0::I:T:::::N:gnu coreutils 13-Dec-2019::::0::
aixtools.gnu.coreutils:aixtools.gnu.coreutils.share:8.31.0.0::I:T:::::N:gnu coreutils universal files::::0::
==============================

See http://www.aixtools.net/index.php/coreutils for more details
15
AIXTOOLS / coreutils - RPM packages versus "AIXTOOLS" installp based
« Last post by Michael on December 13, 2019, 12:36:12 PM »
This discussion has it's origins coming from a discussion here: https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/1006
The problem is GNU run-time dependencies and how the GNU environment may change suddenly - and things break.
From the discussion the key issue seems to be here:
    TASK [TestIBM XL C] failed because it couldn't load module /opt/freeware/lib/libintl.a(libintl.so.8) due to dependent module /usr/lib/libiconv.a not containing libiconv.so.2.  @sxa555 directed me to a comment to the effect:
Code: [Select]
libiconv needs to be fixed, for reasons that are absolutely not clear to me, I removed /usr/lib/libiconv.a and symlinked it to /opt/freeware/lib/libiconv.a. After this, I had a couple of issues related to me running the playbook as a non-root user as my account wasn't able to access the ssh keys in /Vendor_Files/keys, so I took copies of them and  exported my own variables for Jenkins_User_SSHKey, Nagios_User_SSHKey, Zeus_User_SSHKey to get around that problem.  It turned out that I didn't have the necessary files for nagios though, so I ended up skipping the setup for both that and zeus
I want to focus on the highlighted text above - and while I may be wrong, my expectation is the AIX update to 7100-05-05 updated the the file /usr/lib/libiconv.a.On my AIX server at 7100-05-04 this file shows as:

root@x072:[/data/httpd/tools/gnu]ls -l /usr/lib/libiconv.a
-r--r--r--    1 bin      bin          790271 Oct 22 20:20 /usr/lib/libiconv.a
root@x072:[/data/httpd/tools/gnu]ar tv /usr/lib/libiconv.a
r--r--r--     2/2     237919 Jan 03 01:43 2019 shr4.o
r--r--r--     2/2     238041 Jan 03 01:43 2019 shr.o

root@x072:[/data/httpd/tools/gnu]oslevel -s
7100-05-04-1914
root@x072:[/data/httpd/tools/gnu]lslpp -w /usr/lib/libiconv.a
  File                                        Fileset               Type
  ----------------------------------------------------------------------------
  /usr/lib/libiconv.a                         bos.rte.iconv         File

The key thin to notice is that /usr/lib/libiconv.a is OWNED by AIX and installp - and if an OSS RPM overwrites it (perhaps saving what was it in before) installp does not know this. And when AIX updates a file like this, anything an RPM install has done is lost.
This is why I package with no/as little OSS as possible installed. Using GCC as the compiler forces anyone packaging to include things such as gettext (which requires GNU libiconv) and issues such as this as just waiting to happen.
In short, I have no issues with RPM and/or YUM as package managers. But I do have issues that most people using RPM to package, intentionally or not, overwrite key AIX files and either AIX breaks, or an OSS tool breaks during the update cycle. MUCH MUCH better to use only one package manager.
16
AIXTOOLS / Re: curl 7.67
« Last post by Michael on November 26, 2019, 10:15:48 PM »
Thanks for the update.

Let me know if I can help further.
17
AIXTOOLS / Re: curl 7.67
« Last post by quadrazontal on November 26, 2019, 08:44:44 PM »
Thank you. We have a symlink in /usr/lib going to the /opt/lib/libcurl.a that is included in the aixtools lpp package. The Manhattan error message was presented after installing the latest ssh/ssl from IBM and was resolved by backing out the ssl/ssh updates.  We can look to approach IBM as well regarding the statement "I see that IBM has changed the versions without a number to be the ABI 1.0.2 - and that is probably what is breaking things".  Otherwise Manhattan should be able to help our app team by digging into their own libOptilib.so.  -JP
18
AIXTOOLS / Re: PYTHON INSTALLATION
« Last post by Michael on November 26, 2019, 07:41:38 AM »
I am assuming this is not Python related - so I worked on an answer here: http://forums.rootvg.net/administration/what-ksh-shell-(version)-am-i-using/msg2972/#msg2972
19
Administration / What ksh shell (version) am I using?
« Last post by Michael on November 26, 2019, 07:40:37 AM »
The default shell on AIX is /usr/bin/ksh, and it's version is, by definition ksh88 (POSIX norm).

Several other shell behaviors are emulated by this binary based on the program (argv[0]) name - via hard-links.

Code: [Select]
root@x064:[/home/root]find /usr -inum 603 -xdev -ls
  603  289 -r-xr-xr-x  5 bin       bin         294926 Nov 29  2017 /usr/bin/ksh
  603  289 -r-xr-xr-x  5 bin       bin         294926 Nov 29  2017 /usr/bin/psh
  603  289 -r-xr-xr-x  5 bin       bin         294926 Nov 29  2017 /usr/bin/rksh
  603  289 -r-xr-xr-x  5 bin       bin         294926 Nov 29  2017 /usr/bin/sh
  603  289 -r-xr-xr-x  5 bin       bin         294926 Nov 29  2017 /usr/bin/tsh

There is a second ksh version named ksh93.

Code: [Select]
find /usr -inum 91 -xdev -ls
   91 2250 -r-xr-xr-x  2 bin       bin        2303098 Nov 14  2017 /usr/bin/ksh93
   91 2250 -r-xr-xr-x  2 bin       bin        2303098 Nov 14  2017 /usr/bin/rksh93

This version supports additional options that ksh(88) does not.

I further recommend reading: https://en.wikipedia.org/wiki/KornShell - for history, and the AIX documentation (currently) at:

Finally, I also noticed this discussion on ksh and ksh93: https://unix.stackexchange.com/questions/199900/how-can-i-safely-get-the-version-of-ksh
20
AIXTOOLS / Re: PYTHON INSTALLATION
« Last post by tbramha on November 25, 2019, 02:41:13 PM »
What is the command to get the shell version?
Pages: 1 [2] 3 4 ... 10