coreutils - RPM packages versus "AIXTOOLS" installp based

[Michael]

This discussion has it's origins coming from a discussion here: https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/1006
The problem is GNU run-time dependencies and how the GNU environment may change suddenly - and things break.
From the discussion the key issue seems to be here:

• TASK [TestIBM XL C] failed because it couldn't load module /opt/freeware/lib/libintl.a(libintl.so. due to dependent module /usr/lib/libiconv.a not containing libiconv.so.2.  @sxa555 directed me to a comment to the effect:

Code: [Select]

libiconv needs to be fixed, for reasons that are absolutely not clear to me, I removed /usr/lib/libiconv.a and symlinked it to /opt/freeware/lib/libiconv.a. After this, I had a couple of issues related to me running the playbook as a non-root user as my account wasn't able to access the ssh keys in /Vendor_Files/keys, so I took copies of them and  exported my own variables for Jenkins_User_SSHKey, Nagios_User_SSHKey, Zeus_User_SSHKey to get around that problem.  It turned out that I didn't have the necessary files for nagios though, so I ended up skipping the setup for both that and zeus
I want to focus on the highlighted text above - and while I may be wrong, my expectation is the AIX update to 7100-05-05 updated the the file /usr/lib/libiconv.a.On my AIX server at 7100-05-04 this file shows as:

root@x072:[/data/httpd/tools/gnu]ls -l /usr/lib/libiconv.a
-r--r--r--    1 bin      bin          790271 Oct 22 20:20 /usr/lib/libiconv.a
root@x072:[/data/httpd/tools/gnu]ar tv /usr/lib/libiconv.a
r--r--r--     2/2     237919 Jan 03 01:43 2019 shr4.o
r--r--r--     2/2     238041 Jan 03 01:43 2019 shr.o

root@x072:[/data/httpd/tools/gnu]oslevel -s
7100-05-04-1914
root@x072:[/data/httpd/tools/gnu]lslpp -w /usr/lib/libiconv.a
  File                                        Fileset               Type
  ----------------------------------------------------------------------------
  /usr/lib/libiconv.a                         bos.rte.iconv         File

The key thin to notice is that /usr/lib/libiconv.a is OWNED by AIX and installp - and if an OSS RPM overwrites it (perhaps saving what was it in before) installp does not know this. And when AIX updates a file like this, anything an RPM install has done is lost.
This is why I package with no/as little OSS as possible installed. Using GCC as the compiler forces anyone packaging to include things such as gettext (which requires GNU libiconv) and issues such as this as just waiting to happen.
In short, I have no issues with RPM and/or YUM as package managers. But I do have issues that most people using RPM to package, intentionally or not, overwrite key AIX files and either AIX breaks, or an OSS tool breaks during the update cycle. MUCH MUCH better to use only one package manager.

[Michael]

For anyone interested / I just repackaged coreuitls for AIX.
==============================
aixtools.gnu.coreutils:aixtools.gnu.coreutils.man.en_US:8.31.0.0::I:T:::::N:gnu coreutils man pages::::0::
aixtools.gnu.coreutils:aixtools.gnu.coreutils.rte:8.31.0.0::I:T:::::N:gnu coreutils 13-Dec-2019::::0::
aixtools.gnu.coreutils:aixtools.gnu.coreutils.share:8.31.0.0::I:T:::::N:gnu coreutils universal files::::0::
==============================

See http://www.aixtools.net/index.php/coreutils for more details

[sxa]

I'll try and stick all relevant information into this one message - if I've missed anything let me know.

It looks like it might be the same /usr/lib/libiconv.a on TL5SP5 as your TL5SP4 - it's the same size and the MD5 checksum is as follows, so I would expect the issue to be reproducible there too.

$ oslevel -s
7100-05-05-1939
$ ls -l /usr/lib/libiconv.a-DIST
-r--r--r--    1 root     system       790271 Dec 12 08:49 /usr/lib/libiconv.a-DIST
$ md5sum /usr/lib/libiconv.a
827dff9b3bcbc2d062b188f2c685d4eb  /usr/lib/libiconv.a


We haven't hit issues with previous versions installed by the toolbox in the past, but it seems like newer versions of things like coreutils (for example coreutils-8.29-4.ppc vs coreutils-8.29-3.ppc but it's the case with others - I think someone told me that git 2.18 vs 2.20 might have  the same dependency update)

I wouldn't focus on the /usr/lib/libiconv.a -> /opt/freeware/lib/libiconv.a symlink as that was removed - someone else suggested that privately to me as a solution but it broken my ability to build our source code against the system libiconv.a so I backed it out.

The issue we seem to have is that if /usr/lib is in the LIBPATH first (and running from a java agent such as jenkins continuous integration which is a fairly common use case does this automatically) then it fails, and it didn't with previous installations from the AIX toolbox. It looks like there is now a dependency of libiconv.so that has changed from shr4.o in earlier versions, so it doesn't work against the system installed libiconv any more, which is presumably a change in how it was built

$ /opt/freeware/bin/git --version
git version 2.20.1
$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/freeware/lib64/libintl.a(libintl.so.
         /opt/freeware/lib/libiconv.a(libiconv.so.2)
         /opt/freeware/lib/libz.a(libz.so.1)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
$ export LIBPATH=/usr/lib
$ /opt/freeware/bin/git --version
exec(): 0509-036 Cannot load program /opt/freeware/bin/git because of the following errors:
        0509-150   Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded.
        0509-152   Member libiconv.so.2 is not found in archive
$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libintl.a(libintl.so.
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.
dump: /tmp/tmpdir13041786/extract/libiconv.so.2: 0654-106 Cannot open the specified file.
         /usr/lib/libz.a(libz.so.1)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
$

On an older systems that I installed with an older version of git (this output was a system with an oslevel of 7200-02-02-1810 but I've got an earlier 7.1 system which  also works ok with the older tools)

-bash-5.0$ /opt/freeware/bin/git --version
git version 2.8.1
-bash-5.0$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.1)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(shr4.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
-bash-5.0$ export LIBPATH=/usr/lib
-bash-5.0$ /opt/freeware/bin/git --version
git version 2.8.1
-bash-5.0$ ldd /opt/freeware/bin/git
/opt/freeware/bin/git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.1)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(shr4.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)

On the working system it's linked against libiconv(shr4.o) which is in the system AIX libiconv.a and on the failing one it's linked against libiconv(libiconv.so.2). If I do the same test with /opt/freeware/bin/ls I get the same symptoms:

$ /opt/freeware/bin/ls --version
ls (GNU coreutils) 8.29
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Richard M. Stallman and David MacKenzie.
$ ldd /opt/freeware/bin/ls
/opt/freeware/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /opt/freeware/lib/libintl.a(libintl.so.
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/freeware/lib/libiconv.a(libiconv.so.2)
$ export LIBPATH=/usr/lib
$ /opt/freeware/bin/ls --version
exec(): 0509-036 Cannot load program /opt/freeware/bin/ls because of the following errors:
        0509-022 Cannot load module /opt/freeware/lib/libintl.a(libintl.so..
        0509-150   Dependent module /usr/lib/libiconv.a(libiconv.so.2) could not be loaded.
        0509-152   Member libiconv.so.2 is not found in archive
        0509-022 Cannot load module ls_64.
        0509-150   Dependent module /opt/freeware/lib/libintl.a(libintl.so. could not be loaded.
        0509-022 Cannot load module .
$ ldd /opt/freeware/bin/ls
/opt/freeware/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /opt/freeware/lib/libintl.a(libintl.so.
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.

In the case of the ls test above, my "working" system has ls from coreutils-8.29-3, the failing one 8.29-4 - if the only difference is the change of suffix then it is presumably a change that has been made in the AIX toolbox that has changed the dependency so it now depends in libiconv(libiconv.so.2) where it didn't before (Just for completeness on the working system with 8.29-3 /opt/freeware/bin/ls does not appear to have a dependency on libiconv.a at all) I don't think that either OSS or AIX packages are overwriting each other in this instance, but we do have a situation where we have both on the system in different locations and up til recently that hasn't caused a problem. Is it possible to remove the dependency on libiconv.so.2 and switch it back to the previous dependency of shr4.o?

[Michael]

My version of coreutils (this is the 8.29):
root@x072:[/]ldd /opt/bin/ls
/opt/bin/ls needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
And the git I currently use - also self-built - and should be online: though, still quite old (2016):
Note: it does need the GNU files, but I have them in /opt/lib (try to never touch /usr/lib) - and that LIBRARY PATH is included in the executable (/opt/lib/libxxx are my packages, /usr/lib is what comes from AIX).

michael@x071:[/home/michael]ldd /opt/bin/git
/opt/bin/git needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /opt/lib/libintl.a(libintl.so.
         /usr/lib/libcrypto.a(libcrypto64.so)
         /opt/lib/libz.a(libz.so.1.2.
         /opt/lib/libiconv.a(libiconv.so.2)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthread.a(shr_xpg5_64.o)
michael@x071:[/home/michael]ls -l /opt/bin/git
-rwxr-xr-x    1 bin      bin         4005696 Oct 18 2016  /opt/bin/git
michael@x071:[/home/michael]dump -H /opt/bin/git

/opt/bin/git:
dump: /opt/bin/git: 0654-108 file is not valid in the current object file mode.
        Use the -X option to specify the desired object mode.
michael@x071:[/home/michael]dump -X64 -H /opt/bin/git

/opt/bin/git:

                        ***Loader Section***
                      Loader Header Information
VERSION#         #SYMtableENT     #RELOCent        LENidSTR
0x00000001       0x000000de       0x00001485       0x000000b8

#IMPfilID        OFFidSTR         LENstrTBL        OFFstrTBL
0x00000007       0x00015d58       0x00000a48       0x00015e10


                        ***Import File Strings***
INDEX  PATH                          BASE                MEMBER
0      /opt/lib:/usr/vac/lib:/usr/lib:/lib
1                                    libc.a              shr_64.o
2                                    libpthreads.a       shr_xpg5_64.o
3                                    libintl.a           libintl.so.8
4                                    libcrypto.a         libcrypto64.so
5                                    libz.a              libz.so.1.2.8
6                                    libiconv.a          libiconv.so.2
The new git (2.24.1) I built yesterday does have some GNU dependencies: I have to "patchup" the system to see which ones are needed for the executable, versus what was needed for the build process. e.g., I am expecting libcurl to be needed, but ldd by itself won't show that.
root@x065:[/data/prj/aixtools/git-2.24.1]ldd ./git
./git needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthread.a(shr_xpg5.o)
         /usr/lib/libintl.a(libintl.so.
ar: 0707-109 Member name libintl.so.8 does not exist.
dump: /tmp/tmpdir217166/extract/libintl.so.8: 0654-106 Cannot open the specified file.
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libiconv.a(libiconv.so.2)
ar: 0707-109 Member name libiconv.so.2 does not exist.
dump: /tmp/tmpdir217166/extract/libiconv.so.2: 0654-106 Cannot open the specified file.
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)

[Michael]

So, I build on an AIX 5.3 TL7 system as this provides binary compatibility from that point on (this is also the builddate - 1 week, of AIX 6.1 TL0).
As libintl.a and libiconv.a regualrly give issues - when I must install them I install my version, add the IBM members to that archive (in read-only status); move the original to samename.AIX (so I restore them during an uninstall) and then claim their space. There are too many applications that, sadly, depend on these old versions.
After installing aixtools.gnu.libiconv and aixtools.gnu.gettext the libraries look like:
root@x065:[/data/prj/aixtools/git-2.24.1]lslpp -L | grep aixtools | grep rte
  aixtools.buildaix.rte      2.1.2.0    C     F    aixtools buildaix 01-May-2018
  aixtools.gnu.coreutils.rte
  aixtools.gnu.gettext.rte  0.19.8.1    C     F    built 21-Aug-2016 1821 UTC
  aixtools.gnu.libiconv.rte

root@x065:[/data/prj/aixtools/git-2.24.1]ls -l /usr/lib/libiconv.* /usr/lib/libintl.*
-rw-r--r--   1 bin      bin         2433032 Dec 13 13:53 /usr/lib/libiconv.a
-r--r--r--   1 bin      bin          379513 Mar 10 2015  /usr/lib/libiconv.a.AIX
lrwxrwxrwx   1 root     system           18 Dec 13 13:54 /usr/lib/libintl.a -> /opt/lib/libintl.a
lrwxrwxrwx   1 root     system           31 Oct 18 23:01 /usr/lib/libintl.a.AIX -> /usr/opt/freeware/lib/libintl.a

root@x065:[/data/prj/aixtools/git-2.24.1]ar tv /opt/lib/libintl.a
rwxr-xr-x     0/0      79727 Aug 21 18:17 2016 libintl.so.8
rw-r--r--     0/0       3753 Dec 13 13:54 2019 bindtextdom.o
rw-r--r--     0/0       1025 Dec 13 13:54 2019 dcgettext.o
rw-r--r--     0/0      10996 Dec 13 13:54 2019 dcigettext.o
rw-r--r--     0/0       1037 Dec 13 13:54 2019 dcngettext.o
rw-r--r--     0/0        996 Dec 13 13:54 2019 dgettext.o
rw-r--r--     0/0       1025 Dec 13 13:54 2019 dngettext.o
rw-r--r--     0/0       2926 Dec 13 13:54 2019 explodename.o
rw-r--r--     0/0       2397 Dec 13 13:54 2019 finddomain.o
rw-r--r--     0/0        992 Dec 13 13:54 2019 gettext.o
rw-r--r--     0/0       3683 Dec 13 13:54 2019 intl-compat.o
rw-r--r--     0/0       5416 Dec 13 13:54 2019 l10nflist.o
rw-r--r--     0/0       5007 Dec 13 13:54 2019 loadmsgcat.o
rw-r--r--     0/0       3219 Dec 13 13:54 2019 localcharset.o
rw-r--r--     0/0       4516 Dec 13 13:54 2019 localealias.o
rw-r--r--     0/0       1013 Dec 13 13:54 2019 ngettext.o
rw-r--r--     0/0       7400 Dec 13 13:54 2019 plural.o
rw-r--r--     0/0       1821 Dec 13 13:54 2019 textdomain.o
rwxr-xr-x     0/0      58410 Dec 13 13:54 2019 libintl.so.1
root@x065:[/data/prj/aixtools/git-2.24.1]ar tv /usr/opt/freeware/lib/libintl.a
rwxr-xr-x 10604/10000  58410 Dec 12 15:27 2005 libintl.so.1
rw-r--r-- 10604/10000   3683 Dec 12 15:27 2005 intl-compat.o
rw-r--r-- 10604/10000   3753 Dec 12 15:27 2005 bindtextdom.o
rw-r--r-- 10604/10000   1025 Dec 12 15:27 2005 dcgettext.o
rw-r--r-- 10604/10000    996 Dec 12 15:27 2005 dgettext.o
rw-r--r-- 10604/10000    992 Dec 12 15:27 2005 gettext.o
rw-r--r-- 10604/10000   2397 Dec 12 15:27 2005 finddomain.o
rw-r--r-- 10604/10000   5007 Dec 12 15:27 2005 loadmsgcat.o
rw-r--r-- 10604/10000   4516 Dec 12 15:27 2005 localealias.o
rw-r--r-- 10604/10000   1821 Dec 12 15:27 2005 textdomain.o
rw-r--r-- 10604/10000   5416 Dec 12 15:27 2005 l10nflist.o
rw-r--r-- 10604/10000   2926 Dec 12 15:27 2005 explodename.o
rw-r--r-- 10604/10000  10996 Dec 12 15:27 2005 dcigettext.o
rw-r--r-- 10604/10000   1037 Dec 12 15:27 2005 dcngettext.o
rw-r--r-- 10604/10000   1025 Dec 12 15:27 2005 dngettext.o
rw-r--r-- 10604/10000   1013 Dec 12 15:27 2005 ngettext.o
rw-r--r-- 10604/10000   7400 Dec 12 15:27 2005 plural.o
rw-r--r-- 10604/10000   3219 Dec 12 15:27 2005 localcharset.o

And since there are fewer members: also showing the 32+64 bit support:
root@x065:[/data/prj/aixtools/git-2.24.1]ar -Xany -tv /usr/lib/libiconv.a
rwxr-xr-x     0/0     1039231 Feb 11 10:34 2017 libiconv.so.2
rwxr-xr-x     0/0     1014113 Feb 11 15:34 2017 libiconv.so.2
rwxr-xr-x     0/0     112151 Dec 13 13:53 2019 shr.o
rwxr-xr-x     0/0     111901 Dec 13 13:53 2019 shr4.o
rwxr-xr-x     0/0     151987 Dec 13 13:53 2019 shr4_64.o
As to root cause - IMHO - this is because most, if not all, OSS packaged by IBM, oss4aix, Bull - use a GCC compiler - and the compiler needs GNU iconv and, iirc, gettext. And, OSS projects - if they see the GNU libraries they link against them. A compareable problem comes from GCC itself (and this may be the secret cause - a new libgc, or libcg - I forget it name - that GCC uses as it's preferred replacement for libc.a (so I am told).
In a GNU environment these are minor matters - the Linux distributor takes responsibility. With AIX, it becomes very difficult to spot which "3rd party" library is getting in the way.
In any case: I am curious to hear if my packaging would work in your environment. If not, I am willing to make it work - or come close to die trying! :p

[sxa]

Thanks - it looks like what you've got there with the modified /usr/lib/libiconv.a is what I need but don't currently have. Is the backing up to .AIX and putting the new one in place something that should be done automatically when the libiconv RPM package is installed? That sounds pretty similar to what I've done (Taken the plain AIX one and added in libiconv.so.2 into it) so I'm not entirely clear how that didn't happen on my systems unless that's a bit you've just been adding :-)

Also is /opt/lib and /opt/bin something that you do on your local machine to keep it separate from the "production" yum installed stuff in /opt/freeware/{bin/lib}

Totally agree with keeping the GCC and AIX stuff separate as far as possible. Ref the GNU C/C++ standard libraries, quite a few open source projects end up statically linking them in to avoid those problems.

[Michael]

Yes - I have that in my post-install script. Perhaps between -3 and -4 packaging they forgot to keep that in the RPM post install stuff.
I have never seen an RPM save an AIX library so it could be uninstalled later (says cannot be uninstalled instead).
Also, coreutils should not be touching /usr/lib/libintl.a or /usr/lib/libiconv.a - if that is what you meant.
rpm -qa output and I can do some additional research.
Yes, opt/bin and /opt/lib are used so I do not step on AIX or freeware. In fact, I SHOULD be using /opt/aixtools/{bin|lib}. I am considering making that change to my package scripts, and then adding symbolic links in /opt/bin, or /usr/bin. Again, as much as possible - I do not touch /usr/lib - I consider that to be "owned" by IBM-AIX. Anything I put there is liable to be removed by IBM (or overwritten).
As to getting "what I have" - you could consider looking at installing my packages from http://download.aixtools.net/gnu/aixtools.gnu.coreutils.8.31.0.0.I, http://download.aixtools.net/gnu/aixtools.gnu.libiconv.1.15.0.1.I, and http://download.aixtools.net/gnu/aixtools.gnu.gettext.0.19.8.1.I -- although, my coreutils does not need either GNU gettext or GNU libiconv

[sxa]

I'll have a look at that on a new WPAR - thanks.
So the .AIX backups/replacements are the only think you modify in /usr/lib which makes sense. I hadn't realised that you were even doing that, so good to know and I'll see what happens on my clean system :-)

[Michael]

Glad to hear others are using WPAR's!
Unlike RPM's that do not show their scripts when you unpack them (rpmcpio iirc), an installp file can be unpacked and examined using restore -Tqf <installp_file>.For GNU libiconv the file to extract is  ./usr/lpp/aixtools.gnu.libiconv/liblpp.a
The contents, in this case:
root@x072:[/tmp/coreutils]ar tv ./usr/lpp/aixtools.gnu.libiconv/liblpp.a
rw-r--r--     0/0        240 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.al
rw-r--r--     0/0       1824 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.invento ry
rw-r--r--     0/0         81 Aug 25 13:56 2017 aixtools.gnu.libiconv.man.en_US.size
rw-r--r--     0/0        214 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.al
rw-r--r--     0/0       1611 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.inventory
rw-r--r--     0/0         54 Aug 25 13:56 2017 aixtools.gnu.libiconv.share.size
rw-r--r--     0/0        209 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.al
rwxr-x---     0/0       3294 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.config
rw-r--r--     0/0       2113 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.inventory
rw-r--r--     0/0         79 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.size
rwxr-x---     0/0       1852 Aug 25 13:56 2017 aixtools.gnu.libiconv.rte.unconfig

And the scripts that are important, in this case, are aixtools.gnu.libiconv.rte.config and aixtools.gnu.libiconv.rte.unconfig.
In the config script:

ar -X32_64 -xv /usr/lib/libiconv.a.AIX shr.o shr4.o shr4_64.o >/dev/null
strip -X32_64 -e shr.o shr4.o shr4_64.o
slibclean
ar -X32_64 -ro /opt/lib/libiconv.a shr.o shr4.o shr4_64.o >/dev/null

[[ ! -z ${VERBOSE} ]] && \
print -- "copy new library to /usr/lib/libiconv.a"
rm -f /usr/lib/libiconv.a
cp -p /opt/lib/libiconv.a /usr/lib/libiconv.a

The strip -e command makes the loader ignore the files for NEW programs, but makes sure they are available for current programs that need them.
Hope this helps, or at least, is interesting!