curl 7.67

[quadrazontal]

Hi. 
We updated ssl/ssh to the following versions which caused issues with curl, error messages pointing back to ssl library.  Is there a possibility to rebuild?
-JP

openssl.base            1.0.2.2000  COMMITTED  Open Secure Socket Layer
  openssl.license         1.0.2.2000  COMMITTED  Open Secure Socket License
  openssl.man.en_US       1.0.2.2000  COMMITTED  Open Secure Socket Layer
  openssl.base            1.0.2.2000  COMMITTED  Open Secure Socket Layer
 
  openssh.base.client   7.5.102.2000  COMMITTED  Open Secure Shell Commands
  openssh.base.server   7.5.102.2000  COMMITTED  Open Secure Shell Server
  openssh.license       7.5.102.2000  COMMITTED  Open Secure Shell License
  openssh.man.en_US     7.5.102.2000  COMMITTED  Open Secure Shell
  openssh.msg.EN_US     7.5.102.2000  COMMITTED  Open Secure Shell Messages -
  openssh.msg.en_US     7.5.102.2000  COMMITTED  Open Secure Shell Messages -
  openssh.base.client   7.5.102.2000  COMMITTED  Open Secure Shell Commands
  openssh.base.server   7.5.102.2000  COMMITTED  Open Secure Shell Server

[Michael]

Yes, I can rebuild. But I'll have to look at what I had linked to before, and how the members were named before, and understand what is going on. My initial check (which follows) does not show anything failing. -- Skip to bottom for my questions.


These are the contents with openssl.base.1.0.2.1000

root@x065:[/data/prj/aixtools/curl/curl-7.67.0]ar tv /usr/lib/libcrypto.a
rwxr-xr-x 537912/767508 3042942 Oct 18 11:38 2016 libcrypto.so
rwxr-xr-x 537912/767508 3042942 Oct 18 11:38 2016 libcrypto.so.1.0.0
rwxr-xr-x 537912/767508 2257647 Oct 18 11:39 2016 libcrypto.so.0.9.8
root@x065:[/data/prj/aixtools/curl/curl-7.67.0]ar tv /usr/lib/libssl.a
rwxr-xr-x 537912/767508 726474 Oct 18 11:38 2016 libssl.so
rwxr-xr-x 537912/767508 726474 Oct 18 11:38 2016 libssl.so.1.0.0
rwxr-xr-x 537912/767508 510610 Oct 18 11:39 2016 libssl.so.0.9.8


And for
root@x067:[/]lslpp -L | grep openssl
  openssl.base            1.0.2.1801    C     F    Open Secure Socket Layer
 
root@x067:[/]ar tv /usr/lib/libcrypto.a
rwxr-xr-x     0/0     3061022 Jun 03 11:01 2019 libcrypto.so
rwxr-xr-x     0/0     2186744 Jun 03 11:01 2019 libcrypto.so.0.9.8
rwxr-xr-x     0/0     3061022 Jun 03 11:01 2019 libcrypto.so.1.0.0
rwxr-xr-x     0/0     3061022 Jun 03 10:57 2019 libcrypto.so.1.0.2

root@x067:[/]ar tv /usr/lib/libssl.a
rwxr-xr-x     0/0     730481 Jun 03 11:01 2019 libssl.so
rwxr-xr-x     0/0     510766 Jun 03 11:01 2019 libssl.so.0.9.8
rwxr-xr-x     0/0     730481 Jun 03 11:01 2019 libssl.so.1.0.0
rwxr-xr-x     0/0     730481 Jun 03 10:57 2019 libssl.so.1.0.2


So, in any case, I see that IBM has changed the versions without a number to be the ABI 1.0.2 - and that is probably what is breaking things. (i.e., had I linked against archive members libssl.so.1.0.0 and  libcrypto.so.1.0.0 - this would be a non-issue).

Testing on my VM with openssl-1.0.2.1801 compared to openssl-1.0.2.1000 installed:

Seems the dl load of libraries is handled on it's own (see difference in the libcrypto members located)

root@x067:[/data/prj/aixtools/curl/curl-7.67.0]LIBPATH=./lib/.libs ldd ./src/.libs/curl
./src/.libs/curl needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthreads.a(shr_xpg5.o)
         ./lib/.libs/libcurl.a(libcurl.so.4)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libssl.a(libssl.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libcrypto.a(libcrypto.so.1.0.2)

versus openssl.base.1.0.2.1000

root@x065:[/data/prj/aixtools/curl/curl-7.67.0]LIBPATH=./lib/.libs ldd ./src/.libs/curl
./src/.libs/curl needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthreads.a(shr_xpg5.o)
         ./lib/.libs/libcurl.a(libcurl.so.4)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libssl.a(libssl.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libcrypto.a(libcrypto.so.1.0.0)


So, now I have this:

lslpp -L openssl.base
  Fileset                      Level  State  Type  Description (Uninstaller)
  ----------------------------------------------------------------------------
  openssl.base            1.0.2.2000    ?     F    Open Secure Socket Layer

...
root@x067:[/]ar tv /usr/lib/libcrypto.a
rwxr-xr-x     0/0     3068157 Oct 31 08:06 2019 libcrypto.so
rwxr-xr-x     0/0     2186744 Oct 31 08:06 2019 libcrypto.so.0.9.8
rwxr-xr-x     0/0     3068157 Oct 31 08:06 2019 libcrypto.so.1.0.0
rwxr-xr-x     0/0     3068157 Oct 31 07:46 2019 libcrypto.so.1.0.2
root@x067:[/]ar tv /usr/lib/libssl.a
rwxr-xr-x     0/0     730860 Oct 31 08:06 2019 libssl.so
rwxr-xr-x     0/0     510766 Oct 31 08:06 2019 libssl.so.0.9.8
rwxr-xr-x     0/0     730860 Oct 31 08:06 2019 libssl.so.1.0.0
rwxr-xr-x     0/0     730860 Oct 31 07:46 2019 libssl.so.1.0.2



root@x067:[/data/prj/aixtools/curl/curl-7.67.0]LIBPATH=./lib/.libs ldd ./src/.libs/curl
./src/.libs/curl needs:
         /usr/lib/libc.a(shr.o)
         /usr/lib/libpthreads.a(shr_xpg5.o)
         ./lib/.libs/libcurl.a(libcurl.so.4)
         /unix
         /usr/lib/libcrypt.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /usr/lib/libcrypto.a(libcrypto.so)
         /usr/lib/libssl.a(libssl.so)
         /usr/lib/libz.a(libz.so.1)
         /usr/lib/libcrypto.a(libcrypto.so.1.0.2)
root@x067:[/data/prj/aixtools/curl/curl-7.67.0]LIBPATH=./lib/.libs ./src/.libs/curl -V
curl 7.67.0 (powerpc-ibm-aix5.3.7.0) libcurl/7.67.0 OpenSSL/1.0.2t zlib/1.2.3 libssh2/1.9.0
Release-Date: 2019-11-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets


In short, while I do not doubt you have an issue caused, in part, by the latest packaging of openssl.base I cannot see how my packaging is affected by it. I'll need more input.

Question: are you also using OSS packages from any of BULL, AIX ToolBox and/or Michael Perzl? One of the reasons I package as installp is to avoid issues caused by the interaction of the RPM package manager and the installp manager.

p.s. If there are things you need to keep private, send me a PM, and we can discuss how to share delicate information.

[quadrazontal]

Thank you very much for the assessment.  Two flavours of messages relayed from a third party tool which makes use of libcurl are below and were presented after the ssl/ssh update, reverting to operational when backing out the change.  Jokingly, we think the magic number to be 3, not 4.  Yes, regarding OSS, Bull, etc, we purposefully intend or prefer sourcing from installp packaging when presented the option, others being considerably messy however we can follow this lead.  Again, thank you for prising.  -JP 

exec(): 0509-036 Cannot load program /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../bin/Pflt because of the following errors:
        0509-022 Cannot load module /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../lib/libOptiLib.so.
        0509-150   Dependent module /usr/lib/libcurl.a(libcurl.so.4) could not be loaded.
        0509-103   The module has an invalid magic number.
        0509-022 Cannot load module Pflt.
        0509-150   Dependent module /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../lib/libOptiLib.so could not be loaded.

exec(): 0509-036 Cannot load program /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../bin/Pflt because of the following errors:
        0509-022 Cannot load module /usr/lib/libcurl.a(libcurl.so.4).
        0509-150   Dependent module /usr/lib/libssl.a(libssl64.so) could not be loaded.
        0509-152   Member libssl64.so is not found in archive
        0509-022 Cannot load module Pflt.
        0509-150   Dependent module /usr/lib/libcurl.a(libcurl.so.4) could not be loaded.

[Michael]

OK.

Do you have a /usr/lib/libcurl.a? If so, it (should not) be mine. If not, try a symbolic link from /usr/lib/libcurl.a to /opt/lib/libcurl.a.

In general, I do not place libraries in /usr/lib so that I do not step on IBM and others packages.

I suspect if you would look at the module /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../lib/libOptiLib.so. it is only looking in /usr/lib (dump -H will show you that, also ldd /usr/local/manhattan/tms/tpe_engine/CONS/RunTimeEnv/planningeng/../lib/libOptiLib.so. will give you additional hints.)

lslpp -w | grep libcurl.a will tell if other installp packages are providing a libcurl.

The do the say using yum or rpm - I'll have to look it up. If you find that command before I post it - please post, so I learn too!

Michael

[Michael]

re: getting a listing of all RPM packages installed...

The base command is: rpm -qali

As this is packaged based (info block, then file listing) you may want to run the command twice. The first to grep if libcurl.a is even listed as a file installed by any RPM. If you have a lot of RPM packages installed, a second time to determine which package has the file installed.

Michael

[quadrazontal]

Thank you. We have a symlink in /usr/lib going to the /opt/lib/libcurl.a that is included in the aixtools lpp package. The Manhattan error message was presented after installing the latest ssh/ssl from IBM and was resolved by backing out the ssl/ssh updates.  We can look to approach IBM as well regarding the statement "I see that IBM has changed the versions without a number to be the ABI 1.0.2 - and that is probably what is breaking things".  Otherwise Manhattan should be able to help our app team by digging into their own libOptilib.so.  -JP

[Michael]

Thanks for the update.

Let me know if I can help further.