fetchmail, openssl and a sudden failure to authenticate certificates

This post was written by eli on January 9, 2018
Posted Under: Linux

Since around the beginning of December 2017, fetchmail stopped retrieving mails form Gmail servers silently, without issuing any kind of error message. Only when starting fetchmail in the foreground, I got

fetchmail: Server certificate verification error: unable to get local issuer certificate
fetchmail: This means that the root signing certificate (issued for /C=US/O=Google Trust Services/CN=Google Internet Authority G3) is not in the trusted CA certificate locations, or that c_rehash needs to be run on the certificate directory. For details, please see the documentation of --sslcertpath and --sslcertfile in the manual page.
140703549138760:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1060:
fetchmail: SSL connection failed.
fetchmail: socket error while fetching from ...@pop.gmail.com
fetchmail: Query status=2 (SOCKET)

That’s really cute. The SSL connection fails, but fetchmail doesn’t think it should drop me a note about it (like it does when the server refuses for a long time, for example). I should mention that I’m using an old 6.3.17 fetchmail release.

So let’s try the connection following this post (or “man s_client”):

Connect to a secure POP server:

$ openssl s_client -connect pop.gmail.com:995
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = Google Internet Authority G3
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = pop.gmail.com
verify return:1
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=pop.gmail.com
   i:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
 1 s:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
   i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
[ ... ]

Huh? No problem? So why doesn’t fetchmail accept the certificate? Because in my case, certificates were read from a local directory: In .fetchmailrc, I had the “sslcertpath /home/…/.certs/” directive for each and every entry related to Gmail. That made fetchmail accept certificate authorities only from the local directory, which failed suddenly, probably due to a change in Gmail’s certificate chain.

Where did I get this directive from? Probably from the automatic configuration made by fetchmailconf. Hurray.

So the obvious solution was to drop all those “sslcertpath” directives, and all was fine again.

Not really relevant, but…

Since I was playing with openssl, it’s also possible to talk with an HTTPS server directly this way (note the GET / request close to the end):

$ openssl s_client -connect www.google.com:443
depth=3 C = US, O = Equifax, OU = Equifax Secure Certificate Authority
verify return:1
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify return:1
depth=1 C = US, O = Google Inc, CN = Google Internet Authority G2
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = www.google.com
verify return:1
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
   i:/C=US/O=Google Inc/CN=Google Internet Authority G2
 1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
 2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
   i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
Server certificate
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
issuer=/C=US/O=Google Inc/CN=Google Internet Authority G2
No client certificate CA names sent
SSL handshake has read 3481 bytes and written 439 bytes
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
    Protocol  : TLSv1
    Cipher    : AES128-SHA
    Session-ID: A455441C733C2CDD844EEC39DEE96FA7C6D47D38F4A6021097891DF198E60C68
    Master-Key: EA9D1453E32865C2C4F04E969103ADA4E59C8ECC2FA09F6D8F2EDAE5E75E1DE40A18FF60CED7459A4F60679BB230C663
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 100800 (seconds)
    TLS session ticket:
    0000 - 00 26 5c 16 79 2f e1 46-a4 5b 34 30 72 a1 64 b7   .&\.y/.F.[40r.d.
    0010 - 79 20 15 87 62 cd 2a 05-8f 05 ac f7 d7 38 40 66   y ..b.*......8@f
    0020 - 87 0a a9 50 55 ba 5e d2-8f 90 c0 d0 83 25 b8 3e   ...PU.^......%.>
    0030 - b7 7b eb 5a ff 30 27 aa-1b c9 a1 d9 54 c3 aa 7e   .{.Z.0'.....T..~
    0040 - 05 96 83 76 49 90 fe 8e-d9 d7 55 e0 a3 0b 5b df   ...vI.....U...[.
    0050 - aa 28 12 81 02 84 b9 47-97 cd b8 81 b8 ee 2a 1c   .(.....G......*.
    0060 - c2 8b e0 e6 92 ae 4b a3-fb 2a 8e f3 eb f5 43 7b   ......K..*....C{
    0070 - a8 e9 58 c9 22 3a 15 3d-81 a7 0b a8 1a e4 3a 55   ..X.":.=......:U
    0080 - cd 72 04 8d 0e 70 5e 60-5d 19 d7 18 a1 1b ce d9   .r...p^`].......
    0090 - 87 60 78 ec c0 f7 6e 0f-c4 5c e7 06 1a e2 c1 d0   .`x...n..\......
    00a0 - e1 46 df c2 98 d0 da fd-87 eb 9b 0f 93 8a 4c e4   .F............L.
    00b0 - 95 db da 63 b3 e2 78 08-09 75 53 b7 d1 e3 6c d2   ...c..x..uS...l.
    00c0 - b6 6f 02 26 bd 16 e4 ae-a8 01 fa 81 3f e4 55 0d   .o.&........?.U.

    Start Time: 1515438093
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
HTTP/1.0 302 Found
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Location: https://www.google.co.il/?gfe_rd=cr&dcr=0&ei=E8BTWoO6GqfP8AeV05m4Cg
Content-Length: 272
Date: Mon, 08 Jan 2018 19:01:39 GMT
Alt-Svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.co.il/?gfe_rd=cr&amp;dcr=0&amp;ei=E8BTWoO6GqfP8AeV05m4Cg">here</A>.

So it’s a bit like “nc”, only with encryption.

Add a Comment

required, use real name
required, will not be published
optional, your blog address