Since around the beginning of December 2017, fetchmail stopped retrieving mails form Gmail servers silently, without issuing any kind of error message. Only when starting fetchmail in the foreground, I got
fetchmail: Server certificate verification error: unable to get local issuer certificate
fetchmail: This means that the root signing certificate (issued for /C=US/O=Google Trust Services/CN=Google Internet Authority G3) is not in the trusted CA certificate locations, or that c_rehash needs to be run on the certificate directory. For details, please see the documentation of --sslcertpath and --sslcertfile in the manual page.
140703549138760:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1060:
fetchmail: SSL connection failed.
fetchmail: socket error while fetching from ...@pop.gmail.com
fetchmail: Query status=2 (SOCKET)
That’s really cute. The SSL connection fails, but fetchmail doesn’t think it should drop me a note about it (like it does when the server refuses for a long time, for example). I should mention that I’m using an old 6.3.17 fetchmail release.
So let’s try the connection following this post (or “man s_client”):
Connect to a secure POP server:
$ openssl s_client -connect pop.gmail.com:995
CONNECTED(00000003)
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = Google Internet Authority G3
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = pop.gmail.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=pop.gmail.com
i:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
1 s:/C=US/O=Google Trust Services/CN=Google Internet Authority G3
i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
[ ... ]
Huh? No problem? So why doesn’t fetchmail accept the certificate? Because in my case, certificates were read from a local directory: In .fetchmailrc, I had the “sslcertpath /home/…/.certs/” directive for each and every entry related to Gmail. That made fetchmail accept certificate authorities only from the local directory, which failed suddenly, probably due to a change in Gmail’s certificate chain.
Where did I get this directive from? Probably from the automatic configuration made by fetchmailconf. Hurray.
So the obvious solution was to drop all those “sslcertpath” directives, and all was fine again.
Not really relevant, but…
Since I was playing with openssl, it’s also possible to talk with an HTTPS server directly this way (note the GET / request close to the end):
$ openssl s_client -connect www.google.com:443
CONNECTED(00000003)
depth=3 C = US, O = Equifax, OU = Equifax Secure Certificate Authority
verify return:1
depth=2 C = US, O = GeoTrust Inc., CN = GeoTrust Global CA
verify return:1
depth=1 C = US, O = Google Inc, CN = Google Internet Authority G2
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google Inc, CN = www.google.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
i:/C=US/O=Google Inc/CN=Google Internet Authority G2
1 s:/C=US/O=Google Inc/CN=Google Internet Authority G2
i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
2 s:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEdjCCA16gAwIBAgIIWv4BLr9C/xUwDQYJKoZIhvcNAQELBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTcxMjEzMTMyOTExWhcNMTgwMzA3MTMwMTAw
WjBoMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEXMBUGA1UEAwwOd3d3
Lmdvb2dsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDh3H8L
+wzppO5DrRgLvOOF8nM692sA/aaFv0Hr5pQuuDluOwEV+ocY8iKwBWH2XMNTDTDz
KIuqyqAbe/DvUQ+a8HjuEGepWdBe/VY0vuxrjKc7yJs6QImmSt9dUF01LI6zKjyR
B7MNyMCtHHn2DFvD9uGocNqFXAeJrWCs2VbIqP+jj2QdVJ2WK/gV0ybyGmdyZfbw
SdDKSYKt3KK3depWai7CKeYHHNpMY8OFBLi4uWIWA28ZTzIxqb2Ar7aiZUUEzWgf
8Ak4fLsNzEuiCmouhFdBwwxDGGqDgrM+3NFk7kGoBOf2mTH6qcQ+sg2G/rVylE94
2mUhlw2viW/bAzN7AgMBAAGjggFBMIIBPTATBgNVHSUEDDAKBggrBgEFBQcDATAZ
BgNVHREEEjAQgg53d3cuZ29vZ2xlLmNvbTBoBggrBgEFBQcBAQRcMFowKwYIKwYB
BQUHMAKGH2h0dHA6Ly9wa2kuZ29vZ2xlLmNvbS9HSUFHMi5jcnQwKwYIKwYBBQUH
MAGGH2h0dHA6Ly9jbGllbnRzMS5nb29nbGUuY29tL29jc3AwHQYDVR0OBBYEFMY4
s3yyzEnrwiJ/NsB1uEETOXq6MAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUSt0G
Fhu89mi1dvWBtrtiGrpagS8wIQYDVR0gBBowGDAMBgorBgEEAdZ5AgUBMAgGBmeB
DAECAjAwBgNVHR8EKTAnMCWgI6Ahhh9odHRwOi8vcGtpLmdvb2dsZS5jb20vR0lB
RzIuY3JsMA0GCSqGSIb3DQEBCwUAA4IBAQAxixLtjbT18w3iXYj6psMVK2uK21Is
Z4Oi2y5533nHqZXhb3z9K7m1ejwgL/s+bb5+D4HQdKhopO81oBf2Li9ztQ255Q24
nA1p4xkTdPV3UvFPA6R6G4muFZQmJUvIgrH/uZAXQ36K9/8aI8SgawLo1RPDWOxW
pCw4/1SfQ8FgUQmvqb+OkQ5bCXXySRhidZkCUg4DXUNsJ++HATlSEeSb4kdu4Fny
mwRBZma5muwbwRxTHCnbs1A82Ehxi0+DBjifgJy0NMyZbsgiWnyuexNAxoDVIC0I
u2xlpTBqaS1HzPpJ7K9tJ7i6AYe2YCrKo/fYBYBWaw/Q7VNDWk7NvlTs
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=www.google.com
issuer=/C=US/O=Google Inc/CN=Google Internet Authority G2
---
No client certificate CA names sent
---
SSL handshake has read 3481 bytes and written 439 bytes
---
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES128-SHA
Session-ID: A455441C733C2CDD844EEC39DEE96FA7C6D47D38F4A6021097891DF198E60C68
Session-ID-ctx:
Master-Key: EA9D1453E32865C2C4F04E969103ADA4E59C8ECC2FA09F6D8F2EDAE5E75E1DE40A18FF60CED7459A4F60679BB230C663
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
TLS session ticket lifetime hint: 100800 (seconds)
TLS session ticket:
0000 - 00 26 5c 16 79 2f e1 46-a4 5b 34 30 72 a1 64 b7 .&\.y/.F.[40r.d.
0010 - 79 20 15 87 62 cd 2a 05-8f 05 ac f7 d7 38 40 66 y ..b.*......8@f
0020 - 87 0a a9 50 55 ba 5e d2-8f 90 c0 d0 83 25 b8 3e ...PU.^......%.>
0030 - b7 7b eb 5a ff 30 27 aa-1b c9 a1 d9 54 c3 aa 7e .{.Z.0'.....T..~
0040 - 05 96 83 76 49 90 fe 8e-d9 d7 55 e0 a3 0b 5b df ...vI.....U...[.
0050 - aa 28 12 81 02 84 b9 47-97 cd b8 81 b8 ee 2a 1c .(.....G......*.
0060 - c2 8b e0 e6 92 ae 4b a3-fb 2a 8e f3 eb f5 43 7b ......K..*....C{
0070 - a8 e9 58 c9 22 3a 15 3d-81 a7 0b a8 1a e4 3a 55 ..X.":.=......:U
0080 - cd 72 04 8d 0e 70 5e 60-5d 19 d7 18 a1 1b ce d9 .r...p^`].......
0090 - 87 60 78 ec c0 f7 6e 0f-c4 5c e7 06 1a e2 c1 d0 .`x...n..\......
00a0 - e1 46 df c2 98 d0 da fd-87 eb 9b 0f 93 8a 4c e4 .F............L.
00b0 - 95 db da 63 b3 e2 78 08-09 75 53 b7 d1 e3 6c d2 ...c..x..uS...l.
00c0 - b6 6f 02 26 bd 16 e4 ae-a8 01 fa 81 3f e4 55 0d .o.&........?.U.
Start Time: 1515438093
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
GET /
HTTP/1.0 302 Found
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Location: https://www.google.co.il/?gfe_rd=cr&dcr=0&ei=E8BTWoO6GqfP8AeV05m4Cg
Content-Length: 272
Date: Mon, 08 Jan 2018 19:01:39 GMT
Alt-Svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.co.il/?gfe_rd=cr&dcr=0&ei=E8BTWoO6GqfP8AeV05m4Cg">here</A>.
</BODY></HTML>
read:errno=0
So it’s a bit like “nc”, only with encryption.
As the title says: These are the labels of an HPC (High Pin Count) FMC connector, in plain CSV format for easy handling. It was a bit odd to me that I didn’t find this info on the web myself.
Just copy-paste:
A1,GND
A2,DP1_M2C_P
A3,DP1_M2C_N
A4,GND
A5,GND
A6,DP2_M2C_P
A7,DP2_M2C_N
A8,GND
A9,GND
A10,DP3_M2C_P
A11,DP3_M2C_N
A12,GND
A13,GND
A14,DP4_M2C_P
A15,DP4_M2C_N
A16,GND
A17,GND
A18,DP5_M2C_P
A19,DP5_M2C_N
A20,GND
A21,GND
A22,DP1_C2M_P
A23,DP1_C2M_N
A24,GND
A25,GND
A26,DP2_C2M_P
A27,DP2_C2M_N
A28,GND
A29,GND
A30,DP3_C2M_P
A31,DP3_C2M_N
A32,GND
A33,GND
A34,DP4_C2M_P
A35,DP4_C2M_N
A36,GND
A37,GND
A38,DP5_C2M_P
A39,DP5_C2M_N
A40,GND
B1,RES1
B2,GND
B3,GND
B4,DP9_M2C_P
B5,DP9_M2C_N
B6,GND
B7,GND
B8,DP8_M2C_P
B9,DP8_M2C_N
B10,GND
B11,GND
B12,DP7_M2C_P
B13,DP7_M2C_N
B14,GND
B15,GND
B16,DP6_M2C_P
B17,DP6_M2C_N
B18,GND
B19,GND
B20,GBTCLK1_M2C_P
B21,GBTCLK1_M2C_N
B22,GND
B23,GND
B24,DP9_C2M_P
B25,DP9_C2M_N
B26,GND
B27,GND
B28,DP8_C2M_P
B29,DP8_C2M_N
B30,GND
B31,GND
B32,DP7_C2M_P
B33,DP7_C2M_N
B34,GND
B35,GND
B36,DP6_C2M_P
B37,DP6_C2M_N
B38,GND
B39,GND
B40,RES0
C1,GND
C2,DP0_C2M_P
C3,DP0_C2M_N
C4,GND
C5,GND
C6,DP0_M2C_P
C7,DP0_M2C_N
C8,GND
C9,GND
C10,LA06_P
C11,LA06_N
C12,GND
C13,GND
C14,LA10_P
C15,LA10_N
C16,GND
C17,GND
C18,LA14_P
C19,LA14_N
C20,GND
C21,GND
C22,LA18_P_CC
C23,LA18_N_CC
C24,GND
C25,GND
C26,LA27_P
C27,LA27_N
C28,GND
C29,GND
C30,SCL
C31,SDA
C32,GND
C33,GND
C34,GA0
C35,12P0V
C36,GND
C37,12P0V
C38,GND
C39,3P3V
C40,GND
D1,PG_C2M
D2,GND
D3,GND
D4,GBTCLK0_M2C_P
D5,GBTCLK0_M2C_N
D6,GND
D7,GND
D8,LA01_P_CC
D9,LA01_N_CC
D10,GND
D11,LA05_P
D12,LA05_N
D13,GND
D14,LA09_P
D15,LA09_N
D16,GND
D17,LA13_P
D18,LA13_N
D19,GND
D20,LA17_P_CC
D21,LA17_N_CC
D22,GND
D23,LA23_P
D24,LA23_N
D25,GND
D26,LA26_P
D27,LA26_N
D28,GND
D29,TCK
D30,TDI
D31,TDO
D32,3P3VAUX
D33,TMS
D34,TRST_L
D35,GA1
D36,3P3V
D37,GND
D38,3P3V
D39,GND
D40,3P3V
E1,GND
E2,HA01_P_CC
E3,HA01_N_CC
E4,GND
E5,GND
E6,HA05_P
E7,HA05_N
E8,GND
E9,HA09_P
E10,HA09_N
E11,GND
E12,HA13_P
E13,HA13_N
E14,GND
E15,HA16_P
E16,HA16_N
E17,GND
E18,HA20_P
E19,HA20_N
E20,GND
E21,HB03_P
E22,HB03_N
E23,GND
E24,HB05_P
E25,HB05_N
E26,GND
E27,HB09_P
E28,HB09_N
E29,GND
E30,HB13_P
E31,HB13_N
E32,GND
E33,HB19_P
E34,HB19_N
E35,GND
E36,HB21_P
E37,HB21_N
E38,GND
E39,VADJ
E40,GND
F1,PG_M2C
F2,GND
F3,GND
F4,HA00_P_CC
F5,HA00_N_CC
F6,GND
F7,HA04_P
F8,HA04_N
F9,GND
F10,HA08_P
F11,HA08_N
F12,GND
F13,HA12_P
F14,HA12_N
F15,GND
F16,HA15_P
F17,HA15_N
F18,GND
F19,HA19_P
F20,HA19_N
F21,GND
F22,HB02_P
F23,HB02_N
F24,GND
F25,HB04_P
F26,HB04_N
F27,GND
F28,HB08_P
F29,HB08_N
F30,GND
F31,HB12_P
F32,HB12_N
F33,GND
F34,HB16_P
F35,HB16_N
F36,GND
F37,HB20_P
F38,HB20_N
F39,GND
F40,VADJ
G1,GND
G2,CLK1_M2C_P
G3,CLK1_M2C_N
G4,GND
G5,GND
G6,LA00_P_CC
G7,LA00_N_CC
G8,GND
G9,LA03_P
G10,LA03_N
G11,GND
G12,LA08_P
G13,LA08_N
G14,GND
G15,LA12_P
G16,LA12_N
G17,GND
G18,LA16_P
G19,LA16_N
G20,GND
G21,LA20_P
G22,LA20_N
G23,GND
G24,LA22_P
G25,LA22_N
G26,GND
G27,LA25_P
G28,LA25_N
G29,GND
G30,LA29_P
G31,LA29_N
G32,GND
G33,LA31_P
G34,LA31_N
G35,GND
G36,LA33_P
G37,LA33_N
G38,GND
G39,VADJ
G40,GND
H1,VREF_A_M2C
H2,PRSNT_M2C_L
H3,GND
H4,CLK0_M2C_P
H5,CLK0_M2C_N
H6,GND
H7,LA02_P
H8,LA02_N
H9,GND
H10,LA04_P
H11,LA04_N
H12,GND
H13,LA07_P
H14,LA07_N
H15,GND
H16,LA11_P
H17,LA11_N
H18,GND
H19,LA15_P
H20,LA15_N
H21,GND
H22,LA19_P
H23,LA19_N
H24,GND
H25,LA21_P
H26,LA21_N
H27,GND
H28,LA24_P
H29,LA24_N
H30,GND
H31,LA28_P
H32,LA28_N
H33,GND
H34,LA30_P
H35,LA30_N
H36,GND
H37,LA32_P
H38,LA32_N
H39,GND
H40,VADJ
J1,GND
J2,CLK3_M2C_P
J3,CLK3_M2C_N
J4,GND
J5,GND
J6,HA03_P
J7,HA03_N
J8,GND
J9,HA07_P
J10,HA07_N
J11,GND
J12,HA11_P
J13,HA11_N
J14,GND
J15,HA14_P
J16,HA14_N
J17,GND
J18,HA18_P
J19,HA18_N
J20,GND
J21,HA22_P
J22,HA22_N
J23,GND
J24,HB01_P
J25,HB01_N
J26,GND
J27,HB07_P
J28,HB07_N
J29,GND
J30,HB11_P
J31,HB11_N
J32,GND
J33,HB15_P
J34,HB15_N
J35,GND
J36,HB18_P
J37,HB18_N
J38,GND
J39,VIO_B_M2C
J40,GND
K1,VREF_B_M2C
K2,GND
K3,GND
K4,CLK2_M2C_P
K5,CLK2_M2C_N
K6,GND
K7,HA02_P
K8,HA02_N
K9,GND
K10,HA06_P
K11,HA06_N
K12,GND
K13,HA10_P
K14,HA10_N
K15,GND
K16,HA17_P_CC
K17,HA17_N_CC
K18,GND
K19,HA21_P
K20,HA21_N
K21,GND
K22,HA23_P
K23,HA23_N
K24,GND
K25,HB00_P_CC
K26,HB00_N_CC
K27,GND
K28,HB06_P_CC
K29,HB06_N_CC
K30,GND
K31,HB10_P
K32,HB10_N
K33,GND
K34,HB14_P
K35,HB14_N
K36,GND
K37,HB17_P_CC
K38,HB17_N_CC
K39,GND
K40,VIO_B_M2C
After quite a while of working perfectly well, the mini HDMI2AV module I have (in the picture above, mentioned in this post) started producing an unstable picture, and in the end a completely garbled one. It took some time to nail down this specific component in the foodchain, because there was also an HDMI splitter involved.
The problem, as it turned out, was that this module takes voltage from the HDMI plug, if such is available, instead of the dedicated power plug. In my specific setup, it seems like there was some voltage was available, but not enough to drive the device — because the HDMI plug was connected to the HDMI splitter. I suppose some internal power supply switch went into some not-here-not-there kind of situation, and eventually got some permanent damage. The other HDMI2AV unit I have didn’t work either in the same conditions, but probably didn’t reach the point of permanent damage (so it’s working right now).
On an HDMI connector, Pin 18 is +5V, minimum 55 mA, intended originally to feed the monitor with voltage even if it’s shut off, so its DDC (EDID) information can be obtained. Some devices (e.g. cheap HDMI splitters and HDMI to AV converters) might use this voltage instead of the supplied external voltage in some cases.
Not all cables conduct this pin. It’s therefore advisable to check the cable before working with it, when the setup is more than just a direct connection. It’s not easy, even with a multimeter. Pushing a thin wire into the tiny holes at the front may give contact with the relevant pin, but this isn’t bulletproof. Possibly try with an HDMI/DVI adapter (pin 14 on a DVI connector is +5V). Or test with a device that is known to rely on this voltage (e.g. this HDMI2AV module).
The solution in my case was to replace the HDMI2AV module and all cables with such that don’t let the +5V wire through. In particular, it seems like the cable to the TV set (via HDMI) that went to the HDMI splitter (which connects to the HDMI2AV module on its other output) was the issue.
In short
On an embedded ARM-based Lubuntu 16.04, I had LXDE’s logoff dialog window offering suspend as an option, and when that was chosen, the system got itself into some nasty state with network and keyboard off. The serial console was still active, and yet, I was better off without it.
It turned out that the kernel was misconfigured to announce that it supported suspend to RAM:
# cat /sys/power/state
freeze mem
So no wonder that option was presented to the user on GUI. The solution: Turn off the CONFIG_SUSPEND kernel compilation flag. Recompile, deploy, and that’s it:
# cat /sys/power/state
#
And the faulty options were gone.
The rest of this post contains things I jotted down as I wasted time trying to find out what the problem was.
Irrelevant notes
A journey in the sources (more wasted time)
I tried to follow how lxsession-logout, which is LXDE’s program that displays the logout dialog box, decides which low-power modes to offer. And what actually happens when suspend is requested.
- lxsession-logout.c learns if the system can suspend (and hence the button shall be presented) by calling dbus_systemd_CanSuspend().
- which is implemented in lxsession-logout-dbus-interface.c, and calls systemd_query() with “CanSuspend” as its parameter
- which (implemented in the same file) in turn opens a session with “org.freedesktop.login1″ over DBus and issues a query
- Judging by the “BusName=org.freedesktop.login1″ directive in /lib/systemd/system/systemd-logind.service, systemd-logind.service, (running systemd-login) is answering this query.
- Looking in systemd’s login-dbus.c, “CanSuspend” calls the method method_can_suspend(), which in turn calls method_can_shutdown_or_sleep() with “org.freedesktop.login1.suspend” as the key argument, which calls bus_test_polkit() for an answer on that.
- Implemented in systemd/shared/bus-util.c, bus_test_polkit() makes a DBus query on “org.freedesktop.PolicyKit1″
- There are also references to upowerd in lxsession-logout.c, but since stopping this service changes nothing, I focused on logind.
- Judging by the “BusName=org.freedesktop.PolicyKit1″ directive in /lib/systemd/system/polkitd.service, polkitd.service (running /usr/lib/policykit-1/polkitd) answer this.
- Back to login-dbus.c, a “Suspend” request causes a call to method_suspend(), which calls method_do_shutdown_or_sleep(), which calls bus_manager_shutdown_or_sleep_now_or_later(), which calls execute_shutdown_or_sleep(). The “unitname” parameter traverses these calls with the value SPECIAL_SUSPEND_TARGET. There are several checks on the way that may cancel the request, but this is the chain to fulfilling it.
- execute_shutdown_or_sleep() issues a DBus request on “org.freedesktop.systemd1″ (I have a wild guess which process is on the other side).
- Apparently (without following the execution chain), systemd/src/sleep.c is responsible for putting the system in suspend mode and such. Among others, it writes to /sys/power/state. This is where it hit me that maybe the kernel’s configuration was the problem.
As systemd seems to be here to stay (or at least I hope so), this is a post of random notes to self that I jot down as I explore it. It will probably grow with time, and become a mixture of basic issues and rather advanced stuff.
Also see my post on systemd services as cronjobs, which also discusses templates and some other finer details.
Useful references
- man systemd.service and man systemd.unit as well as others. Really. These are the best sources, it turns out.
- The excellent Systemd for Admins series (with several relevant and specific topics).
- The primer for systemd: Basic concepts explained.
- Red Hat’s guide to creating custom targets (and daemons)
- The FAQ (with actually useful info!)
- On the Network Target (and how to run a target only when the network is up)
- man systemd.special for a list of built-in targets, their meaning and recommended use
- man systemd.timer
- man systemd.time for how to express time events with systemd
- systemd.kill on how systemd kills services
- systemd.exec
systemctl is the name of the game
Forget “service”, “telinit” and “initctl”. “systemctl” is the new swiss knife for starting, stopping, enabling and disabling services, as well as obtaining information on how services are doing. And it’s really useful.
To get an idea on what runs on the system and what unit triggered it off, go
# systemctl status
Note that “systemd status” lists, among others, all processes initiated by each login session for each user. Which is an extremely useful variant of “ps”.
And just a list of all services
# systemctl
Ask about a specific service:
# systemctl status ssh
● ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2017-12-01 10:37:21 IST; 1h 17min ago
Main PID: 1018 (sshd)
CGroup: /system.slice/ssh.service
└─1018 /usr/sbin/sshd -D
Dec 01 12:26:26 machine sshd[2841]: Accepted publickey for eli from 192.168.1.12 port 45220 ssh2: RSA SHA256:xxx
Dec 01 12:26:26 machine sshd[2841]: pam_unix(sshd:session): session opened for user eli by (uid=0)
Show the service unit’s file (note that the file name in effect appears as a comment in the first row):
$ systemctl cat ssh
# /lib/systemd/system/ssh.service
[Unit]
Description=OpenBSD Secure Shell server
After=network.target auditd.service
ConditionPathExists=!/etc/ssh/sshd_not_to_be_run
[Service]
EnvironmentFile=-/etc/default/ssh
ExecStartPre=/usr/sbin/sshd -t
ExecStart=/usr/sbin/sshd -D $SSHD_OPTS
ExecReload=/usr/sbin/sshd -t
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartPreventExitStatus=255
Type=notify
RuntimeDirectory=sshd
RuntimeDirectoryMode=0755
[Install]
WantedBy=multi-user.target
Alias=sshd.service
Really, isn’t it sweet?
There’s also systemctl show for an extensive printout of all assignments, explicit and implicit.
Turning off a service: Find it with the “systemctl status” command above (or just “systemctl”), and then (this is an example of a service not found in the status, because it’s an LSB service);
# systemctl disable tvheadend
tvheadend.service is not a native service, redirecting to systemd-sysv-install
Executing /lib/systemd/systemd-sysv-install disable tvheadend
insserv: warning: current start runlevel(s) (empty) of script `tvheadend' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script `tvheadend' overrides LSB defaults (0 1 6).
And “enable” for enabling a service.
Needless to say, services can be started, stopped and restarted with “systemctl X service” where X is either start, stop or restart.
For a list of all services (including disactivated):
$ systemctl --all
and then there’s a whole range of systemctl list-this-and-that, which are really useful. For example (try them out!):
$ systemctl list-dependencies
$ systemctl list-timers
$ systemctl list-unit-files
$ systemctl list-sockets
No more fishing in /var/log/syslog
/var/log/syslog is still there, but forget about it: journalctl is the way read logs. And it doesn’t require root privileges, which is reason enough.
To get the log message since the current boot:
$ journalctl -b
(that alone justifies using the utility).
There’s also the -u flag to see the logs from a specific (systemd) unit (systemctl status gives that as well), -g for grep, and the -f (follow) flag as in tail -f.
A fast shutdown
Maybe the most annoying thing about systemd is that if some process gets stuck, the shutdown waits for it forever. That is, three minutes typically. To fix this edit both /etc/systemd/system.conf and /etc/systemd/user.conf and make them say
DefaultTimeoutStopSec=5s
DefaultTimeoutAbortSec=5s
This typically requires uncommenting the assignment for DefaultTimeoutStopSec, and add the latter. The result of this setting is a reduction of the delay to 10 seconds (these two add up).
A reboot is required for this to take effect.
What makes a systemd service run (on boot)
See /etc/systemd/system/multi-user.target.wants for a list of services that are activated on boot. In particular note that not all are symlinks to .service unit files.
General memo jots
- Always run the
systemctl daemon-reload command after creating new unit files or modifying existing unit files. Otherwise, the systemctl start or systemctl enable commands could fail due to a mismatch between states of systemd and actual service unit files on disk.
- Services are best run in the foreground. Unlike classic UNIX services, there’s no point in daemonizing. All processes belonging to the relevant service are enclosed in a Cgroup anyhow, and systemd handles the daemonizing for Type=simple services. In a clean and uniform manner.
- Unit files’ suffix indicate their type. When the non-suffix part of two files is the same, they indicate a functional relationship. For example, systemd-tmpfiles-clean.timer says when to launch systemd-tmpfiles-clean.service. Or that systemd-ask-password-console.path gives the path to be watched, and systemd-ask-password-console.service is the service to fire off.
- After= doesn’t imply a dependency, and Requires= doesn’t imply the order of starting services. Both are needed if one service depends on the other running when it starts.
- The Type= directive’s main influence is determining when the service is active, i.e. when other services that depend on it can be launched.
- There’s also “loginctl” which lists the current users and their sessions
Where to find unit files
The configuration files are considered in the following order (later overrules earlier):
- /lib/systemd/system — where installation scripts write to
- /run/systemd/system — runtime files
- /etc/systemd/system — per-system user preferences
Per-user files can be found in ~/.config/systemd/user and possibly also in ~/.local/share/systemd/user.
Keeping the service under control
The control on services is quite impressive. Both container virtualization and systemd use Cgroups, so there’s a bit of container flavor to this whole thing.
From man systemd.exec:
- User= to run as a certain user. This also sets the group information of the user, so there’s no need to use Group= in addition to this.
- WorkingDirectory= for setting the cwd of the service (there’s also RootDirectory= for a chroot).
- NoNewPrivileges= to prevent privileges elevation. The easy and efficient way, according to the man page.
- SecureBits= set to noroot, to prevent the process from gaining root. More fine-grained than NoNewPrivileges.
- A whole lot of Limit*= assignments for limiting resource usage
- OOMScoreAdjust= for making it less or more eligible for OOM killer
- ProtectSystem= and ProtectHome= for preventing access to certain directories from the processes in the control group.
- ReadWritePaths=, ReadOnlyPaths=, InaccessiblePaths= are more fine-grained in choosing in which directories the service is allowed to do what.
- PrivateTmp= for creating private /tmp and /var/tmp for the service.
- Environment= (and possibly EnvironmentFile=) for setting environment variables
- StandardInput= allows feeding the process’ stdin with data from a file (or other sources, e.g. ttys, sockets and more), or with literal data from the unit file, with StandardInputText= or StandardInputData=
- StandardOutput= and StandardError= define where the respective outputs go. Default to the system journal.
It’s also possible to create runtime directories that are removed when the service terminates (e.g. RuntimeDirectory=).
There is a whole range of other sandboxing options, including disabling networking (leaving lo only). It’s also possible to restrict system calls
KillMode
By default, KillMode=control-group, so all processes in the group are killed with the signal specified in KillSignal (defaults to SIGTERM). It then sends a SIGKILL after TimeoutStopSec seconds, assuming that SendSIGKILL=yes, which is the default and definitely recommended setting (see man systemd.kill).
Setting KillMode=mixed is like control-group, but the initial SIGTERM is only sent to the main process. This is useful if it catches this signal and shuts down the other processes nicely. And if it doesn’t, the big hammer goes on all processes after TimeoutStopSec.
I’m not clear on what happens if SIGKILL doesn’t really kill some process (due to e.g. being stuck in an uninterruptible sleep). I guess the service would be considered stopped anyhow, but it appears like this isn’t documented.
User Systemd?
User-mode systemd is in principle the same as the mainstream services, with the main difference that they are intended to run with a specific user ID, and while that user is logged in. So the original idea behind this concept is to have certain processes running in the background while this user has a session (i.e. is logged in) and turn them off when this user logs out. This is an excellent page on the matter.
But if the service is supposed to run regardless of whether the user is logged in or not, it’s typically wiser to make it a regular systemd service, and set the User= assignment in the unit file to select the relevant user for execution. The only advantage with User Systemd is if that user needs the capability to make changes in the service unit files, and it doesn’t have root on the computer. So I opted this out.
It’s important to note however that the processes generated by systemd don’t belong to a session, and they don’t have the environment variables set by .bashrc or anything of that sort. They run independently. Their only relation with the user logging in is when they live or not, and even that isn’t always true (see notes on lingering below).
Another important thing is that WantedBy (in the service unit file’s Install section) should be set to default.target and not multi-user.target, like a system-wide service. And that makes sense: The latter target is something related to the entire system.
And then there’s “lingering”, which means that the user service runs even when the user isn’t logged in. Effectively, the service turns into a regular service, kicked off on boot (if enabled), just with user privileges and with the definition files put in a user directory. To do this, go
# loginctl enable-linger username
This makes the login manager kick off the services as soon as it’s started — that is, at boot.
Enabling console autologin on tty1 and ttyPS0
Following this page and as explained on this page, add /etc/systemd/system/getty@tty1.service.d/autologin.conf (after creating the getty@tty1.service.d directory) as follows:
[Service]
ExecStart=
ExecStart=-/sbin/agetty -a root --noclear %I $TERM
Note that the filename autologin.conf has no significance. It’s suffix and the directory it resides in that matter.
The idea is to override the ExecStart parameter given in /lib/systemd/system/getty@.service template unit, which reads
ExecStart=-/sbin/agetty --noclear %I $TERM
but otherwise have it running the same. The reason for two ExecStart lines is that the empty assignment clears the existing assignment (or otherwise it would have been added on top of it), and the second sets the command.
Note the %I substitute parameter, which stands for the instance of the current tty.
The leading dash means that the exit value of the command is ignored, and may be nonzero without the unit considered as failed (see man systemd.service).
This can’t be repeated with ttyPS0, because systemd goes another way for setting up the serial console: At an early stage in the boot, systemd-getty-generator automatically sets a target, serial-getty@ttyPS0.service, which is implemented by the /lib/systemd/system/serial-getty@.service template unit.
# systemctl status
[ ... ]
│ ├─system-serial\x2dgetty.slice
│ │ └─serial-getty@ttyPS0.service
│ │ └─2337 /sbin/agetty --keep-baud 115200 38400 9600 ttyPS0 vt220
So the solution is adding /etc/systemd/system/serial-getty@ttyPS0.service.d/autologin.conf saying
[Service]
ExecStart=
ExecStart=-/sbin/agetty -a root --keep-baud 115200,38400,9600 %I $TERM
Disable renaming of Ethernet interfaces
Ditch those tedious Ethernet interface names (e.g. enp3s0, enp0s31f6, wlp2s0, really, come on), and bring back the good old eth0, eth1 etc. Note that the kernel still assigns the good old interface names. It’s systemd that renames them. So the trick is simple: Mask the default naming policy, which causes this to happen:
# ln -s /dev/null /etc/systemd/network/99-default.link
(try man systemd.link for more info, and there’s a lot — including how to change MAC address)
And update the initramfs:
# update-initramfs -u
It won’t work without updating the initramfs, because the network interface names are set way before the root filesystem is mounted. The files are copied into the initramfs’ /lib/systemd/network/ directory (note that it’s /lib, even though they’re saved in /etc on the root filesystem. Which makes sense, because on the initramfs there’s no point with the /lib vs. /etc distinction).
Also, renaming can’t be used to achieve persistent allocation of ethN names (so says the manpage of systemd.link) because of a race with the kernel’s name assignments. As of kernel v4.15, that is.
Resetting failed services
To get the system out of the “degraded” state due to an irrelevant failed service, for example:
● lvm2-pvscan@253:9.service loaded failed failed LVM2 PV scan on device 253:9
Just go (as root)
# systemctl reset-failed
and the failed service disappears, returning the overall status to “running” again (assuming there’s no real problem).
Adding attributes to a sysV service
All legacy init.d-base services have an automatically generated unit file generated on their behalf. For example, mysql’s service file can be found as /var/run/systemd/generator.late/runlevel3.target.wants/mysql.service. Copy this file into /etc/systemd/system and edit it according to your needs. Most important: Add an [Install] section.
ExecStart will still points at an sysV init script, but since this unit file overrides the sysV script, the parameters in the new unit file will be those in effect.
Then reload the systemd daemon and enable the service e.g.
# systemctl enable mysql
Synchronizing state for mysql.service with sysvinit using update-rc.d...
Executing /usr/sbin/update-rc.d mysql defaults
Executing /usr/sbin/update-rc.d mysql enable
So this doesn’t look encouraging, and the files in /etc/init.d/ are indeed updated. Verify that the symbolic link for enabling the service has been created (usually in /etc/systemd/system/multi-user.target.wants/). Then restart the service and check with “systemctl status mysql” that the service is enabled and that the new unit file is mentioned in the “Loaded” part.
Ditching NetworkManager
I can’t say I was very fond of it ever (not only because of the capital letters in its name), and systemd can now do its job. Odds are that NetworkManager will become history in a matter of a few years, so better give it the boot now:
# systemctl disable NetworkManager
# systemctl disable NetworkManager-wait-online
A word of warning: The GUI for handling Ethernet connection requires NetworkManager, so for better or for worse, it won’t work anymore.
I also ditched ModemManager, as I have a solution for my ADSL modem. But I’m not sure about others.
And enable systemd’s cutie instead
# systemctl enable systemd-networkd
You might want to get rid of the service that waits for networking to be “online”:
# systemctl mask systemd-networkd-wait-online.service
This service runs /lib/systemd/systemd-networkd-wait-online, which is supposed to wait for at least one Ethernet card being configured. In practice, it waited until its 120 seconds timeout, and then said it failed. As a result, some services that depend on the network.online were kicked off only after these two minutes, and the overall system’s status was marked as “degraded”.
What does “online” mean, and why is it important? Good question, discussed here. My conclusion: As any contemporary Linux system should be able to tolerate hotplugging of its NICs, there’s no need to wait. Handle them as they appear.
A simple definition file for a NIC can be, for example, /etc/systemd/network/20-eth0.network
[Match]
MACAddress=1c:1b:0d:45:0f:eb
[Network]
DHCP=yes
Note that I detected the card by its MAC address. It’s also possible to select it by its ifconfig name, and other methods. But this is safest.
DHCP is “no” by default. Replace it with a line saying e.g.
Address=10.10.10.10/16
for a static IP address. Go “man systemd.network” for the whole set of options. They cover basically everything needed.
After making changes to such file (or adding one), go
# systemctl restart systemd-networkd
to make them effective. There’s no need to update the initramfs. In fact, .network files aren’t copied into it (unlike .link files, as said above).
As for responding to hotplugging events of network devices, there’s this post.
Introduction
August 2019 update: I’ve disactivated the service described below on my own machine, because in the end, the system didn’t recover properly from the condition it was supposed to solve. No solution, in the end.
Having installed Linux Mint 18.1 (kernel 4.4.0-53) with overlayroot on a Gigabyte Brix BACE-3160 (see other notes on this here and here) for my living room media center, I had an issue with bringing up the computer every now and then. Namely, I had a blank screen with mouse pointer only, 100% CPU on the cinnamon process, and some 50% on cinnamon-settings-daemon, eating all RAM (8 GB) gradually until the computer crashes (as it’s swapless).
The whole point with an overlayroot based media center is that it behaves like any electrical appliance. In particular, you know it will recover well from a power outage. And not crash because of some silly desktop manager messing up.
I’m not alone: There’s a bug report on this. There’s also this Git Hub issues page seems to point at the problem, in particular as it was closed pointing at a commit, which was merged into the main repository saying “Add detection for accountsservice background as it’s ubuntu only”.
It seems to be more common with SSD disks (because they’re faster).
And still there’s no solution available, almost a year later. Don’t tell me to upgrade. Upgrading means trading known bugs for new, unknown ones.
Which leaves me with making a workaround. In this case, a watchdog service, which monitors the resident memory usage of the process having the command line “cinnamon”. If it goes above 300 MiB during its first five minutes (more or less), restard the mdm service, which in turn restarts Cinnamon. Plain and disgusting.
The ironic truth is that I haven’t been able to test the watchdog for real, as I’ve failed to repeat the problem, despite a lot of power cycles. But I have verified that it works by lowering the memory threshold, and confirmed that the mdm service is indeed restarted. I also know from previous experience that restarting mdm helps.
The stuff
This should be copied to /usr/local/bin/cinnamon-watchdog.pl (executable at least by root):
#!/usr/bin/perl
use warnings;
use strict;
my $cmd = 'cinnamon';
my $limit = 200; # Maximal resident memory size in MiB
my $timeout = 300; # Approximately how long this script should run, in seconds
my $pid;
my $timer = 0;
my $rss = 0;
eval {
while (1) {
$timer++;
if ($timer >= $timeout) {
if (defined $pid) {
logger("Quitting. cinnamon process seems to behave well with $rss MiB resident memory");
} else {
logger("Quitting without finding the cinnamon process")
}
last;
}
sleep(1);
undef $pid if ((defined $pid) && !is_target($cmd, $pid));
$pid = find_target($cmd) unless (defined $pid);
next unless (defined $pid); # No target process running yet?
$rss = resident_memory($pid);
if ($rss > $limit) {
$timer = 0;
logger("Restarting cinnamon: Has reached $rss MiB resident memory");
system('/bin/systemctl', 'restart', 'mdm') == 0
or die("Failed to restart mdm service\n");
}
}
};
if ($@) {
logger("Process died: $@");
exit(1);
}
exit(0);
######## SUBROUTINES ########
sub find_target {
my $target = shift;
foreach my $entry (</proc/*>) {
my ($pid) = ($entry =~ /^\/proc\/(\d+)$/);
next unless (defined $pid);
return $pid
if (is_target($target, $pid));
}
return undef;
}
sub is_target {
my ($target, $pid) = @_;
my ($cmdline, $c);
local $/;
open (F, "/proc/$pid/cmdline") or return 0;
$cmdline = <F>;
close F;
return 0 unless (defined $cmdline);
($c) = ($cmdline =~ /^([^\x00]*)/);
return $target eq $c;
}
sub resident_memory {
my ($pid) = @_;
local $/;
open (F, "/proc/$pid/statm") or return 0;
my $mem = <F>;
close F;
return 0 unless (defined $mem);
my @entries = ($mem =~ /(\d+)/g);
return 0 unless (defined $entries[1]); # Should never happen
return int($entries[1] / 256); # Convert pages to MiBs
}
sub logger {
my $msg = shift;
system('/usr/bin/logger', "cinnamon-watchdog: $msg") == 0
or warn("Failed to call logger\n");
}
And this service unit file to /etc/systemd/system/cinnamon-watchdog.service:
[Unit]
Description=Cinnamon watchdog service
[Service]
ExecStart=/usr/local/bin/cinnamon-watchdog.pl
Type=simple
[Install]
WantedBy=multi-user.target
Enable service:
# systemctl enable cinnamon-watchdog
Created symlink from /etc/systemd/system/multi-user.target.wants/cinnamon-watchdog.service to /etc/systemd/system/cinnamon-watchdog.service.
That’s it. The service will be active on the next reboot
A few comments on the Perl script
- The script monitors the memory consumption because it’s a simple and safe indication of the problem, not the problem itself.
- It accesses the files under /proc/ directly rather than obtaining the information from e.g. ps. I suppose the proc format is more stable than ps’ output. The ps utility obtains its information from exactly the same source.
- The script exits voluntarily after slightly more than $timeout seconds after its invocation or its last intervention. In other words, if the cinnamon process hogs memory after its first five minutes, the script won’t be there to do anything. Neither should it. Its purpose is very specific.
- The 200 MiB limit is based upon experience with my own system: It usually ends up with 123 MiB or so.
Random vaguely related notes
Before going for this ugly solution, I actually tried to find the root of the problem (and obviously failed). Below are some random pieces of info I picked up while doing so.
The mdm setup files, in this order:
- /usr/share/mdm/defaults.conf
- /usr/share/mdm/distro.conf
- /etc/mdm/mdm.conf
Things I tried that didn’t help:
- removing Background line from /var/lib/AccountsService/users/ as suggested below (didn’t make any difference)
- Replacing autologin with a timed login of 10 seconds (I suspected that the problem was that Accountservice wasn’t up when it was required to say which background to put).
Additional ramblings
It seemed to be a result of the Accountservice daemon attempting to fiddle with the background image.
So in /var/lib/AccountsService/users/ there’s a file called “eli” (my user name, obviously), which says
[User]
Background=/usr/share/backgrounds/linuxmint/default_background.jpg
SystemAccount=false
So what happens if I remove the Background line? Nothing. That is, the problem remains as before.
A sample .xsession-errors when things went wrong
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/u
pstart: Connection refused
syndaemon: no process found
/etc/mdm/Xsession: Beginning session setup...
localuser:eli being added to access control list
** Message: couldn't access control socket: /run/user/1010/keyring/control: No s
uch file or directory
** Message: couldn't access control socket: /run/user/1010/keyring/control: No s
uch file or directory
SSH_AUTH_SOCK=/run/user/1010/keyring/ssh
SSH_AUTH_SOCK=/run/user/1010/keyring/ssh
(cinnamon-settings-daemon:2002): power-plugin-WARNING **: session inhibition not
available, cinnamon-session is not available
(cinnamon-settings-daemon:2002): power-plugin-WARNING **: session inhibition not
available, cinnamon-session is not available
Introduction
These are my notes as I made Pulseaudio work on an ARM v7-based Embedded Lubuntu 16.04, which doesn’t support Pulseaudio otherwise.
The goal: On a mini-distribution based upon Lubuntu, for use of others, make Pulseaudio work even though the Lubuntu desktop won’t start it. In fact, it’s supposed to run even without any X server running.
Being an embedded distribution, basically everything (except for certain daemons that drop their privileges) runs as root. Nothing one should try on a desktop computer, but it’s a very common practice on embedded mini-distros. Raspbian excluded.
The problem
There are basically two ways to run pulseaudio: Per-user (actually, per-session) and in system mode, which is discouraged, mostly for security reasons. Given that pulseaudio runs as root on a system where the user is root, I’m not so sure it would have made such a difference. This way or another, I went for user mode.
Which bring me to the actual problem: On a systemd-based OS, pulseaudio expects the XDG_RUNTIME_DIR environment variable to be set to /run/user/UID on its invocation, where UID is the number of the user which shall have access to Pulseaudio. Not only that, this directory must exist when pulseaudio is launched.
Some systemd background
The /run/user/ tmpfs-mounted directory is maintained by pam_systemd (see man pam_systemd, and possibly this too), which adds an entry with a UID when the respective user starts its first session (typically by logging in somehow), and removes this directory (and its content) when the last session for this user is finished.
Among other things pam_systemd does when a user creates its first session, is starting a new instance of the system service user@.service, which runs the systemd user manager instance (typically the template service /lib/systemd/system/user@.service). Which in turn calls “/lib/systemd/systemd –user” with the user ID set to the relevant uid by virtue of a “User=” directive in the said service unit file. So this is how we end up with something like:
# systemctl status
[ ... ]
└─user.slice
└─user-0.slice
├─user@0.service
│ └─init.scope
│ ├─2227 /lib/systemd/systemd --user
│ └─2232 (sd-pam)
├─session-c1.scope
│ ├─2160 /bin/login -f
│ ├─2238 -bash
[ ... ]
An important aspect of the systemd –user call is that User Services are executed as required: The new user-systemd reads unit files (i.e. the *.wants directories) from ~/.config/systemd/user/, /etc/systemd/user/ and /usr/lib/systemd/user/.
This feature allows certain services to run as long as a specific user has at least one session open, running with this user’s UID. Quite helpful for the Pulseaudio service.
From a practical point of view, the difference from regular systemd services is that the service files are put in the systemd/user directory rather than systemd/system. The calls to systemctl also need to indicate that we’re dealing with user services. The –user flag makes systemctl relate to user services that are enabled or disabled for each user separately, while the –global flag relates to user services that are enabled or disabled for all users. This is reflected in the following:
# systemctl --user enable tryuser
Created symlink from /root/.config/systemd/user/default.target.wants/tryuser.service to /etc/systemd/user/tryuser.service.
which is just for the user calling systemctl, versus
# systemctl --global enable tryuser
Created symlink /etc/systemd/user/default.target.wants/tryuser.service, pointing to /etc/systemd/user/tryuser.service.
which enables a user service for each user that has a session in the system. Note that in both cases, the original service unit file was in the same directory, /etc/systemd/user/.
Another little remark: Since the launch of a user service depends on a very specific event (at least one user having a session), the WantedBy directive is typically set ot default.target. No need to fuss.
And almost needless to say, there might be several user services running in parallel, one for each user having an active session. Note however that neither “sudo” or “su” generate a new session, as they merely start a process (with a shell) with another user (root).
A pulseaudio service
This solution isn’t perfect, but will probably work well for most people. That is, those who simply log in as themselves (or use auto-login).
Add this as/etc/systemd/user/my_pulseaudio.service:
[Unit]
Description=User Pulseaudio service
After=dbus.service
Requires=dbus.service
[Service]
Environment="XDG_RUNTIME_DIR=/run/user/%U"
ExecStart=/usr/bin/pulseaudio
Type=simple
[Install]
WantedBy=default.target
Note that the service depends and runs after dbus.service, since Pulseaudio attempts to connect to DBus, among others. Probably not very important, as DBus is likely to already run when a user session starts. Besides, in my specific case, the connection with DBus failed, and yet Pulseaudio worked fine. From /var/log/syslog:
Nov 30 16:28:18 localhost pulseaudio[2093]: W: [pulseaudio] main.c: Unable to contact D-Bus: org.freedesktop.DBus.Error.NotSupported: Unable to autolaunch a dbu
s-daemon without a $DISPLAY for X11
Also pay attention to the %U substitution into the numeric UID, while setting the environment.
To enable this service for all users,
# systemctl --global enable my_pulseaudio
Created symlink /etc/systemd/user/default.target.wants/my_pulseaudio.service, pointing to /etc/systemd/user/my_pulseaudio.service.
The multi-user problem
In a way, it’s quite interesting that the standard method for using Pulseaudio gives access to a single user. In particular, as Pulseaudio was originally developed to allow simultaneous access of the sound hardware by several pieces of software. Even so, it ended up tuned to the single user scenario: One single user sitting in front the computer. If there are processes belonging to other users, they are bogus users, such as “apache” or “dovecot”, which are intended for controlling privileges.
In recent distributions, it seems like pulseaudio is started by the X server, with the user owning it (i.e. the one who has logged in, typically through the GUI login interface). Once again, access to the sound card is given to one single user, who is supposedly sitting in front of a computer. This is the model Microsoft grew its operating systems with.
There’s always the system mode alternative or allowing TCP connections, but these are not mainstream. In theory, there should have been some privilege access mechanism for Pulseaudio, allowing an arbitrary group of clients to connect as required. But since the by far most common usage scenario of sound is part of a GUI experience for the user in front of the machine, Pulseaudio was shaped accordingly.
The main problem of the solution above is that it doesn’t work in a multi-user scenario: The first user creating a session will have audio access. If another user creates a session, the attempt to launch pulseaudio will fail, leaving this user without sound.
To demonstrate how absurd it can get, suppose that user X connects to the system through ssh (and hence generates a session). This user will have pulseaudio running on its behalf, even though it can’t play any sound on the system. Then user Y logs in on the GUI console, but its pulseaudio service fails to launch, because there’s already one running. To make things even more absurd, if user X logs out, its pulseaudio service is killed, and a service for user Y doesn’t start, because what would kick it off?
It doesn’t help checking if the session that launched my_pulseaudio has a console (man loginctl for example), because user X might log in with ssh first (my_pulseaudio launched, but doesn’t activate a pulseaudio process) and then through the console (my_pulseaudio won’t be launched).
In reality, none of these problems are expected to happen often. We all end up logging into a computer with one specific user.
A less preferred alternative: Waiting for /run/user/0
As I tried to find a proper solution, I came up with the idea to launch the service when /run/user/o is created. It’s a working solution in my specific case, where only root is expected to log in.
The idea is simple. Make a path-based launch of the service when /run/user/0 is created.
/etc/systemd/system/my_pulseaudio.path is then:
[Unit]
Description=Detection of session for user 0 before launching Pulseaudio
[Path]
PathExists=/run/user/0
[Install]
WantedBy=paths.target
And the following is /etc/systemd/system/my_pulseaudio.service:
[Unit]
Description=Pulseaudio for user 0
After=dbus.service
Requires=dbus.service
[Service]
Environment="XDG_RUNTIME_DIR=/run/user/0"
ExecStart=/usr/bin/pulseaudio
Type=simple
and then enable the service with
# systemctl enable my_pulseaudio.path
Created symlink from /etc/systemd/system/paths.target.wants/my_pulseaudio.path to /etc/systemd/system/my_pulseaudio.path.
This works effectively like the user service for pulseaudio suggested above, but only for user 0. It might be modified to watch for changes in /run/user, and launch a script which evaluates which actions to take (possibly killing one pulseaudio daemon in favor of another?). But it will still not solve the case where a user logs in with ssh first, and then through the console. So the perfect solution should probably hook on session starts and terminations. However that is done.
Background
On November 1, 2017, the Israeli broadcast authorities split Channel 2 into two channels, 12 and 13. To avoid unfair competition with the other channels, as channel 2 might lose viewers due to the confusion, channel 10 was moved to 14 as well. This requires a rescan of DVB receivers. The changes are rather minor, it turns out.
For the record, I still haven’t attempted to receive the DVT-T2 channels. Not enough motivation, and it probably requires some work with my DVB-T2 dongle (drivers etc.)
The scan
Running a scan in Haifa, Israel, on November 1st 2017 (quite evidently, I received on 538 MHz):
$ dvbv5-scan /usr/share/dvb/dvb-t/il-All
Cannot calc frequency shift. Either bandwidth/symbol-rate is unavailable (yet).
Scanning frequency #1 514000000
(0x00)
Scanning frequency #2 538000000
Lock (0x1f)
Service Ch 11, provider Idan +: digital television
Service Ch 23, provider Idan +: digital television
Service Ch 12 Keshet, provider Idan +: digital television
Service Ch 13 Reshet, provider Idan +: digital television
Service Ch 14 10, provider Idan +: digital television
Service Ch 99, provider Idan +: digital television
Service Tarbut, provider Idan +: digital radio
Service Bet, provider Idan +: digital radio
Service Gimmel, provider Idan +: digital radio
Service Makan, provider Idan +: digital radio
Service Moreshet, provider Idan +: digital radio
Service Kan88, provider Idan +: digital radio
Service Musica, provider Idan +: digital radio
Service Reka, provider Idan +: digital radio
Service Galatz, provider Idan +: digital radio
Service Galgalatz, provider Idan +: digital radio
Service Radios, provider Idan +: digital radio
Service Kol Hai, provider Idan +: digital radio
Service Kol Barama, provider Idan +: digital radio
Service Lev HaMdina, provider Idan +: digital radio
Service CLASSICAL bu, provider Idan +: digital radio
Looking at the new dvb_channel.conf, the following changes are evident:
- “Ch 1″ has been renamed to “Ch 11″, and a new audio PID has been added to it (2563)
- “Ch 2″ has been replaced with “Ch 12 Keshet”. Two subtitle PIDs has been dropped (2610 and 2609)
- “Ch 10″ has been replaced with “Ch 14 10″, same PIDs for video/audio, but the subtitles have moved from 2642, 2641 and 2640 to 1039, 1038 and 1037.
- “Ch 33″ has been replaced with “Ch 13 Reshet”, with subtitle PID 1033 added.
- The radio channel “Aleph” has been renamed to “Tarbut”
- The radio channel “Dalet” has been renamed to “Makan”
- The radio channel “88FM” has been renamed to “Kan88″.
- A new radio channel, “Kol Hai” has been added with service ID 40.
So from a DVB point of view, “Ch 12 Keshet” is the new Channel 2.
The updated dvb_channel.conf
Obtained with the dvb5-scan as shown above:
[Ch 11]
SERVICE_ID = 1
VIDEO_PID = 2561
AUDIO_PID = 2562 2563
PID_06 = 1025
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Ch 23]
SERVICE_ID = 2
VIDEO_PID = 3585
AUDIO_PID = 3586
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Ch 12 Keshet]
SERVICE_ID = 3
VIDEO_PID = 2593
AUDIO_PID = 2594 2595
PID_06 = 2608
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Ch 13 Reshet]
SERVICE_ID = 4
VIDEO_PID = 2657
AUDIO_PID = 2658
PID_06 = 1033
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Ch 14 10]
SERVICE_ID = 5
VIDEO_PID = 2625
AUDIO_PID = 2626 2627
PID_06 = 1039 1038 1037
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Ch 99]
SERVICE_ID = 6
VIDEO_PID = 2689
AUDIO_PID = 2690
PID_06 = 2704
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Tarbut]
SERVICE_ID = 21
AUDIO_PID = 2817
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Bet]
SERVICE_ID = 22
AUDIO_PID = 2833
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Gimmel]
SERVICE_ID = 23
AUDIO_PID = 2849
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Makan]
SERVICE_ID = 24
AUDIO_PID = 2865
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Moreshet]
SERVICE_ID = 25
AUDIO_PID = 2881
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Kan88]
SERVICE_ID = 26
AUDIO_PID = 2897
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Musica]
SERVICE_ID = 27
AUDIO_PID = 2913
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Reka]
SERVICE_ID = 28
AUDIO_PID = 2929
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Galatz]
SERVICE_ID = 29
AUDIO_PID = 2945
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Galgalatz]
SERVICE_ID = 30
AUDIO_PID = 2961
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Radios]
SERVICE_ID = 36
AUDIO_PID = 3057
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Kol Hai]
SERVICE_ID = 40
AUDIO_PID = 3121
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Kol Barama]
SERVICE_ID = 41
AUDIO_PID = 3137
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[Lev HaMdina]
SERVICE_ID = 42
AUDIO_PID = 3153
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVBT
[CLASSICAL bu]
SERVICE_ID = 45
AUDIO_PID = 3201
FREQUENCY = 538000000
MODULATION = QAM/16
BANDWIDTH_HZ = 8000000
INVERSION = AUTO
CODE_RATE_HP = 2/3
CODE_RATE_LP = 1/2
GUARD_INTERVAL = 1/4
TRANSMISSION_MODE = 8K
HIERARCHY = NONE
DELIVERY_SYSTEM = DVB
The updated stream list
Obtained with ffplay, as shown on this post.
Program 1
Metadata:
service_name : ?Ch 11
service_provider: ?Idan +
Stream #0:26[0x401](heb): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:6[0xa01]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:27[0xa02]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:28[0xa03]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 2
Metadata:
service_name : ?Ch 23
service_provider: ?Idan +
Stream #0:0[0xe01]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1[0xe02]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 3
Metadata:
service_name : ?Ch 12 Keshet
service_provider: ?Idan +
Stream #0:17[0xa21]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:18[0xa22]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:19[0xa23]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:20[0xa30](heb): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Program 4
Metadata:
service_name : ?Ch 13 Reshet
service_provider: ?Idan +
Stream #0:15[0x409](heb): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:7[0xa61]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:16[0xa62]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 5
Metadata:
service_name : ?Ch 14 10
service_provider: ?Idan +
Stream #0:32[0x40d](heb): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:33[0x40e](rus): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:34[0x40f](ara): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:29[0xa41]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:35[0xa42]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:36[0xa43]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 6
Metadata:
service_name : ?Ch 99
service_provider: ?Idan +
Stream #0:21[0xa81]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg), 720x576 [SAR 12:11 DAR 15:11], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:22[0xa82]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:23[0xa90](heb): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Program 21
Metadata:
service_name : Tarbut
service_provider: Idan +
Stream #0:30[0xb01]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 22
Metadata:
service_name : Bet
service_provider: Idan +
Stream #0:24[0xb11]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 23
Metadata:
service_name : Gimmel
service_provider: Idan +
Stream #0:31[0xb21]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 24
Metadata:
service_name : Makan
service_provider: Idan +
Stream #0:12[0xb31]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 25
Metadata:
service_name : Moreshet
service_provider: Idan +
Stream #0:11[0xb41]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 26
Metadata:
service_name : Kan88
service_provider: Idan +
Stream #0:14[0xb51]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 27
Metadata:
service_name : Musica
service_provider: Idan +
Stream #0:3[0xb61]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 28
Metadata:
service_name : Reka
service_provider: Idan +
Stream #0:8[0xb71]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 29
Metadata:
service_name : Galatz
service_provider: Idan +
Stream #0:13[0xb81]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 30
Metadata:
service_name : Galgalatz
service_provider: Idan +
Stream #0:5[0xb91]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 36
Metadata:
service_name : Radios
service_provider: Idan +
Stream #0:2[0xbf1]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 40
Metadata:
service_name : Kol Hai
service_provider: Idan +
Stream #0:25[0xc31]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 41
Metadata:
service_name : Kol Barama
service_provider: Idan +
Stream #0:9[0xc41]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 42
Metadata:
service_name : Lev HaMdina
service_provider: Idan +
Stream #0:4[0xc51]: Audio: aac_latm (HE-AACv2) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Program 45
Metadata:
service_name : CLASSICAL bu
service_provider: Idan +
Stream #0:10[0xc81]: Audio: aac_latm (HE-AAC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Let’s hope they don’t fiddle with this anymore…
Mail with gmail.com as From address vanishing
It started really bad: Someone asked me why he hasn’t received an answer from me in two weeks, and I had answered his mail the same day I got his.
It turned out that Gmail had thrown mail into the black hole without any warning. Probably the updated DMARC policy, which has been mentioned for long now.
Solution: When using gmail.com as a From address, also use smtp.gmail.com as the outgoing SMTP mail server. This ensures that the mail arrives properly to gmail.com recipients as well.
Bonus: The sent mails appear in Google’s web interface’s Sent folder as well. I would have done well without this favor.
That was really annoying, frankly speaking.
SPF entries
There’s another comprehensive post on SPF, DKIM and DMARC. It’s important to keep these entries properly set and to update them when the outgoing servers change.
This (SPF) mechanism helps spam filters tell if the sender of the mail is authentic, by looking for an SPF record in the domain name’s TXT record.
For example, since I’m relaying through my web host’s server, billauer.co.il, the desired SPF record should read:
v=spf1 +a +mx +ip4:208.76.80.100 +ip4:46.232.183.0/24 +ip4:23.83.208.1/20 +ip4:177.153.0.128/25 +ip4:191.252.57.0/25 -all
- The +a part means to pass the mail if the A entry if the sending domain appears
- The +mx means the same for the MX entry
- The other IP4 parts say that if these addresses (or address blocks) appear, pass the mail
- Finally, the -all part in the end says that if none of the previous entires matches, drop the mail
(In Cpanel, go to EMail > Authentication to set this up)
A word about “include records”. I used to have one saying “+include:relay.mailchannels.net”, which means “get the SPF record from relay.mailchannels.net”, and add whatever records they have. Which makes sense in a way, since their servers are expected to appear on the list. On the other hand, if this record is missing (or their DNS temporary out of business), it’s a fatal error. So I’m not happy about this idea. The solution above, copying their records (which is the long list of IP4 address blocks), is suboptimal in that I may miss some new servers or so, but this can’t cause a fatal error.
Checking the current SPF record
(actually, it has changed since)
$ dig txt billauer.co.il
; <<>> DiG 9.6.2-P2-RedHat-9.6.2-5.P2.fc12 <<>> txt billauer.co.il
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1406
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2
;; QUESTION SECTION:
;billauer.co.il. IN TXT
;; ANSWER SECTION:
billauer.co.il. 14400 IN TXT "v=spf1 +a +mx +ip4:208.76.80.100 +ip4:46.232.183.0/24 +ip4:23.83.208.1/20 +ip4:177.153.0.128/25 +ip4:191.252.57.0/25 -all"
;; AUTHORITY SECTION:
billauer.co.il. 28920 IN NS ns2.totalchoicehosting.com.
billauer.co.il. 28920 IN NS ns1.totalchoicehosting.com.
;; ADDITIONAL SECTION:
ns1.totalchoicehosting.com. 4736 IN A 208.76.85.138
ns2.totalchoicehosting.com. 131050 IN A 192.96.216.208
;; Query time: 173 msec
;; SERVER: 194.90.0.1#53(194.90.0.1)
;; WHEN: Thu Sep 7 18:23:52 2017
;; MSG SIZE rcvd: 256
Checking directly on the authoritative DNS is better when monitoring quick changes.
General
Gtkwave is a simple and convenient viewer of electronic waveforms. It’s a free software tool at its best: A bit rough to start working with, but after a while it becomes clear that the decisions have been made by someone who uses the tool himself. Really recommended.
So here are my jumpstart notes for the next time I’ll need it.
Invocation
$ gtkwave this.vcd &
Note to self: My own utility for generating VCD files from raw dumps, named dump2vcd, is in the utils repository.
Saving a signal view set
To have a certain set of signals shown, as well as a certain time region shows, create a “Save File”: In the top menu go File > Write Save File. The automatic filename extension is .sav.
At the next session, just read the save file with File > Read Save File. Or from the command line e.g.
$ gtkwave this.vcd allview.sav &
Markers
Note that hovering the mouse pointer over a certain signal’s trace will make it sticky on transitions.
- Plain left-click: Sets the “current position”. Values seen at left.
- Left click and drag: Time difference shown at the top
- Right-click and drag: Zoom in
- Middle-button (wheel) click: Set the “base marker”. Then move around the primary marker with plain left-clicks, and the time difference between the base marker and primary will appear as the top.
Other nice stuff
- Find Next / Previous Edge (plain right/left arrows at the toolbar). Select one or several signals on the Signals list, and click on these icons to jump to the next edge of those signals.
- If a set of signal are named with index suffixes, e.g. state[0] and state[1], Gtkwave automagically shows them as a vector (e.g. state[1:0]).
