This happened a while ago. After applying some updates to my LXD containers and restarted everything, I was surprised to discover that all my websites depending on MariaDB weren’t able to connect to the server.
When I tried to connect for the CLI on one of the affected containers, I go this weird error:
[email protected] ~# mysql --host mysql.kokoro.infra -u wordpress -p Enter password: ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 107 "Transport endpoint is not connected"
With the IPv4 of the MariaDB container:
[email protected] ~# mysql --host 10.80.245.19 -u wordpress -p Enter password: ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0 "Internal error/check (Not system error)"
These are some unusual errors that seem more low-level than I’m used to. I couldn’t find much about them. I tried to disable AppArmor on the containers without luck.
Then this helped. What happens if we use netcat to connect to MariaDB?
[email protected] ~# nc 10.80.245.19 3306 Host '10.80.245.15' is not allowed to connect to this MariaDB server
Now, that’s a much more explicit error compared to what the MariaDB client was returning.
The MySQL users were using their respective
@<container>.lxd hostname. For example
[email protected]. I don’t know what changed with the updates I made, but something did. After updating all my users to use the IPv4 of their respective container instead of the
.lxd domains, I was able to connect:
[email protected] ~# nc 10.80.245.19 330 u 5.5.5-10.4.13-MariaDB-1:10.4.13+maria~stretch[...]
[email protected] ~# mysql --host 10.80.245.19 -u wordpress -p Enter password: Welcome to the MariaDB monitor. Commands end with ; or \\g. Your MariaDB connection id is 80 Server version: 10.4.13-MariaDB-1:10.4.13+maria~stretch-log mariadb.org binary distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\\h' for help. Type '\\c' to clear the current input statement. MariaDB [(none)]>
Not sure what happened…