MariaDB Server Unresponsive

Oops, it’s in /var/log/messages now or dmesg - interested in boot diagnostics not the startup output.

I would say probably not if it was bad memory it would causes issues on host server not just one VM.

2 Likes

https://pastebin.com/aewmSSjv

https://pastebin.com/X1PDhLbE

Warning: AMD Processor - this hardware has not undergone upstream testing. Please consult http://wiki.centos.org/FAQ for more information

Install a 4.x or 5.x kernel off ELRepo

[Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR c0010001 is 530076)

I doubt this is significant but something to explore after upgrading the kernel depending upon tuned profile (tuned-adm active should report “virtual-guest”) or power management settings.

Edit: it’s detected as a generic K7 processor, which may have something to do with it.

AMD Common KVM processor (fam: 0f, model: 06, stepping: 01)

Any chance you can expose the processor as something more native?

Is it safe for a server in production?

Changed and now it says:

smpboot: CPU0: AMD Ryzen 9 3900X 12-Core Processor (fam: 17, model: 71, stepping: 00)

It’s safer than your server going down spontaneously and running into an unrecoverable corruption event with InnoDB. Stick with 4.x long-term kernels if changing the CPU vendor doesn’t resolve your kernel panic. Also read that you’re nesting virtualization in the BP; perhaps this is related?

1 Like

Shit. Happened again now with the other guest.

Time to say good bye to Proxmox this time. I will rebuild the server as a single host.

Thank you so much @nem for your help.

1 Like

Instead of KVM you could try container based VM eg OpenVZ 7 not sure if it would work any better.

Might just be better to have single host probably less problematic.

1 Like

This is really really weird.

As a summary, I was using 2 VMs on Proxmox with nested virtualization: one for web server and another for database. There were times when the DB started to hang, VM completely frozen. Then the web server VM did the same too.

Removed Proxmox and now the whole VM is a stack of CentOS 8 + Direct Admin + OLS + Maria DB. It was installed yesterday and a moment ago MariaDB started to use the entire CPU and everything else stopped responding. After restarting mariadb server, the website came back completely normal.

Wwwwhhhyyyy.

How’s the memory usage?

Just 90M on swap.

mysqladmin processlist - you have to see what’s going on to make a conjecture, then go from there. Any long-running queries at that time when your CPU usage swells? What’s your queries/second rate in a normal setting and when your CPU usage swells? Something isn’t like the others and that’s what you need to focus on now.

1 Like

Bit late but check if you have ran into any MariaDB 10.4 query optimizer bugs some can chew up cpu cycles due to unoptimized query optimizer paths taken. Some of the bugs and MariaDB jira issue trackers for 10.4+ are listed and mentioned in the thread on my forum as it was brought up by some of my users https://community.centminmod.com/threads/anyone-using-mariadb-10-4-8-on-live-and-busy-site.18327/

Personally, I’d sticking to MariaDB 10.3 until 10.4+ matures more. You can track MariaDB official Jira bug tracker for issues that come up too.

1 Like

Sorry to hijack, but since you’re here… What’s your opinion on MySQL 8? MariaDB just had another bad release in roughly a year and it’s had me rethink which MySQL derivative to go with.

While Googling malformed communication packet errors, I did come across one on MySQL 8 so maybe MariaDB folks recent update is backporting stuff from MySQL 8 too ? I’m still privately testing MySQL 8 so not really in production use other than a few Centmin Mod users wanting MySQL 8 so I wrote a MariaDB 10 to MySQL 8 switching routine for fresh installs. I’m more cautious due to the higher likelihood of MariaDB 10.4+ and MySQL 8.0+ being one way upgrades compared to MySQL 5.7 and MariaDB <=10.3. While I am more versed in technicality of the major downgrade process. End users of mine wouldn’t and providing free support for MySQL 8/MariaDB 10.4 downgrades or switches wouldn’t be feasible.

I’m already testing backported PHP mysqlnd fixes malformed communication packet errors for PHP 5.6, 7.0, 7.1, 7.2 using PHP 7.3 mysqlnd code which seems to fix the MariaDB update issues https://community.centminmod.com/threads/mariadb-10-5-7-10-4-16-10-3-26-10-2-35-and-10-1-48-now-available.20632/#post-86953 for at least Wordpress and Xenforo forums. So they don’t require upgrading to PHP 7.3+

1 Like