CloudStack Management Server Database Initialization Failure with Galera Cluster #12659
Description
Description:
We are experiencing an issue during the database initialization of our CloudStack environment. Although the database configuration script completes successfully, the logs indicate failures during the process. We need guidance to resolve this issue and ensure proper HA setup with Galera and ProxySQL.
Environment Overview
- Database backend: MariaDB Galera Cluster
- Load balancing layer: ProxySQL
- Virtual IP managed by: Keepalived
- Cloud platform: Apache CloudStack
- CloudStack Management Servers: 3 nodes (HA setup)
- ProxySQL Servers: 3 nodes (HA setup)
- ProxySQL version : 3.0.5-60-g7e9e009
- Database: MariaDB Galera Cluster (3 nodes)
- Database Version : 10.6.25-MariaDB
- OS: AlmaLinux release 9.7
- CloudStack Version: cloudstack-management-4.22.0.0-shapeblue0.noarch
- For troubleshooting purpose firewalld is stopped
Current Setup
- We have multiple Galera nodes configured in synchronous replication mode.
- ProxySQL is deployed on two nodes with Keepalived managing a Virtual IP (VIP).
- The application connects to the database via the VIP address.
- VIP failover is working correctly.
- Database connectivity through the VIP is verified and functioning.
- Manual MySQL connections via the VIP are successful.
When I see the /etc/cloudstack/management/db. properties file it has pointed to the VIP and but in the logs it showing trying to connect local host
Logs :
2026-02-18 16:11:50,951 INFO [c.c.u.d.T.Transaction] (main:[]) (logid:) Using the following URI to connect to cloud database [jdbc:mysql://10.1.2.7:6033/cloud?autoReconnect=true&prepStmtCacheSize=517&cachePrepStmts=true&sessionVariables=sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION'&serverTimezone=UTC&scrollTolerantForwardOnly=true].
2026-02-18 16:11:50,955 DEBUG [c.c.u.d.DriverLoader] (main:[]) (logid:) Successfully loaded DB driver com.mysql.cj.jdbc.Driver
2026-02-18 16:11:50,955 DEBUG [c.c.u.d.T.Transaction] (main:[]) (logid:) Creating datasource for database: cloud with connection pool lib: hikaricp
2026-02-18 16:11:56,481 DEBUG [c.c.u.d.T.Transaction] (main:[]) (logid:) Creating default datasource for database: cloud with connection pool lib: hikaricp
2026-02-18 16:11:56,487 DEBUG [c.c.u.d.T.Transaction] (main:[]) (logid:) Creating default datasource for database: cloud_usage with connection pool lib: hikaricp
2026-02-18 16:11:56,488 DEBUG [c.c.u.d.T.Transaction] (main:[]) (logid:) Creating default datasource for database: simulator with connection pool lib: hikaricp
2026-02-18 16:11:56,489 WARN [c.c.u.d.T.Transaction] (main:[]) (logid:) Unable to load db configuration, using defaults with 5 connections. Falling back on assumed datasource on localhost:3306 using username:password=cloud:cloud. Please check your configuration com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
at com.zaxxer.hikari.pool.HikariPool.throwPoolInitializationException(HikariPool.java:584)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:571)
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:98)
at com.zaxxer.hikari.HikariDataSource.(HikariDataSource.java:80)
at com.cloud.utils.db.TransactionLegacy.createHikaricpDataSource(TransactionLegacy.java:1325)
at com.cloud.utils.db.TransactionLegacy.createDataSource(TransactionLegacy.java:1279)
at com.cloud.utils.db.TransactionLegacy.initDataSource(TransactionLegacy.java:1106)
at com.cloud.utils.db.TransactionLegacy.(TransactionLegacy.java:1025)
at com.cloud.utils.db.Merovingian2.(Merovingian2.java:69)
at com.cloud.utils.db.Merovingian2.createLockController(Merovingian2.java:89)
at com.cloud.server.LockControllerListener.(LockControllerListener.java:33)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
Additional Observation:
In the management server logs, the above entries are repeatedly printed:
Despite the service starting, port 8080 does not appear to be listening on the server initially.
After manually restarting the management service, port 8080 becomes active and accessible.
I apologize if this ticket doesn’t fully follow the usual format. I would really appreciate any guidance or advice from the team on this issue.