What is this alert?
When a message with severity FATAL is written to the Greenplum Database log on the master host, Command Center raises an alert.
A message with severity level FATAL indicates an error occurred that caused the current session to fail. Fatal errors can be caused by user error, such as misspelling a database name, or by system error, such as an out of memory error, failure to access a resource, or network failure.
Here are some examples that could result in FATAL messages:
- A user or application attempts to connect to a database that does not exist.
- The master host is unable to create a connection to a segment host.
- A memory allocation request is refused due to low memory.
- A user or administrator cancels a command.
To avoid excessive alert emails, Command Center does not raise alerts for fatal password authentication errors.
What to do?
The message text often provides the information needed to determine the cause and remedy for the error.
Recurring errors without an evident cause require investigation to find the problem.
Here are some questions to consider.
Does the Command Center Dashboard indicate a normal Database State? Also check the Dashboard for low disk space, unusually high CPU or Memory usage, and a large number of blocked queries. Problems indicated on the Dashboard can help identify the source of problems.
Does the error occur only when the Greenplum Database system is under heavy load? Some system resources can be depleted under heavy load and may require reconfiguration. For example:
- The SSH service on segment hosts may limit the number of active connections.
- Out of memory errors could require more conservative settings for system memory parameters or tuning resource groups to limit the number of concurrent queries or the amount of memory allocated to queries.
Does the message occur with a specific query, UDF, or application? Check the Greenplum Database log files and application log files for messages that help to identify the location and cause of the error.
If the error continues and you cannot find the cause, create a ticket with VMware Support. Support may ask you to provide log files, core files, configuration files, schemas, and system information so that they can help you to resolve the problem.
