Unfortunately, having a list of IPs per RHQ Server is not supported. You'd have to do your second suggestion: make sure the agents' /etc/hosts resolve correctly.
This is addressed in the HA docs:
"Thus, it is critical that every RHQ Agent be able to resolve the Endpoint Address set for every RHQ Server in the HA Server cloud. So, when defining the RHQ Server in the installer, it is important that the Endpoint Address be public to the degree that the RHQ Agent population can resolve the RHQ Server's address and be able to reach the RHQ Server via the defined address and port"
and in the blue box just below that:
"Each server has a public endpoint address associated with it (which can be either a hostname or IP address). Those server public endpoints are used as failover list entries. Therefore, it is very important that all servers are assigned public endpoint addresses that are resolvable by all agents and that all agents have connectivity to those addresses."