Features/Cluster

Updated Cluster Stack with Enhanced Failover Support

Summary

This is a significant update to the clustering stack - both for high availability and load balancing.

Owner

• Name: FabioMassimoDiNitto
• email: fdinitto@redhat.com
• Name: LonHohberger
• email: lhh@redhat.com

Current status

• Targeted release: Fedora 42
• Last updated: 26 Mar 2012
• Percentage of completion: 100%

Detailed Description

Upstream has been developing a new version of the cluster stack which is currently shipped with Fedora 16. The major features are:

• Improved quorum subsystem which is integrated in to the Corosync Cluster Engine (95% complete)
• A new command-line interface for administration of both the Corosync Cluster Engine and the Pacemaker Cluster Resource Manager as well as monitoring (50% complete)
• Convergence on a single set of resource agents from the Linux clustering community
• Enhanced fencing support provided by stonith-ng, which is part of the Pacemaker project.
• Separation of GFS2-specific utilities in to a separate project (100%)
• Separation of DLM-specific utilities in to a separate project (90%)

Note that this transition includes deprecations:

• The cluster package, including CMAN, DLM utilities, GFS2 utilities, fenced, and all associated libraries.
• The rgmanager package. All users are advised to use Pacemaker for their failover needs.
• The openais package. This provided some of the SA Forum AIS APIs, which were rarely used.
• The heartbeat package. Upstream development has stopped and equivalent or superior functionality is provided by the Corosync package.
• The piranha package. Superior functionality is provided by the Keepalived package.
• The ricci package. Similar agent-style communication can be provided by Matahari or other projects.
• The luci package. This will be replaced by Sunzi, although it is not clear this part will make Fedora 17.
• The clustermon package. This was part of the ricci/luci administration stack and is no longer required.
• Several agents in the resource-agents package which previously relied on examination of cluster.conf at run-time.

Benefit to Fedora

The increased reliability and versatility of the cluster components included in Fedora 17 allow administrators to deploy Fedora in environments where greater availability and clustered file systems are required. Removal of rgmanager matches the capabilities offered by several other Linux distributions and allows the Fedora community to consort efforts on a single failover stack.

Scope

This change impacts several utilities and other applications, including:

• The DLM, which used AIS Checkpoints to replicate data about POSIX locks (100% complete)
• Clustered LVM, which used CMAN APIs, will need to be ported to corosync. (?% complete)
• Fence-virt, which used CMAN APIs and AIS Checkpoints, will need to be ported to corosync. (0% complete)
• Pacemaker, which used CMAN APIs, will need to be ported to corosync (100% complete)
• Clustered mirroring, which uses AIS Checkpoints will need to be ported to corosync (?% complete)
• QPid, which used CMAN APIs, will need to be ported to corosync (?% complete)

How To Test

The initial use cases required for high availability are documented in the Clusters from Scratch manual.

Information about how to run GFS2 in standalone mode will be provided as development nears completion.

User Experience

Some differences will be noticeable by users:

• Users of the rgmanager package will need to learn the Pacemaker package, which is a more dynamic, complex, full-featured cluster resource manager.
• Users who relied on cluster.conf (whether using ccs, luci, or hand-editing) will need to learn:
  • The crm and/or pcs commands and become familiar with the CIB, which is an XML database of the failover resource configuration combined with some run-time state information.
  • The layout, syntax, and use of corosync.conf
• Users who relied on the clustat command will need to learn the following commands:
  • cibadmin
  • crm_mon
  • crm_resource

Dependencies

• corosync (2.0)
• pacemaker
• cluster-glue
• dlm (in review currently)
• libqb
• pcs (in review currently)

All packagers and upstreams have been informed upfront and worked together to achieve this goal.

Contingency Plan

None necessary

Documentation

• Clusters from Scratch
• Corosync FAQ

Release Notes

• The update to version 2.0 (needle) of the Corosync Cluster Engine offers:
  • API stability guarantees for the duration of Corosync 2.0's lifetime
  • High performance, cluster-wide messaging.
    • The performance of corosync 2.0 is nearly 10 times that of corosync 1.x for smaller (<64kb) message sizes
    • The CPU utilization of corosync 2.0 is approximately 1/5th of corosync 1.x
  • Improved quorum subsystem
  • The confdb API has been replaced with the cmap API for configuration management.
• The update to version 1.1.7 of the Pacemaker Cluster Resource Manager offers:
  • Improved fencing subsystem which can be used in either GFS2 standalone environments or resource-driven clusters (which may or may not include GFS2)
  • Improved logging via the libqb framework

Comments and Discussion

• See Talk:Features/Cluster