Cloudera Enterprise Upgrade Guide

The following topics provide an overview of the Cloudera Enterprise upgrade process and include complete procedures for upgrading Cloudera Manager and your CDH clusters:

Procedures for upgrading Cloudera Director are discussed in the Cloudera Director documentation. See Upgrading Cloudera Director.

Upgrade Overview

Upgrading consists of two major steps, upgrading Cloudera Manager and upgrading a CDH cluster.

Upgrading Cloudera Manager
Getting Started Upgrading Cloudera Manager Upgrading the Cloudera Manager Agents Backing Up Cloudera Manager Upgrading the Cloudera Manager Server Upgrading Cloudera Navigator Data Encryption After You Upgrade Cloudera Manager

When you upgrade Cloudera Manager, you use RPM-based package commands to upgrade the software on the Cloudera Manager server host and then Cloudera Manager manages upgrading the Cloudera Manager Agents on the remaining managed hosts. Cloudera Navigator is also upgraded when you upgrade Cloudera Manager.

Upgrading a CDH Cluster
Getting Started Upgrading CDH Backing Up CDH CDH 6 Pre-Upgrade Migration Steps CDH 6 Post-Upgrade Migration Steps Upgrading the CDH Cluster Run the Upgrade CDH Wizard

When you upgrade a CDH cluster, you use Cloudera Manager to upgrade the CDH software using Cloudera parcels across an entire cluster or you can manually install the software on all cluster hosts using RPM-based package commands and then Cloudera Manager completes the service upgrades.

You are not required to upgrade Cloudera Manager and CDH at the same time, but the versions of Cloudera Manager and CDH must be compatible. Cloudera Manager 6.0 can manage clusters running CDH 5.7 up to CDH 5.14 and as long as the major+minor version of Cloudera Manager is equal or higher than the major+minor version of CDH. For example:
  • Supported:
    • Cloudera Manager 6.0.0 and CDH 5.14.0
    • Cloudera Manager 5.14.0 and CDH 5.13.0
    • Cloudera Manager 5.13.1 and CDH 5.13.3
  • Not Supported:
    • Cloudera Manager 5.14.0 and CDH 6.0.0
    • Cloudera Manager 5.12 and CDH 5.13
    • Cloudera Manager 6.0.0 and CDH 5.6

Continue reading:

Assessing the Impact of an Upgrade

Plan for a sufficient maintenance window to perform an upgrade. Depending on which components you are upgrading, the number of hosts in your cluster, and the type of hardware, you might need up to a full day to upgrade your cluster. Before you begin the upgrade, you need to gather some information; these steps are also detailed in the Cloudera Manager and CDH upgrade procedures.

Before upgrading, consult the release notes for Cloudera Manager and CDH to learn about API changes, deprecated features, new features, and incompatible changes.

See

Also check the Cloudera Enterprise 6 Requirements and Supported Versions page to make sure that you are using a supported operating system, JDK, database, and other components.

Upgrades are not supported for all versions of Cloudera Enterprise. See Supported Upgrade Paths.
There are three types of upgrades: major, minor, and maintenance:
Major Upgrade from Cloudera Manager and CDH 5.x to 6.x or higher
A major upgrade typically has the following characteristics:
  • Large changes to functionality and update of Hadoop to a more recent version
  • Incompatible changes in data formats
  • Significant changes and additions to the user interface in Cloudera Manager
  • Database schema changes for Cloudera Manager that are automatically handled by the upgrade process
  • Significant down time is required to upgrade the cluster.
  • Client configurations are redeployed.

Minor Upgrades
Minor upgrades upgrade your software to a higher minor version of a major release—for example from version 6.0.0 to version 6.1.0—and typically include the following:
  • New functionality
  • Bug fixes
  • Potential database schema changes for Cloudera Manager that are handled automatically
  • Client configurations are redeployed.
Incompatible changes or changes to data formats are generally not introduced in minor upgrades.
Maintenance Upgrades

Maintenance upgrades fix critical bugs or address security issues. No new functionality or incompatible changes are introduced. The version numbers for maintenance releases differ only in the third digit, for example, when upgrading from version 6.0.0 to 6.0.1.

To upgrade to a maintenance release, you only need to perform a subset of the Minor version upgrade steps. Follow the same procedures as for minor version upgrades but skip the steps that are labeled as follows:

[Not required for CDH maintenance release upgrades.].

Overview of Upgrading Cloudera Manager

Cloudera Manager Upgrade

To upgrade Cloudera Manager, you perform the following tasks:
  1. Back up the Cloudera Manager server databases, working directories, and several other entities. These backups can be used to restore your Cloudera Manager deployment if there are problems during the upgrade.
  2. Upgrade the Cloudera Manager server software on the Cloudera Manager host using package commands from the command line (for example, yum on RHEL systems). Cloudera Manager automates much of this process and is recommend for upgrading and managing your CDH clusters.
  3. Upgrade the Cloudera Manager agent software on all cluster hosts. The Cloudera Manager upgrade wizard can upgrade the agent software (and, optionally, the JDK), or you can install the agent and JDK software manually. The CDH software is not upgraded during this process.

For Cloudera Manager upgrade steps, see Upgrading Cloudera Manager.

If you are upgrading from Cloudera Manager 5.x to a higher version of Cloudera Manager 5.x, you can also upgrade Cloudera Manager using tarballs. See Upgrading Cloudera Manager 5 Using Tarballs for procedures to upgrade to the latest version of Cloudera Manager 5.x. (Click Other Versions to locate the document for earlier versions.)

Overview of Upgrading CDH

CDH upgrades contain updated versions of the Hadoop software and other components. You can use Cloudera Manager to upgrade CDH for major, minor, and maintenance upgrades. The procedures vary depending on the version of Cloudera Manager you are using and the version of CDH you are upgrading to. You can use Cloudera Manager to upgrade CDH using either parcels or packages.

After completing preparatory steps, you use the Cloudera Manager upgrade wizard to complete the upgrade. If you use parcels (recommended), have enabled HDFS High Availability, and have a Cloudera Enterprise license, you can perform a rolling upgrade that does not require you to take the cluster offline during the upgrade.

For CDH upgrade steps, see Upgrading the CDH Cluster.

CDH Upgrades Using Parcels or Packages

Upgrading CDH Using Parcels (Recommended)

Upgrading CDH using parcels is the preferred method because parcels are managed by Cloudera Manager, which automatically downloads, distributes, and activates the correct versions of the software. If you have enabled high availability for HDFS, and have an Enterprise License, you can perform a rolling upgrade to upgrade CDH without cluster down time. For an easier upgrade experience, consider switching from packages to parcels so that Cloudera Manager can automate more of the process. You can also switch from packages to parcels when upgrading CDH.

Upgrading CDH Using Packages

This option is the most time consuming and requires you to log in using ssh and execute a series of package commands on all hosts in your cluster. Cloudera recommends that you instead upgrade your cluster using parcels, which allows Cloudera Manager to distribute the upgraded software to all hosts in the cluster without having to log in to each host. If you installed the cluster using packages, you can upgrade using parcels and the cluster will use parcels for subsequent upgrades.

Overview of Upgrading Cloudera Navigator Components

Cloudera Navigator Metadata and Audit servers are automatically upgraded when you upgrade Cloudera Manager. You can also optionally upgrade other Cloudera Navigator components such as Cloudera Navigator Key Trustee Server, Cloudera Navigator Key HSM, and Cloudera Navigator Encrypt. You do not have to upgrade these components along with Cloudera Manager or CDH upgrades. For compatibility information, see: Product Compatibility Matrix for Cloudera Navigator Encryption (Cloudera Manager 6.x) or Product Compatibility Matrix for Cloudera Navigator Encryption (Cloudera Manager 5.x).

See Upgrading Cloudera Navigator Data Encryption.