Skip to main content
Version: 2.7.5

Operations

Metrics Collector

Pid file locations

DaemonDefault UserPid File Path
Metrics Collector APIams/var/run/ambari-metrics-collector/ambari-metrics-collector.pid
Metrics Collector Hbaseams/var/run/ambari-metrics-collector/hbase-ams-master.pid

Log file locations

DaemonLog File Path
Metrics Collector API/var/log/ambari-metrics-collector/ambari-metrics-collector.log
/var/log/ambari-metrics-collector/ambari-metrics-collector.out
Metrics Collector HBase/var/log/ambari-metrics-collector/hbase-ams-master-<hostname>.log
/var/log/ambari-metrics-collector/hbase-ams-master-<hostname>.out

Manually restart Metrics Collector

Stop command

su - ams -c '/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ stop'

Start command

su - ams -c '/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ start'

Metrics Monitor

Pid File location

/var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid

Log File location

/var/log/ambari-metrics-monitor/ambari-metrics-monitor.out

Manually restart Metrics Monitor Stop command

su - ams -c '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf stop'

Start command

su - ams -c '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf start'

Build Instructions

The ambari-metrics-assembly package builds the assemblies (rpm/deb/msi) for various platforms.

Following binaries can be found in the ambari-metrics-assembly/target folder after build is successful.

ambari-metrics-collecor-<ambari-version>.<arch>
ambari-metrics-monitor-<ambari-version>.<arch>
ambari-hadoop-sink-<ambari-version>.<arch>

Note: Ambari Metrics needs to be built before Ambari Server

RPM packages

cd ambari-metrics
mvn clean package -Dbuild-rpm

Debian packages

Same instruction as above, change the maven target to build-deb

Windows msi

TBU

Command line parameters

ParameterDefault ValueComment
hbase.tarhttp://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.0.0/tars/hbase-1.1.1.2.3.0.0-2557.tar.gzHBase tarball. This is default version for Ambari 2.1.2
hbase.folderhbase-1.1.1.2.3.0.0-2557-
hadoop.tarhttp://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.0.0/tars/hadoop-2.7.1.2.3.0.0-2557.tar.gzHadoop tarball, used for native libs. This is default version for Ambari 2.1.2
hadoop.folderhadoop-2.7.1.2.3.0.0-2557-

Note

After the change introducted by AMBARI-18915 - Update AMS pom to use Apache hbase,hadoop,phoenix tarballs REOPENED AMS uses hadoop tars downloaded from Apache by default. Since that version of libhadoop is not built with libsnappy, the following config change in ams-site is needed to make AMS start up correctly.

timeline.metrics.hbase.compression.scheme = None

Disk space utilization guidance

Num of NodesMETRIC_RECORD (MB)METRIC_RECORD_MINUTE (MB)METRIC_RECORD_HOURLY (MB)METRIC_RECORD_DAILY (MB)METRIC_AGGREGATE (MB)METRIC_AGGREGATE_MINUTE (MB)METRIC_AGGREGATE_HOURLY (MB)METRIC_AGGREGATE_DAILY (MB)TOTAL (GB)
505120270024510150030528110
10010240540049020150030528118
3003072016200147060150030528149
50051200270002450100150030528181
800819204320039201601500305281128

NOTE:

The above guidance has been derived from looking at AMS disk utilization in actual clusters. The ACTUAL numbers have been obtained by observing an actual cluster with the basic services (HDFS, YARN, HBase) installed along with Storm, Kafka and Flume. Kafka and Flume generate metrics only while a job is running. If those services are being used heavily, additional disk space is recommended. We ran sample jobs with STORM and KAFKA while deriving these numbers to make sure there is some contribution.

Actual disk utilization data

Num of NodesMETRIC_RECORD (MB)METRIC_RECORD_MINUTE (MB)METRIC_RECORD_HOURLY (MB)METRIC_RECORD_DAILY (MB)METRIC_AGGREGATE (MB)METRIC_AGGREGATE_MINUTE (MB)METRIC_AGGREGATE_HOURLY (MB)METRIC_AGGREGATE_DAILY (MB)TOTAL (GB)
21201751715451361611
3294513.41104261.810.5
1010245404921433.63052813.3

Phoenix Schema

Phoenix Tables

Table NameDescriptionPurge Interval(default)
METRIC_RECORDData per metric per host at 10 seconds precision with 1 minute aggregates.1 day
METRIC_RECORD_MINUTEData per metric per host at 5 minute precision1 week
METRIC_RECORD_HOURLYData per metric per host at 1 hour precision30 days
METRIC_RECORD_DAILYData per metric per host at 1 day precision1 year
METRIC_AGGREGATECluster wide aggregates per metric at 30 seconds precision1 week
METRIC_AGGREGATE_MINUTECluster wide aggregates per metric at 5 minute precision30 days
METRIC_AGGREGATE_HOURLYCluster wide aggregates per metric at 1 hour precision1 year
METRIC_AGGREGATE_DAILYCluster wide aggregates per metric at 1 day precision2 years

Connecting to Phoenix

  • Unpack Phoenix (4.2.0+) tarball onto the Metric Collector host
  • Change director to phonenix-4.*/bin
  • Edit sqlline.py and search for "java", replace java with full path to the java executable, example: "/usr/jdk64/jdk1.8.0_40/bin/java"
  • Connect command: Ambari versions 2.2.0 and below : ./sqlline.py localhost:61181:/hbase Ambari versions > 2.2.0 :
./sqlline.py localhost:61181:/ams-hbase-unsecure (embedded mode) and <cluster-zookeeper-quorum-host>:<cluster_zookeeper_port>:/ams-hbase-unsecure (distributed mode)