Apache Pig is a platform for creating MapReduce programs used with Hadoop. It consists of a high-level language (Pig Latin) for expressing data analysis programs, coupled with infrastructure for evaluating these programs. Learn more at pig.apache.org.
This charm supports running Pig in two execution modes:
- Local Mode: Pig runs using your local host and file system. Specify local
mode using the -x flag:
pig -x local - Mapreduce Mode: Pig runs using a Hadoop cluster and HDFS. This is the default
mode; you can, optionally, specify it using the -x flag:
pigorpig -x mapreduce
This charm is intended to be deployed via the apache-analytics-pig bundle:
juju quickstart apache-analytics-pig
This will deploy the Apache Hadoop platform with Apache Pig available to execute Pig Latin jobs on your data. Once deployment is complete, you can run Pig in a variety of modes:
Run Pig in local mode on the Pig unit with the following:
juju ssh pig/0
pig -x local
MapReduce mode is the default for Pig. To run in this mode, ssh to the Pig unit and run pig as follows:
juju ssh pig/0
pig
The services provide extended status reporting to indicate when they are ready:
juju status --format=tabular
This is particularly useful when combined with watch to track the on-going
progress of the deployment:
watch -n 0.5 juju status --format=tabular
The message for each unit will provide information about that unit's state.
Once they all indicate that they are ready, you can perform a "smoke test"
to verify that Spark is working as expected using the built-in smoke-test
action:
juju action do pig/0 smoke-test
After a few seconds or so, you can check the results of the smoke test:
juju action status
You will see status: completed if the smoke test was successful, or
status: failed if it was not. You can get more information on why it failed
via:
juju action fetch <action-id>