Analytics/Cluster/Hive/Troubleshooting

Intro

Tips for troubleshooting issues with queries in Hadoop/Hive

Out Of Memory errors

On the hive client you can increase the heap size before launching hive:

export HADOOP_HEAPSIZE=1024

Search through logs

If your job is finished, you can find all of the job logs in HDFS at:

/mnt/hdfs/var/log/hadoop-yarn/apps/<user>/

Where USER is your username (echo $USER at a shell prompt) and application_id is the application_XXXXXX_XXXX number that yarn gives you when your app starts.

Logstash searching is in the works, see Analytics/Cluster/Logstash