本人原创，转载请注明出处：

下载Hadoop程序包，下载地址：http://hadoop.apache.org/releases.html#Download

如果是在CentOS服务器安装，则执行：

yum install hadoop-1.2.1-1.x86_64.rpm

如果是在Linux或者Mac OS X开发环境下，可以下载bin或者源码包，然后解压缩即可。

验证hadoop二进制执行文件（假设放在~/Developments/toolkits/hadoop-1.2.1文件夹中）：

cd ~/Developments/toolkits/hadoop-1.2.1

执行hadoop程序：

bin/hadoop

 

Usage: hadoop [--config confdir] COMMANDwhere COMMAND is one of:  namenode -format     format the DFS filesystem  secondarynamenode    run the DFS secondary namenode  namenode             run the DFS namenode  datanode             run a DFS datanode...

出现hadoop命令用法帮助，表示二进制文件可执行。

创建Hello Hadoop的Java项目：

按照《Hadoop权威指南（Hadoop: The Definitive Guide）》的例子，创建3个程序文件。

MaxTemperature.java

 

 

/** * Created with IntelliJ IDEA. * User: james * Date: 8/27/13 * Time: 11:33 AM * To change this template use File | Settings | File Templates. */import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Job;import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;public class MaxTemperature {    public static void main(String[] args) throws Exception {        if (args.length != 2) {            System.err.println("Usage: MaxTemperature  
");            System.exit(-1);        }        Job job = new Job();        job.setJarByClass(MaxTemperature.class);        job.setJobName("Max temperature");        FileInputFormat.addInputPath(job, new Path(args[0]));        FileOutputFormat.setOutputPath(job, new Path(args[1]));        job.setMapperClass(MaxTemperatureMapper.class);        job.setReducerClass(MaxTemperatureReducer.class);        job.setOutputKeyClass(Text.class);        job.setOutputValueClass(IntWritable.class);        System.exit(job.waitForCompletion(true) ? 0 : 1);    }}

 

 MaxTemperatureMapper.java

 

/** * Created with IntelliJ IDEA. * User: james * Date: 8/27/13 * Time: 11:28 AM * To change this template use File | Settings | File Templates. */import java.io.IOException;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Mapper;public class MaxTemperatureMapper        extends Mapper
    
      {    private static final int MISSING = 9999;    @Override    public void map(LongWritable key, Text value, Context context)            throws IOException, InterruptedException {        String line = value.toString();        String year = line.substring(15, 19);        int airTemperature;        if (line.charAt(87) == '+') { // parseInt doesn't like leading plus signs            airTemperature = Integer.parseInt(line.substring(88, 92));        } else {            airTemperature = Integer.parseInt(line.substring(87, 92));        }        String quality = line.substring(92, 93);        if (airTemperature != MISSING && quality.matches("[01459]")) {            context.write(new Text(year), new IntWritable(airTemperature));        }    }}
    

MaxTemperatureReducer.java

 

 

/** * Created with IntelliJ IDEA. * User: james * Date: 8/27/13 * Time: 11:32 AM * To change this template use File | Settings | File Templates. */import java.io.IOException;import org.apache.hadoop.io.IntWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Reducer;public class MaxTemperatureReducer        extends Reducer
    
      {    @Override    public void reduce(Text key, Iterable
     
       values,                       Context context)            throws IOException, InterruptedException {        int maxValue = Integer.MIN_VALUE;        for (IntWritable value : values) {            maxValue = Math.max(maxValue, value.get());        }        context.write(key, new IntWritable(maxValue));    }}
     
    

需要将hadoop-core-1.2.1.jar文件添加到项目的库中，这个jar文件在解压缩的文件夹中 

编译之，假设项目编译到文件夹~/Developments/hello-hadoop/out/production/hello-hadoop/中，将这个文件夹位置输出到HADOOP_CLASSPATH：

export HADOOP_CLASSPATH=~/Developments/hello-hadoop/out/production/hello-hadoop/

另外还要注意定义JAVA_HOME，以Mac OS X为例：

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/

下载天气数据（ http://hadoopbook.com/code.html ），上面有1901年和1902年的天气例子数据。

进入hadoop文件夹：

cd ~/Developments/toolkits/hadoop-1.2.1

执行例子程序（这个MaxTemperature是hadoop程序通过HADOOP_CLASSPATH查找到的）：

bin/hadoop MaxTemperature 1901 output

 

 

2013-10-15 17:56:40.412 java[5522:1703] Unable to load realm info from SCDynamicStore13/10/15 17:56:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable13/10/15 17:56:41 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.13/10/15 17:56:41 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).13/10/15 17:56:41 INFO input.FileInputFormat: Total input paths to process : 113/10/15 17:56:41 WARN snappy.LoadSnappy: Snappy native library not loaded13/10/15 17:56:42 INFO mapred.JobClient: Running job: job_local1783370164_000113/10/15 17:56:42 INFO mapred.LocalJobRunner: Waiting for map tasks13/10/15 17:56:42 INFO mapred.LocalJobRunner: Starting task: attempt_local1783370164_0001_m_000000_013/10/15 17:56:42 INFO mapred.Task:  Using ResourceCalculatorPlugin : null13/10/15 17:56:42 INFO mapred.MapTask: Processing split: file:/Users/james/Developments/hello-hadoop/out/production/hello-hadoop/1901:0+88819013/10/15 17:56:42 INFO mapred.MapTask: io.sort.mb = 10013/10/15 17:56:42 INFO mapred.MapTask: data buffer = 79691776/9961472013/10/15 17:56:42 INFO mapred.MapTask: record buffer = 262144/32768013/10/15 17:56:42 INFO mapred.MapTask: Starting flush of map output13/10/15 17:56:42 INFO mapred.MapTask: Finished spill 013/10/15 17:56:42 INFO mapred.Task: Task:attempt_local1783370164_0001_m_000000_0 is done. And is in the process of commiting13/10/15 17:56:42 INFO mapred.LocalJobRunner:13/10/15 17:56:42 INFO mapred.Task: Task 'attempt_local1783370164_0001_m_000000_0' done.13/10/15 17:56:42 INFO mapred.LocalJobRunner: Finishing task: attempt_local1783370164_0001_m_000000_013/10/15 17:56:42 INFO mapred.LocalJobRunner: Map task executor complete.13/10/15 17:56:42 INFO mapred.Task:  Using ResourceCalculatorPlugin : null13/10/15 17:56:42 INFO mapred.LocalJobRunner:13/10/15 17:56:42 INFO mapred.Merger: Merging 1 sorted segments13/10/15 17:56:42 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 72206 bytes13/10/15 17:56:42 INFO mapred.LocalJobRunner:13/10/15 17:56:42 INFO mapred.Task: Task:attempt_local1783370164_0001_r_000000_0 is done. And is in the process of commiting13/10/15 17:56:42 INFO mapred.LocalJobRunner:13/10/15 17:56:42 INFO mapred.Task: Task attempt_local1783370164_0001_r_000000_0 is allowed to commit now13/10/15 17:56:42 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1783370164_0001_r_000000_0' to output13/10/15 17:56:42 INFO mapred.LocalJobRunner: reduce > reduce13/10/15 17:56:42 INFO mapred.Task: Task 'attempt_local1783370164_0001_r_000000_0' done.13/10/15 17:56:43 INFO mapred.JobClient:  map 100% reduce 100%13/10/15 17:56:43 INFO mapred.JobClient: Job complete: job_local1783370164_000113/10/15 17:56:43 INFO mapred.JobClient: Counters: 1713/10/15 17:56:43 INFO mapred.JobClient:   File Output Format Counters13/10/15 17:56:43 INFO mapred.JobClient:     Bytes Written=2113/10/15 17:56:43 INFO mapred.JobClient:   File Input Format Counters13/10/15 17:56:43 INFO mapred.JobClient:     Bytes Read=88819013/10/15 17:56:43 INFO mapred.JobClient:   FileSystemCounters13/10/15 17:56:43 INFO mapred.JobClient:     FILE_BYTES_READ=184898613/10/15 17:56:43 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=24595113/10/15 17:56:43 INFO mapred.JobClient:   Map-Reduce Framework13/10/15 17:56:43 INFO mapred.JobClient:     Reduce input groups=113/10/15 17:56:43 INFO mapred.JobClient:     Map output materialized bytes=7221013/10/15 17:56:43 INFO mapred.JobClient:     Combine output records=013/10/15 17:56:43 INFO mapred.JobClient:     Map input records=656513/10/15 17:56:43 INFO mapred.JobClient:     Reduce shuffle bytes=013/10/15 17:56:43 INFO mapred.JobClient:     Reduce output records=113/10/15 17:56:43 INFO mapred.JobClient:     Spilled Records=1312813/10/15 17:56:43 INFO mapred.JobClient:     Map output bytes=5907613/10/15 17:56:43 INFO mapred.JobClient:     Total committed heap usage (bytes)=33135001613/10/15 17:56:43 INFO mapred.JobClient:     SPLIT_RAW_BYTES=14113/10/15 17:56:43 INFO mapred.JobClient:     Map output records=656413/10/15 17:56:43 INFO mapred.JobClient:     Combine input records=013/10/15 17:56:43 INFO mapred.JobClient:     Reduce input records=6564

查看输出结果

ls output/

 

 

_SUCCESS     part-r-00000

 

vi output/part-r-00000

 

1901    317