上传到hadoop目录中

cobertura-maven-plugin是只什么鬼?

cobertura-maven-plugin举凡一个校验单元测试用例覆盖率的工具,可以充分成一个测试覆盖率报告,可以让单元测试用例编写提供参考.

6、yarn env中环境变量配置,尤其是java home路径配置 


以健康运转无至1分钟结果就出来了,若出现这种状况相似,说明遇到问题了

表格式

默认情况下,插件会于target/site下生成html格式的报表.html格式的被人拘禁比较直观,但是若报表要导入到任何系统或者xml解析起来简单.让咱省怎么安排吧.

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <version>2.7</version>

    <configuration>
         <!--这个标签一般不在这配置,一般跟着命令后面作为参数使用-->
        <!--<format>xml</format>-->
        <formats>
            <format>xml</format>
            <format>html</format>
        </formats>
    </configuration>
</plugin>

骨子里还是简单的.就是这起少单同级标签<format><formats>被人口犯晕.其实也无为难,<formats>的安排允许同时输出多单格式的报表.
<format>的用意还多之常作为命令参数传入,比如mvn clean cobertura:cobertura -Dcobertura.report.format=xml,从这我们为便明白了<format>要比<formats>先级高.

7、yarn -site.xml中布置有误,比如配置内存还是虚拟机分配的内存过小

一个妙趣横生的试验

以还好的晓cobertura-maven-plugin令,我们今天做个有趣的试,观察target目录及操纵台下有了什么.

  • 1.我们执行命令mvn clean先清理掉target遭到的文件.

  • 2.执行命令mvn compile编译源文件.

    图片 1

  • 3.执行mvn cobertura:dump-datafile命令,后台日志显示执行失败.

    [ERROR] Failed to execute goal org.codehaus.mojo:cobertura-maven-plugin:2.7:dump-datafile (default-cli) on project learn-maven: Unable to dump nonexistent dataFile [/Users/qyfmac/git/learn-maven/target/cobertura/cobertura.ser] -> [Help 1]
    
  • 4.执行命令mvn cobertura:instrument符为编译的文件.我们见面意识差不多矣几单文本于target目录里.

    图片 2

  • 5.执行命令mvn cobertura:dump-datafile打印出覆盖率信息.

    [INFO] --- cobertura-maven-plugin:2.7:dump-datafile (default-cli) @ learn-maven ---
    [INFO] Cobertura: Loaded information on 1 classes.
    [INFO] <?xml version="1.0"?>
    [INFO] <coverage line-rate="0%" branch-rate="100%" lines-covered="0" lines-valid="3" branches-covered="0" branches-valid="0" version="2.1.1" timestamp="1449841922764">
    [INFO] <package name="com.qyf404.learn.maven" line-rate="0%" branch-rate="100%" lines-covered="0" lines-valid="3" branches-covered="0" branches-valid="0" />
    [INFO] </coverage>
    
  • 6.执行命令mvn cobertura:cobertura变动覆盖率报告.

    图片 3

  • 7.还实施mvn cobertura:dump-datafile打印出覆盖率信息.

    [INFO] --- cobertura-maven-plugin:2.7:dump-datafile (default-cli) @ learn-maven ---
    [INFO] Cobertura: Loaded information on 1 classes.
    [INFO] <?xml version="1.0"?>
    [INFO] <coverage line-rate="100%" branch-rate="100%" lines-covered="3" lines-valid="3" branches-covered="0" branches-valid="0" version="2.1.1" timestamp="1449842157556">
    [INFO] <package name="com.qyf404.learn.maven" line-rate="100%" branch-rate="100%" lines-covered="3" lines-valid="3" branches-covered="0" branches-valid="0" />
    [INFO] </coverage>
    

试行做扫尾了.平联袂7步,执行了mvn cobertura:cobertura命令3次.

  1. 先是坏施行报错,是坐标记文件并未成形,没有找到要分析的文件.
  2. 仲破施行,打印出了日记,但是测试用例还不曾实施,所以测试覆盖率还是始于状态,多数也0%.
  3. 其三不良实施,打印出了日记,显示了测试覆盖情况.

前面Hadoop安装分点儿管辖份,一凡虚机的装和部署, 二凡是Hadoop集群安装与布局

goals介绍

总体来说cobertura-maven-plugin的采用还是生简单的.下面我们来看望cobertura-maven-plugin里除了cobertura:cobertura命令还有哪些命令.执行mvn cobertura:help咱见面赢得该插件的一个赞助文档.

qyfmac$ mvn cobertura:help
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building learn-maven 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- cobertura-maven-plugin:2.7:help (default-cli) @ learn-maven ---
[INFO] Mojo's Maven plugin for Cobertura 2.7
  This is the Mojo's Maven plugin for Cobertura. Cobertura is a free Java tool
  that calculates the percentage of code accessed by tests. It can be used to
  identify which parts of your Java program are lacking test coverage.

This plugin has 8 goals:

cobertura:check
  Check the coverage percentages for unit tests from the last instrumentation,
  and optionally fail the build if the targets are not met. To fail the build
  you need to set configuration/check/haltOnFailure=true in the plugin's
  configuration.

cobertura:check-integration-test
  Check the coverage percentages for unit tests and integration tests from the
  last instrumentation, and optionally fail the build if the targets are not
  met. To fail the build you need to set configuration/check/haltOnFailure=true
  in the plugin's configuration.

cobertura:clean
  Clean up the files that Cobertura Maven Plugin has created during
  instrumentation.

cobertura:cobertura
  Instrument the compiled classes, run the unit tests and generate a Cobertura
  report.

cobertura:cobertura-integration-test
  Instrument the compiled classes, run the unit tests and integration tests and
  generate a Cobertura report.

cobertura:dump-datafile
  Output the contents of Cobertura's data file to the command line.

cobertura:help
  Display help information on cobertura-maven-plugin.
  Call mvn cobertura:help -Ddetail=true -Dgoal=<goal-name> to display parameter
  details.

cobertura:instrument
  Instrument the compiled classes.


[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.822 s
[INFO] Finished at: 2015-12-03T16:23:59+08:00
[INFO] Final Memory: 8M/156M
[INFO] ------------------------------------------------------------------------

俺们逐一介绍一下.

  • cobertura:check

    依据新型的源码标记(生成的class文件)校验测试用例的覆盖率,如果没有高达要求,则行失败.

  • cobertura:check-integration-test

    斯命令和cobertura:check功用是一样的,区别是两岸绑定的maven生命周期不一样.cobertura:check绑定了test,
    cobertura:check-integration-test绑定了verify.再说之知晓些,maven生命周期中发生一个凡test飞得单元测试,还有一个是integration-test跑的三合一测试.而verify前就是integration-test.即cobertura:check-integration-testcobertura:check包含的测试用例更多.

  • cobertura:clean

    其一好理解,就是理清掉目录/target/cobertura/中得文件.目前意识里头就是一个文书cobertura.ser.

  • cobertura:cobertura

    此插件的根本命令.标记为编译的文本,运行单元测试,生成测试报告.

  • cobertura:cobertura-integration-test

    cobertura:cobertura举行了一如既往的业务,区别是带有了集成测试用例.

  • cobertura:dump-datafile

    每当命令执行输出覆盖率数据.数据因是变化的class文件.这个令自己莫打明白他的义何在.以末端一个有趣的实验我们见面为此这令来又好的晓cobertura-maven-plugin.

  • cobertura:help

    不解释.

  • cobertura:instrument

    记为编译的class文件.执行之命令会在目录/target/generated-classes/cobertura下生成一法class文件.

老三、运行示例

统计范围

突发性我们而举行覆盖率统计时,不需全方位代码都开的,为了还可靠之统计我们怀念掌握之代码的测试用例覆盖率,我们就是需加以有安排,来控制统计范围.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>cobertura-maven-plugin</artifactId>
  <version>2.7</version>
  <configuration>
    <instrumentation>
      <!-- 这种方式我没试验成功.如果有谁配置生效了,请留言告知 http://qyf404.cnblogs.com
      <ignores>
        <ignore>com.example.boringcode.*</ignore>
      </ignores>
      -->
      <!--忽略列表.根据class文件路径进行匹配-->
      <excludes>
        <exclude>com/qyf404/learn/**/App.class</exclude>
      </excludes>
    </instrumentation>
  </configuration>
</plugin>

具体我们看一下之类步骤:

校验规则

诚如我们实施cobertura:coberturacobertura:cobertura-integration-test不畏够了.但是当开持续集成时,我们或许想会自动化去查测试用例的盖情况,比如当覆盖率低于70%常包装失败.这个时段便因此到cobertura:checkcobertura:check-integration-test命令了.

cobertura-maven-plugin同意用户配置有平整,来校验测试覆盖率.

<project>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>cobertura-maven-plugin</artifactId>
        <version>2.7</version>
        <configuration>
          <check>
            <!--每个类的分支覆盖率-->
            <branchRate>85</branchRate>
            <!--每个类的代码行覆盖率-->
            <lineRate>85</lineRate>
            <!--校验失败则中止.即为true时如果校验失败则打包执行失败,反之则只是打出告警日志-->
            <haltOnFailure>true</haltOnFailure>
            <!--整体的分支覆盖率-->
            <totalBranchRate>85</totalBranchRate>
            <!--整体的代码行覆盖率-->
            <totalLineRate>85</totalLineRate>
            <!--每个包的分支行覆盖率-->
            <packageBranchRate>85</packageBranchRate>
            <!--每个包的代码行覆盖率-->
            <packageLineRate>85</packageLineRate>
            <regexes>
              <!--通过表达式设定覆盖率伐值-->
              <regex>
                <pattern>com.qyf404.learn.*</pattern>
                <branchRate>90</branchRate>
                <lineRate>80</lineRate>
              </regex>
            </regexes>
          </check>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

默认这个校验是待独自敲命令的,当然好因实际需要,在execution里配置goalphase来好命令的绑定.比如在卷入前召开校验,可以这样配置execution.

<plugins>
  ...
  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>cobertura-maven-plugin</artifactId>
    <version>2.7</version>
    <configuration>
      <check>
      ...
      </check>
    </configuration>
    <executions>
      <execution>
        <goals>
          <goal>check</goal>
        </goals>
        <phase>prepare-package</phase>
      </execution>
    </executions>
  </plugin>
  ...
</plugins>

[hadoop@kencentos01 bin]$ hdfs dfs -put /usr/mydata/test1.txt
/usr/hadoop/wordcount/input

结语

好了,cobertura-maven-plugin就算介绍这么多矣,更多cobertura-maven-plugin的运用办法大家好参照官文档

正文的演示代码这里可以下至
https://github.com/qyf404/learn-maven/tree/cobertura-maven-plugin.

好查下相应的log, (hadoop安装目录下logs),比如 yarn 相关日志

helloword

cobertura-maven-plugin的使用也充分粗略,首先你如果来源码,然后一旦出对这个源码编写的测试代码,最后以pom.xml丁配置达到cobertura-maven-plugin尽一行命令就可了.

我们先来准备一个源码和测试用例:

图片 4

要被测试的代码

package com.qyf404.learn.maven;

public class App {
    public int add(int a, int b) {
        return a + b;
    }
    public int subtract(int a, int b) {
        return a - b;
    }
}

测试用例代码

package com.qyf404.learn.maven;

import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.junit.experimental.categories.Category;

public class AppTest {
    private App app;
    @Before
    public void setUp() {
        app = new App();
    }
    @Test
    public void testAdd() throws InterruptedException {
        int a = 1;
        int b = 2;
        int result = app.add(a, b);
        Assert.assertEquals(a + b, result);
    }
    @Test()
    public void testSubtract() throws InterruptedException {
        int a = 1;
        int b = 2;
        int result = app.subtract(a, b);
        Assert.assertEquals(a - b, result);
    }
    @After
    public void tearDown() throws Exception {
    }
}

pom.xml配置如下:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.qyf404</groupId>
    <artifactId>learn-maven</artifactId>
    <version>1.0-SNAPSHOT</version>

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <version>2.7</version>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

数量都备好了,我们实行以下maven命令mvn cobertura:cobertura,执行完后会当target目录里找到site目,用浏览器打开其中的index.html,这就算是测试用例执行完后cobertura-maven-plugin得发之覆盖率报告.

图片 5

晓内容如下:

图片 6

5、应权限导致默认目录不可用,可于指定 input目录,便于发现问题

跨了尽

cobertura-maven-plugin的跳过执行之布局主要由于三种:

  1. <configuration>中配置:

    <configuration>
        <skip>true</skip>
    </configuration>
    
  2. <properties>中配置:

    <properties>
        <cobertura.skip>true</cobertura.skip>
    </properties>
    
  3. 令中配备mvn clean cobertura:cobertura -Dcobertura.skip=true.

 注: output文件夹为全新的,之前有去

分析后台日志

咱们又省这后台日志打印的始末

qyfmac$ mvn clean cobertura:cobertura
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building learn-maven 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ learn-maven ---
[INFO] Deleting /Users/qyfmac/git/learn-maven/target
[INFO] 
[INFO] >>> cobertura-maven-plugin:2.7:cobertura (default-cli) > [cobertura]test @ learn-maven >>>
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ learn-maven ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/qyfmac/git/learn-maven/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ learn-maven ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to /Users/qyfmac/git/learn-maven/target/classes
[INFO] 
[INFO] --- cobertura-maven-plugin:2.7:instrument (default-cli) @ learn-maven ---
[INFO] Cobertura 2.1.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
[INFO] Cobertura: Saved information on 1 classes.
[INFO] Cobertura: Saved information on 1 classes.

[INFO] Instrumentation was successful.
[INFO] NOT adding cobertura ser file to attached artifacts list.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ learn-maven ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/qyfmac/git/learn-maven/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ learn-maven ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to /Users/qyfmac/git/learn-maven/target/test-classes
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ learn-maven ---
[INFO] Surefire report directory: /Users/qyfmac/git/learn-maven/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.qyf404.learn.maven.AppTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.335 sec
[INFO] Cobertura: Loaded information on 1 classes.
[INFO] Cobertura: Saved information on 1 classes.

Results :

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] <<< cobertura-maven-plugin:2.7:cobertura (default-cli) < [cobertura]test @ learn-maven <<<
[INFO] 
[INFO] --- cobertura-maven-plugin:2.7:cobertura (default-cli) @ learn-maven ---
[INFO] Cobertura 2.1.1 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
[INFO] Cobertura: Loaded information on 1 classes.
Report time: 159ms

[INFO] Cobertura Report generation was successful.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.624 s
[INFO] Finished at: 2015-12-08T16:42:47+08:00
[INFO] Final Memory: 22M/227M
[INFO] ------------------------------------------------------------------------
  • 旋即行日志[INFO] >>> cobertura-maven-plugin:2.7:cobertura (default-cli) > [cobertura]test @ learn-maven >>>晓我们cobertura-maven-plugin绑定到了maven生命周期test上.

  • 即行日志[INFO] --- cobertura-maven-plugin:2.7:instrument (default-cli) @ learn-maven ---报我们在完成项目源码编译后,执行了cobertura:instrument,对项目源码做了标记.

  • 马上简单尽日志

    [INFO] Cobertura: Loaded information on 1 classes.
    [INFO] Cobertura: Saved information on 1 classes.
    

    每当实践测试用例时起有之,很明白cobertura-maven-plugin当实行测试用例时开了若干事情(这才是独猜想,具体涉及了何是要拘留源码的).

  • 立行日志[INFO] Cobertura Report generation was successful.报我们转移覆盖率报告成功.而之前实施了测试用例,而且转移报告用了前面生成的class文件.

由此分析日志,我们着力能猜测及:

  1. cobertura-maven-plugin绑定到了maven生命周期test上.
  2. 源码编译完成后,cobertura-maven-plugin变更了好之如出一辙模仿class文件来号源码,存放于/target/generated-classes/cobertura/目录里.
  3. 测试用例执行时,cobertura-maven-plugin改了上下一心变的那套标记源码的class文件,并依据当下套class文件充分成了最后的覆盖率报告.

2、检查防火墙,开放有hadoop用到之端口,若为简单,可径直关闭防火墙

参考

  • 合法文档:http://www.mojohaus.org/cobertura-maven-plugin/
  • pom文件详解:http://www.blogjava.net/jianyue/articles/227932.html

关于作者

其次、上传自定义之公文及input目录

图片 7

季、查看结果

[hadoop@kencentos01 bin]$ hdfs dfs -mkdir -p
/usr/hadoop/wordcount/input

图片 8

五、可能遇到的问题

图片 9

查看output 目录看结果,查看对应生成文件part-r-0000情

[hadoop@kencentos01 hadoop-2.7.3]$

图片 10

这里我们运行下hadoop
自带示例wordcount,当然在举行这个示例前提是Hadoop已布置好,且曾开行运行

此是均等浅创多级目录

如若创建文件 /usr/mydata/test1.txt,里面自行输入有词组

4、hadoop用户权限不敷

广或的题材由产生:

 

一、创建input目录

Hdfs dfs –put /usr/mydata/test*.txt  /usr/hadoop/worcount/input

图片 11

 bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount
/usr/hadoop/worcount/input  /usr/hadoop/worcount/output

单词计数主要功用是:统计一多元文件文件被每个单词出现的次数,比如 hello
world,hello,统计结果就是hello 2,world 1

每当即时中间运行示例时,有或撞mapreduce job 一直于运作,不停歇的情

1、hostname配置出无意,即检查hosts文件中布置,注释掉无关的host行

Wordcount(单词计数)是无限简易不过直观看到MapReduce思想,可以这么当是MapReduce版”Hello
World”

图片 12

3、检查节点是否正规运作,如slave机子上datanode

Hadoop 配置和运作 wordcount示例,也是个MapReduce示例

若是创建多只公文可以用通配符来代表