Package org.apache.nutch.parse

## Class ParserChecker

- java.lang.Object
- - org.apache.hadoop.conf.Configured
  - - org.apache.nutch.util.AbstractChecker
    - - org.apache.nutch.parse.ParserChecker

- All Implemented Interfaces:
    `Configurable`, `Tool`

      public class ParserChecker
      extends AbstractChecker

  Parser checker, useful for testing parser. It also accurately reports possible fetching and parsing failures and presents protocol status signals to aid debugging. The tool enables us to retrieve the following data from any url:
  1. `contentType`: The URL `Content` type.
  2. `signature`: Digest is used to identify pages (like unique ID) and is used to remove duplicates during the dedup procedure. It is calculated using `MD5Signature` or `TextProfileSignature`.
  3. `Version`: From `ParseData`.
  4. `Status`: From `ParseData`.
  5. `Title`: of the URL
  6. `Outlinks`: associated with the URL
  7. `Content Metadata`: such as X-AspNet-Version, Date, Content-length, servedBy, Content-Type, Cache-Control, etc.
  8. `Parse Metadata`: such as CharEncodingForConversion, OriginalCharEncoding, language, etc.
  9. `ParseText`: The page parse text which varies in length depdnecing on `content.length` configuration.

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `protected boolean` | `checkRobotsTxt` |
    `protected boolean` | `dumpText` |
    `protected boolean` | `followRedirects` |
    `protected String` | `forceAsContentType` |
    `protected HashMap<String,​String>` | `metadata` |
    `protected URLNormalizers` | `normalizers` |

    - ### Fields inherited from class org.apache.nutch.util.AbstractChecker

      `keepClientCnxOpen, stdin, tcpPort, usage`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `ParserChecker()` |

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `static void` | `main​(String[] args)` |
    `protected int` | `process​(String url, StringBuilder output)` |
    `int` | `run​(String[] args)` |

    - ### Methods inherited from class org.apache.nutch.util.AbstractChecker

      `getProtocolOutput, parseArgs, processSingle, processStdin, processTCP, run`

    - ### Methods inherited from class org.apache.hadoop.conf.Configured

      `getConf, setConf`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface org.apache.hadoop.conf.Configurable

      `getConf, setConf`

- - ### Field Detail

    - #### normalizers

          protected URLNormalizers normalizers

    - #### dumpText

          protected boolean dumpText

    - #### followRedirects

          protected boolean followRedirects

    - #### checkRobotsTxt

          protected boolean checkRobotsTxt

    - #### metadata

          protected HashMap<String,​String> metadata

    - #### forceAsContentType

          protected String forceAsContentType

  - ### Constructor Detail

    - #### ParserChecker

          public ParserChecker()

  - ### Method Detail

    - #### run

          public int run​(String[] args)
                  throws Exception

      Throws:
        `Exception`

    - #### process

          protected int process​(String url,
                                StringBuilder output)
                         throws Exception

      Specified by:
        `process` in class `AbstractChecker`
      Throws:
        `Exception`

    - #### main

          public static void main​(String[] args)
                           throws Exception

      Throws:
        `Exception`