Package org.apache.nutch.protocol.file

## Class File

- java.lang.Object
- - org.apache.nutch.protocol.file.File

- All Implemented Interfaces:
    `Configurable`, `Pluggable`, `Protocol`

      public class File
      extends Object
      implements Protocol

  This class is a protocol plugin used for file: scheme. It creates `FileResponse` object and gets the content of the url from it. Configurable parameters are `file.content.limit` and `file.crawl.parent` in nutch-default.xml defined under "file properties" section.
  Author:
    John Xing

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `protected static org.slf4j.Logger` | `LOG` |

    - ### Fields inherited from interface org.apache.nutch.protocol.Protocol

      `X_POINT_ID`

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `File()` |

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `Configuration` | `getConf()` | Get the `Configuration` object
    `ProtocolOutput` | `getProtocolOutput​(Text url, CrawlDatum datum)` | Creates a `FileResponse` object corresponding to the url and return a `ProtocolOutput` object as per the content received
    `crawlercommons.robots.BaseRobotRules` | `getRobotRules​(Text url, CrawlDatum datum, List<Content> robotsTxtContent)` | No robots parsing is done for file protocol.
    `static void` | `main​(String[] args)` | Quick way for running this class.
    `void` | `setConf​(Configuration conf)` | Set the `Configuration` object
    `void` | `setMaxContentLength​(int maxContentLength)` | Set the length after at which content is truncated.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOG

          protected static final org.slf4j.Logger LOG

  - ### Constructor Detail

    - #### File

          public File()

  - ### Method Detail

    - #### setConf

          public void setConf​(Configuration conf)

      Set the `Configuration` object
      Specified by:
        `setConf` in interface `Configurable`

    - #### getConf

          public Configuration getConf()

      Get the `Configuration` object
      Specified by:
        `getConf` in interface `Configurable`

    - #### setMaxContentLength

          public void setMaxContentLength​(int maxContentLength)

      Set the length after at which content is truncated.
      Parameters:
        `maxContentLength` - max content in bytes

    - #### getProtocolOutput

          public ProtocolOutput getProtocolOutput​(Text url,
                                                  CrawlDatum datum)

      Creates a `FileResponse` object corresponding to the url and return a `ProtocolOutput` object as per the content received
      Specified by:
        `getProtocolOutput` in interface `Protocol`
      Parameters:
        `url` - Text containing the url
        `datum` - The CrawlDatum object corresponding to the url
      Returns:
        `ProtocolOutput` object for the content of the file indicated by url

    - #### main

          public static void main​(String[] args)
                           throws Exception

      Quick way for running this class. Useful for debugging.
      Parameters:
        `args` - run with no args to print help
      Throws:
        `Exception` - if there is a fatal error running this class with the given input

    - #### getRobotRules

          public crawlercommons.robots.BaseRobotRules getRobotRules​(Text url,
                                                                    CrawlDatum datum,
                                                                    List<Content> robotsTxtContent)

      No robots parsing is done for file protocol. So this returns a set of empty rules which will allow every url.
      Specified by:
        `getRobotRules` in interface `Protocol`
      Parameters:
        `url` - URL to check
        `datum` - page datum
        `robotsTxtContent` - container to store responses when fetching the robots.txt file for debugging or archival purposes. Instead of a robots.txt file, it may include redirects or an error page (404, etc.). Response `Content` is appended to the passed list. If null is passed nothing is stored.
      Returns:
        robot rules (specific for this URL or default), never null