org.apache.nutch.hostdb

## Class UpdateHostDbReducer

- java.lang.Object
- - org.apache.hadoop.mapreduce.Reducer<Text,NutchWritable,Text,HostDatum>
  - - org.apache.nutch.hostdb.UpdateHostDbReducer

-     public class UpdateHostDbReducer
      extends Reducer<Text,NutchWritable,Text,HostDatum>

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

      `Reducer.Context`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected static boolean` | `checkFailed`
    `protected static boolean` | `checkKnown`
    `protected static boolean` | `checkNew`
    `protected ThreadPoolExecutor` | `executor`
    `protected static boolean` | `force`
    `protected static long` | `now`
    `protected static String[]` | `numericFields`
    `protected static Text[]` | `numericFieldWritables`
    `protected Integer` | `numResolverThreads`
    `protected static int[]` | `percentiles`
    `protected static Integer` | `purgeFailedHostsThreshold`
    `protected BlockingQueue<Runnable>` | `queue`
    `protected static Integer` | `recheckInterval`
    `protected ResolverThread` | `resolverThread`
    `protected static String[]` | `stringFields`
    `protected static Text[]` | `stringFieldWritables`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `UpdateHostDbReducer()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `cleanup(Reducer.Context context)` Shut down all running threads and wait for completion.
    `protected boolean` | `isEligibleForCheck(HostDatum datum)` Determines whether a record is eligible for recheck.
    `void` | `reduce(Text key, Iterable<NutchWritable> values, Reducer.Context context)`
    `void` | `setup(Reducer.Context context)` Configures the thread pool and prestarts all resolver threads.
    `protected boolean` | `shouldCheck(HostDatum datum)` Determines whether a record should be checked.

    - ### Methods inherited from class org.apache.hadoop.mapreduce.Reducer

      `run`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### resolverThread

          protected ResolverThread resolverThread

    - #### numResolverThreads

          protected Integer numResolverThreads

    - #### purgeFailedHostsThreshold

          protected static Integer purgeFailedHostsThreshold

    - #### recheckInterval

          protected static Integer recheckInterval

    - #### checkFailed

          protected static boolean checkFailed

    - #### checkNew

          protected static boolean checkNew

    - #### checkKnown

          protected static boolean checkKnown

    - #### force

          protected static boolean force

    - #### now

          protected static long now

    - #### numericFields

          protected static String[] numericFields

    - #### stringFields

          protected static String[] stringFields

    - #### percentiles

          protected static int[] percentiles

    - #### numericFieldWritables

          protected static Text[] numericFieldWritables

    - #### stringFieldWritables

          protected static Text[] stringFieldWritables

    - #### queue

          protected BlockingQueue<Runnable> queue

    - #### executor

          protected ThreadPoolExecutor executor

  - ### Constructor Detail

    - #### UpdateHostDbReducer

          public UpdateHostDbReducer()

  - ### Method Detail

    - #### setup

          public void setup(Reducer.Context context)

      Configures the thread pool and prestarts all resolver threads.
      Overrides:
        `setup` in class `Reducer<Text,NutchWritable,Text,HostDatum>`

    - #### reduce

          public void reduce(Text key,
                             Iterable<NutchWritable> values,
                             Reducer.Context context)
                      throws IOException,
                             InterruptedException

      Overrides:
        `reduce` in class `Reducer<Text,NutchWritable,Text,HostDatum>`
      Throws:
        `IOException`
        `InterruptedException`

    - #### shouldCheck

          protected boolean shouldCheck(HostDatum datum)

      Determines whether a record should be checked.
      Parameters:
        `datum` -
      Returns:
        boolean

    - #### isEligibleForCheck

          protected boolean isEligibleForCheck(HostDatum datum)

      Determines whether a record is eligible for recheck.
      Parameters:
        `datum` -
      Returns:
        boolean

    - #### cleanup

          public void cleanup(Reducer.Context context)

      Shut down all running threads and wait for completion.
      Overrides:
        `cleanup` in class `Reducer<Text,NutchWritable,Text,HostDatum>`

Copyright © 2020 The Apache Software Foundation