Overview

Module jobs

• Description
• Function Index
• Function Details

Authors: : Ulf Wiger (ulf@wiger.net).

Description

Function Index

Function Details

add_counter/2

Adds a named counter to the load regulator on the current node. Fails if there already is a counter the name Name.

add_group_rate/2

Adds a group rate regulator to the load regulator on the current node. Fails if there is already a group rate regulator of the same name.

add_queue/2

Installs a new queue in the load regulator on the current node.

Valid options are:

* {regulators, Rs}, where Rs is a list of rate- or counter-based regulators. Valid regulators listed below. Default: [].

* {type, Type} - type of queue. Valid types listed below. Default: fifo.

* {action, Action} - automatic action to perform for each request. Valid actions described below. Default: undefined.

* {check_interval, I} - If specified (in ms), this overrides the interval derived from any existing rate regulator. Note that regardless of how often the queue is checked, enough jobs will be dispatched at each interval to maintain the highest allowed rate possible, but the check interval may thus affect how many jobs are dispatched at the same time. Normally, this should not have to be specified.

* {max_time, T}, specifies how long (in ms) a job is allowed to wait in the queue before it is automatically rejected. If undefined, no limit is imposed.

* {max_size, S}, indicates how many items can be queued before requests are automatically rejected. Strictly speaking, size is whatever the queue behavior reports as the size; in the default queue behavior, it is the number of elements in the queue. If undefined, no limit is imposed.

* {link, undefined | pid()}, links the queue to a process. The queue will automatically be removed if the process dies. Default is undefined, which means that the queue will not be linked. An exception will be raised if the option value is anything other than undefined or a valid pid. Note that if the referenced pid is not alive at queue creation time, the queue will be removed directly afterwards. An info report will be issued when the queue is removed.

* {mod, M}, indicates which queue behavior to use. Default is jobs_queue.

In addition, some 'abbreviated' options are supported:

* {standard_rate, R} - equivalent to [{regulators,[{rate,[{limit,R}, {modifiers,[{cpu,10},{memory,10}]}]}]}]

* {standard_counter, C} - equivalent to [{regulators,[{counter,[{limit,C}, {modifiers,[{cpu,10},{memory,10}]}]}]}]

* {producer, F} - equivalent to {type, {producer, F}}

* passive - equivalent to {type, {passive, fifo}}

* approve | reject - equivalent to {action, approve | reject}

Regulators

1. {limit, Limit} where Limit is the maximum rate (requests/sec)
2. {modifiers, Mods}, control feedback-based regulation. See below.
3. {name, Name}, optional. The default name for the regulator is {rate, QueueName, N}, where N is an index indicating which rate regulator in the list is referred. Currently, at most one rate regulator is allowed, so N will always be 1.

1. {limit, Limit}, where Limit is the number of concurrent jobs allowed.
2. {increment, Incr}, increment per job. Default is 1.
3. {modifiers, Mods}, control feedback-based regulation. See below.

* {named_counter, Name, Incr}, use an existing counter, incrementing it with Incr for each job. Name can either refer to a named top-level counter (see add_counter/2), or a queue-specific counter (these are named {counter,Qname,N}, where N is an index specifying their relative position in the regulators list - e.g. first or second counter).

* {group_rate, R}, refers to a top-level group rate R. See add_group_rate/2.

Types

* fifo | lifo - these are the types supported by the default queue behavior. While lifo may sound like an odd choice, it may have benefits for stochastic traffic with time constraints: there is no point to 'fairness', since requests cannot control their place in the queue, and choosing the 'freshest' job may increase overall goodness critera.

1. A fun of arity 0, indicating a stateless producer
2. A fun of arity 2, indicating a stateful producer
3. {M, F, A}, indicating a stateless producer
4. {Mod, Args} indicating a stateful producer

* {action, approve | reject}, specifies an automatic response to every request. This can be used to either block a queue (reject) or set it as a pass-through ('approve').

Modifiers

Jobs supports feedback-based modification of regulators.

The sampler framework sends feedback messages of type [{Modifier, Local, Remote::[{node(), Level}]}].

Each regulator can specify a list of modifier instructions:

* {Modifier, Local, Remote} - Modifier can be any label used by the samplers (see jobs_sampler). Local and Remote indicate increments in percent by which to reduce the limit of the given regulator. The Local increment is used for feedback info pertaining to the local node, and the Remote increment is used for remote indicators. Local is given as a percentage value (e.g. 10 for 10 %). The Remote increment is either {avg, Percent} or {max, Percent}, indicating whether to respond to the average load of other nodes or to the most loaded node. The correction from Local and the correction from Remote are summed before applying to the regulator limit.

* {Modifier, Local} - same as above, but responding only to local indications, ignoring the load on remote nodes.

* {Modifier, F::function((Local, Remote) -> integer())} - the function F(Local, Remote) is applied and expected to return a correction value, in percentage units.

* {Modifier, {Module, Function}} - Module:Function(Local Remote) is applied an expected to return a correction value in percentage units.

For example, if a rate regulator has a limit of 100 and has a modifier, {cpu, 10}, then a feedback message of {cpu, 2, _Remote} will reduce the rate limit by 2*10 percent, i.e. down to 80.

Note that modifiers are always applied to the preset limit, not the current limit. Thus, the next round of feedback messages in our example will be applied to the preset limit of 100, not the 80 that resulted from the previous feedback messages. A correction value of 0 will reset the limit to the preset value.

ask/1

Asks permission to run a job of Type. Returns when permission granted.

ask_queue/2

Sends a synchronous request to a specific queue.

delete_counter/1

Deletes a named counter from the load regulator on the current node. Returns true if there was in fact such a counter; false otherwise.

delete_group_rate/1

delete_queue/1

Deletes the named queue from the load regulator on the current node. Returns true if there was in fact such a queue; false otherwise.

dequeue/2

Extracts up to N items from a passive queue

Note that this function only works on passive queues. An exception will be raised if the queue doesn't exist, or if it isn't passive.

This function will block until at least one item can be extracted from the queue (see enqueue/2). No more than N items will be extracted.

done/1

Signals completion of an executed task.

enqueue/2

Inserts Item` into a passive queue. Note that this function only works on passive queues. An exception will be raised if the queue doesnt exist, or isn't passive.

info/1

job_info/1

Retrieves job-specific information from the Opaque data object.

modify_counter/2

modify_group_rate/2

modify_queue/2

Modifies queue parameters of existing queue.

modify_regulator/4

queue_info/1

queue_info/2

run/2

Executes Function() when permission has been granted by job regulator.

  case jobs:ask(Type) of
     {ok, Opaque} ->
        try Function()
          after
            jobs:done(Opaque)
        end;
     {error, Reason} ->
        erlang:error(Reason)
  end.
  

Overview

Generated by EDoc