NAME
dmover_backend_register,
dmover_backend_unregister,
dmover_session_create,
dmover_session_destroy,
dmover_request_alloc,
dmover_request_free,
dmover_process,
dmover_done —
hardware-assisted data mover interface
SYNOPSIS
#include <dev/dmover/dmovervar.h>
Client interface routines:
int
dmover_session_create(
const
char *,
struct
dmover_session **);
void
dmover_session_destroy(
struct
dmover_session *);
struct dmover_request *
dmover_request_alloc(
struct
dmover_session *,
dmover_buffer *);
void
dmover_request_free(
struct
dmover_request *);
void
dmover_process(
struct
dmover_request *);
Back-end interface routines:
void
dmover_backend_register(
struct
dmover_backend *);
void
dmover_backend_unregister(
struct
dmover_backend *);
void
dmover_done(
struct
dmover_request *);
DESCRIPTION
The
dmover facility provides an interface to hardware-assisted
data movers. This can be used to copy data from one location in memory to
another, clear a region of memory, fill a region of memory with a pattern, and
perform simple operations on multiple regions of memory, such as an XOR,
without intervention by the CPU.
The drivers for hardware-assisted data movers present themselves to
dmover by registering their capabilities. When a client
wishes to use a
dmover function, it creates a session for
that function, which identifies back-ends capable of performing that function.
The client then enqueues requests on that session, which the back-ends process
asynchronously. The client may choose to block until the request is completed,
or may have a call-back invoked once the request has been completed.
When a client creates a session, the
dmover facility
identifies back-ends which are capable of handling the requested function.
When a request is scheduled for processing, the
dmover
scheduler will identify the best back-end to process the request from the list
of candidate back-ends, in an effort to provide load balancing, while
considering the relative performance of each back-end.
A
dmover function always has one output region. A function may
have zero or more input regions, or may use an immediate value as an input.
For functions which use input regions, the lengths of each input region and
the output region must be the same. All
dmover functions
with the same name will have the same number of and type inputs. If a back-end
attempts to register a function which violates this invariant, behavior is
undefined.
The
dmover facility supports several types of buffer
descriptors. For functions which use input regions, each input buffer
descriptor and the output buffer descriptor must be of the same type. This
restriction may be removed in a future revision of the interface.
The
dmover facility may need to interrupt request processing
and restart it. Clients of the
dmover facility should take
care to avoid unwanted side-effects should this occur. In particular, for
functions which use input regions, no input region may overlap with the output
region.
DATA STRUCTURES
The
dmover facility shares several data structures between the
client and back-end in order to describe sessions and requests.
typedef enum {
DMOVER_BUF_LINEAR,
DMOVER_BUF_UIO
} dmover_buffer_type;
typedef struct {
void *l_addr;
size_t l_len;
} dmover_buf_linear;
typedef union {
dmover_buf_linear dmbuf_linear;
struct uio *dmbuf_uio;
} dmover_buffer;
Together, these data types are used to describe buffer data structures which the
dmover facility understands. Additional buffer types may be
added in future revisions of the
dmover interface.
The
dmover_assignment structure contains the information
about the back-end to which a request is currently assigned. It contains the
following public members:
-
-
- struct dmover_backend
*das_backend
- This is a pointer to the back-end.
-
-
- const struct dmover_algdesc
*das_algdesc
- This is a pointer to the algorithm description provided by
the back-end for the request's function.
The
dmover_session structure contains the following public
members:
-
-
- void *dses_cookie
- This is a pointer to client private data.
-
-
- int dses_ninputs
- This is the number of inputs used by the selected
function.
The
dmover_request structure contains the following public
members:
-
-
- TAILQ_ENTRY(dmover_request)
dreq_dmbq
- Linkage on the back-end's queue of pending requests.
-
-
- struct dmover_session
*dreq_session
- Pointer to the session with which this request is
associated. This is intended for use by the back-end.
-
-
- struct dmover_assignment
*dreq_assignment
- Pointer to the dmover_assignment
structure which describes the back-end to which the request is currently
assigned. The back-end is assigned when the request is scheduled with
dmover_process().
-
-
- void (*dreq_callback)(struct
dmover_request *)
- This is a pointer to an optional call-back function
provided by the client. If provided, the call-back is invoked when the
request is complete. This field must be
NULL
if
DMOVER_REQ_WAIT is set in
dreq_flags.
-
-
- void *dreq_cookie
- This is a pointer to client private data specific to the
request.
-
-
- void *dreq_dmbcookie
- This is a pointer to back-end private data, for use while
the back-end is actively processing a request.
-
-
- volatile int dreq_flags
- The following flags are defined:
-
-
- DMOVER_REQ_DONE
- The request has been completed. If not using a
call-back, the client may poll this bit to determine if a request has
been processed.
-
-
- DMOVER_REQ_ERROR
- An error has occurred while processing the
request.
-
-
- DMOVER_REQ_RUNNING
- The request is currently being executed by the
back-end. Once a command is running, it cannot be cancelled, and must
run to completion.
-
-
- DMOVER_REQ_WAIT
- If set by the client,
dmover_process() will wait for the request to
complete using
cv_wait(9). This flag
may only be used if the caller has a valid thread context. If this
flag is set, a callback may not be used.
-
-
- int dreq_error
- If the DMOVER_REQ_ERROR bit is set, this
contains the errno(2) value
indicating the error that occurred during processing.
-
-
- dmover_buffer_type
dreq_outbuf_type
- The type of the output buffer.
-
-
- dmover_buffer
dreq_outbuf
- The output buffer.
-
-
- uint8_t
dreq_immediate[8]
- This is the input for algorithms which use an immediate
value. Values smaller than 8 bytes should use the least-significant bytes
first. For example, a 32-bit integer would occupy bytes 0, 1, 2, and
3.
-
-
- dmover_buffer_type
dreq_inbuf_type
- The type of the input buffer. This is only used if the
dmover function has one or more inputs.
-
-
- dmover_buffer
*dreq_inbuf
- A pointer to an array of input buffers. This is only used
if the dmover function has one or more inputs. The
number of inputs, and thus the number of valid elements in the array, is
specified by the algorithm description for the session.
CLIENT INTERFACE
The following functions are provided to the client:
-
-
- dmover_session_create(function,
sessionp)
-
The dmover_session_create() function creates a data mover
session for the specified data movement function
function. A handle to the new session is returned in
sessionp.
The following are valid data movement function names:
-
-
- “zero”
- Fill a memory region with zeros. This algorithm has an
input count of 0.
-
-
- “fill8”
- Fill a memory region with an 8-bit pattern. This
algorithm has an input count of 0. The pattern is provided in the
dreq_imm8 member of the
dmover_request structure.
-
-
- “copy”
- Copy a memory region from one location to another. This
algorithm has an input count of 1.
-
-
- “xor2”
- Perform an XOR operation on 2 inputs. This algorithm
has an input count of 2.
-
-
- “xor3”
- Perform an XOR operation on 3 inputs. This algorithm
has an input count of 3.
-
-
- “xor4”
- Perform an XOR operation on 4 inputs. This algorithm
has an input count of 4.
-
-
- “xor5”
- Perform an XOR operation on 5 inputs. This algorithm
has an input count of 5.
-
-
- “xor6”
- Perform an XOR operation on 6 inputs. This algorithm
has an input count of 6.
-
-
- “xor7”
- Perform an XOR operation on 7 inputs. This algorithm
has an input count of 7.
-
-
- “xor8”
- Perform an XOR operation on 8 inputs. This algorithm
has an input count of 8.
Users of the dmover facility are encouraged to use the
following aliases for the well-known function names, as doing so saves
space and reduces the chance of programming errors:
-
-
- DMOVER_FUNC_ZERO
- “zero”
(dmover_funcname_zero)
-
-
- DMOVER_FUNC_FILL8
- “fill8”
(dmover_funcname_fill8)
-
-
- DMOVER_FUNC_COPY
- “copy”
(dmover_funcname_copy)
-
-
- DMOVER_FUNC_XOR2
- “xor2”
(dmover_funcname_xor2)
-
-
- DMOVER_FUNC_XOR3
- “xor3”
(dmover_funcname_xor3)
-
-
- DMOVER_FUNC_XOR4
- “xor4”
(dmover_funcname_xor4)
-
-
- DMOVER_FUNC_XOR5
- “xor5”
(dmover_funcname_xor5)
-
-
- DMOVER_FUNC_XOR6
- “xor6”
(dmover_funcname_xor6)
-
-
- DMOVER_FUNC_XOR7
- “xor7”
(dmover_funcname_xor7)
-
-
- DMOVER_FUNC_XOR8
- “xor8”
(dmover_funcname_xor8)
-
-
- dmover_session_destroy(session)
-
The dmover_session_destroy() function tears down a data
mover session and releases all resources associated with it.
-
-
- dmover_request_alloc(session,
inbuf)
-
The dmover_request_alloc() function allocates a
dmover request structure and associates it with the
specified session. If the inbuf argument is not
NULL
, inbuf is used as the
array of input buffer descriptors in the request. Otherwise, if
inbuf is NULL
and the
dmover function requires input buffers, the input buffer
descriptors will be allocated automatically using
malloc(9).
If the request structure or input buffer descriptors cannot be allocated,
dmover_request_alloc() return
NULL
to indicate failure.
-
-
- dmover_request_free(req)
-
The dmover_request_free() function frees a
dmover request structure. If the
dmover function requires input buffers, and the input
buffer descriptors associated with req were
allocated by dmover_request_alloc(), then the input
buffer descriptors will also be freed.
-
-
- dmover_process(req)
-
The dmover_process() function submits the
dmover request req for processing.
The call-back specified by the request is invoked when processing is
complete.
The
dmover_session_create() and
dmover_session_destroy() functions must not be called from
interrupt context.
The
dmover_request_alloc(),
dmover_request_free(), and
dmover_process() functions may be called from interrupt
handlers at levels
IPL_VM,
IPL_SOFTCLOCK,
and
IPL_SOFTNET, or in non-interrupt context.
The request completion call-back is called from a software interrupt handler at
IPL_SOFTCLOCK.
BACK-END INTERFACE
A back-end describes the
dmover functions it can perform using
an array of
dmover_algdesc structures:
struct dmover_algdesc {
const char *dad_name; /* algorithm name */
void *dad_data; /* opaque algorithm description */
int dad_ninputs; /* number of inputs */
};
The
dad_name member points to a valid
dmover
function name which the client may specify. The
dad_data
member points to a back-end-specific description of the algorithm.
A back-end presents itself to the
dmover facility using the
dmover_backend structure. The back-end must initialize
the following members of the structure:
-
-
- const char *dmb_name
- This is the name of the back-end.
-
-
- u_int dmb_speed
- This is an estimate of the number of kilobytes/second that
the back-end can process.
-
-
- void *dmb_cookie
- This is a pointer to back-end private data.
-
-
- const struct dmover_algdesc
*dmb_algdescs
- This points to an array of
dmover_algdesc structures which describe the
functions the data mover can perform.
-
-
- int dmb_nalgdescs
- This is the number of elements in the
dmb_algdescs array.
-
-
- void (*dmb_process)(struct
dmover_backend *)
- This is the entry point to the back-end used to process
requests.
When invoked by the
dmover facility, the back-end's
(*dmb_process)() function should examine the pending request
queue in its
dmover_backend structure:
-
-
- TAILQ_HEAD(, dmover_request)
dmb_pendreqs
- This is the queue of pending requests.
-
-
- int dmb_npendreqs
- This is the number of requests in the
dmb_pendreqs queue.
If an error occurs when processing the request, the
DMOVER_REQ_ERROR bit must be set in the
dreq_flags member of the request, and the
dreq_error member set to an
errno(2) value to indicate the
error.
When the back-end has finished processing the request, it must call the
dmover_done() function. This function eventually invokes the
client's call-back routine.
If a hardware-assisted data mover uses interrupts, the interrupt handlers should
be registered at IPL_VM.
The following functions are provided to the back-ends:
-
-
- dmover_backend_register(backend)
-
The dmover_backend_register() function registers the
back-end backend with the dmover
facility.
-
-
- dmover_backend_unregister(backend)
-
The dmover_backend_unregister() function removes the
back-end backend from the dmover
facility. The back-end must already be registered.
-
-
- dmover_done(req)
-
The dmover_done() function is called by the back-end when
it has finished processing a request, whether the request completed
successfully or not.
The
dmover_backend_register() and
dmover_backend_unregister() functions must not be called
from interrupt context.
The
dmover_done() function may be called at
IPL_VM,
IPL_SOFTCLOCK,
IPL_SOFTNET, or in non-interrupt context.
EXAMPLES
The following is an example of a client using
dmover to
zero-fill a region of memory. In this example, the CPU will be able to context
switch to another thread and perform work while the hardware-assisted data
mover clears the specified block of memory.
int
hw_bzero(void *buf, size_t len)
{
struct dmover_session *dses;
struct dmover_request *dreq;
int error;
error = dmover_session_create(DMOVER_FUNC_ZERO, &dses);
if (error)
return (error);
dreq = dmover_request_alloc(dses, NULL);
if (dreq == NULL) {
dmover_session_destroy(dses);
return (ENOMEM);
}
dreq->dreq_flags = DMOVER_REQ_WAIT;
dreq->dreq_callback = NULL;
dreq->dreq_outbuf.dreq_outbuf_type = DMOVER_BUF_LINEAR;
dreq->dreq_outbuf.dmbuf_linear.l_addr = buf;
dreq->dreq_outbuf.dmbuf_linear.l_len = len;
dmover_process(dreq);
error = (dreq->dreq_flags & DMOVER_REQ_ERROR) ?
dreq->dreq_error : 0;
dmover_request_free(dreq);
dmover_session_destroy(dses);
return (error);
}
SEE ALSO
queue(3),
dmoverio(4)
HISTORY
The
dmover facility first appeared in
NetBSD
2.0.
AUTHORS
The
dmover facility was designed and implemented by
Jason R. Thorpe
⟨thorpej@wasabisystems.com⟩ and contributed by Wasabi Systems,
Inc.
BUGS
The mechanism by which a back-end should advertise its performance to the
request scheduler is not well-defined. Therefore, the load-balancing mechanism
within the request scheduler is also not well-defined.