NAME
puffs_framebuf —
buffering and event
handling for networked file systems
LIBRARY
puffs Convenience Library (libpuffs, -lpuffs)
SYNOPSIS
#include <puffs.h>
struct puffs_framebuf *
puffs_framebuf_make();
void
puffs_framebuf_destroy(
struct
puffs_framebuf *pufbuf);
void
puffs_framebuf_recycle(
struct
puffs_framebuf *pufbuf);
int
puffs_framebuf_reserve_space(
struct
puffs_framebuf *pufbuf,
size_t space);
int
puffs_framebuf_putdata(
struct puffs_framebuf
*pufbuf,
const void *data,
size_t
dlen);
int
puffs_framebuf_putdata_atoff(
struct
puffs_framebuf *pufbuf,
size_t offset,
const void *data,
size_t dlen);
int
puffs_framebuf_getdata(
struct puffs_framebuf
*pufbuf,
void *data,
size_t
dlen);
int
puffs_framebuf_getdata_atoff(
struct
puffs_framebuf *pufbuf,
size_t offset,
void *data,
size_t dlen);
size_t
puffs_framebuf_telloff(
struct
puffs_framebuf *pufbuf);
size_t
puffs_framebuf_tellsize(
struct
puffs_framebuf *pufbuf);
size_t
puffs_framebuf_remaining(
struct
puffs_framebuf *pufbuf);
int
puffs_framebuf_seekset(
struct
puffs_framebuf *pufbuf,
size_t offset);
int
puffs_framebuf_getwindow(
struct puffs_framebuf
*pufbuf,
size_t offset,
void
**winp,
size_t *winlen);
int
puffs_framev_enqueue_cc(
struct puffs_cc
*pcc,
int fd,
struct
puffs_framebuf *pufbuf,
int flags);
void
puffs_framev_cb(
struct puffs_usermount
*pu,
int fd,
struct puffs_framebuf
*pufbuf,
void *arg,
int
flags,
int error);
void
puffs_framev_enqueue_cb(
struct puffs_usermount
*pu,
int fd,
struct puffs_framebuf
*pufbuf,
puffs_framebuf_cb fcb,
void *fcb_arg,
int flags);
void
puffs_framev_enqueue_justsend(
struct
puffs_usermount *pu,
int fd,
struct puffs_framebuf *pufbuf,
int
waitreply,
int flags);
void
puffs_framev_enqueue_directsend(
struct
puffs_usermount *pu,
int fd,
struct puffs_framebuf *pufbuf,
int
flags);
void
puffs_framev_enqueue_directreceive(
struct
puffs_usermount *pu,
int fd,
struct puffs_framebuf *pufbuf,
int
flags);
int
puffs_framev_framebuf_ccpromote(
struct
puffs_framebuf *pufbuf,
struct puffs_cc *pcc);
int
puffs_framev_enqueue_waitevent(
struct
puffs_cc *pcc,
int
fd,
int *what);
int
puffs_framev_readframe_fn(
struct
puffs_usermount *pu,
struct puffs_framebuf
*pufbuf,
int fd,
int *done);
int
puffs_framev_writeframe_fn(
struct
puffs_usermount *pu,
struct puffs_framebuf
*pufbuf,
int fd,
int *done);
int
puffs_framev_cmpframe_fn(
struct
puffs_usermount *pu,
struct puffs_framebuf *cmp1,
struct puffs_framebuf *cmp2,
int
*notresp);
void
puffs_framev_gotframe_fn(
struct
puffs_usermount *pu,
struct puffs_framebuf
*pufbuf);
void
puffs_framev_fdnotify_fn(
struct
puffs_usermount *pu,
int fd,
int
what);
void
puffs_framev_init(
struct puffs_usermount
*pu,
puffs_framev_readframe_fn rfb,
puffs_framev_writeframe_fn wfb,
puffs_framev_cmpframe_fn cmpfb,
puffs_framev_gotframe_fn gotfb,
puffs_framev_fdnotify_fn fdnotfn);
int
puffs_framev_addfd(
struct
puffs_usermount *pu,
int
fd,
int what);
int
puffs_framev_enablefd(
struct
puffs_usermount *pu,
int
fd,
int what);
int
puffs_framev_disablefd(
struct
puffs_usermount *pu,
int
fd,
int what);
int
puffs_framev_removefd(
struct
puffs_usermount *pu,
int
fd,
int error);
void
puffs_framev_unmountonclose(
struct
puffs_usermount *pu,
int fd,
int
what);
DESCRIPTION
The
puffs_framebuf routines provide buffering and an event
loop structured around the buffers. It operates on top of the puffs
continuation framework,
puffs_cc(3), and multiplexes
execution automatically to an instance whenever one is runnable.
The file system is entered in three different ways:
- An event arrives from the kernel and the
puffs_ops(3) callbacks
are called to start processing the event.
- A file system which has sent out a request receives a
response. Execution is resumed from the place where the file system
yielded.
- A request from a peer arrives. A request is an incoming
PDU which is not a response to any outstanding request.
puffs_framebuf is used by defining various callbacks and
providing I/O descriptors, which are then monitored for activity by the
library. A descriptor, when present, can be either enabled or disabled for
input and output. If a descriptor is not enabled for a certain direction, the
callbacks will not be called even if there were activity on the descriptor.
For example, even if a network socket has been added and there is input data
in the socket buffer, the read callback will be called only if the socket has
been enabled for reading.
File descriptors are treated like sockets: they have two sides, a read side and
a write side. The framework determines that one side of the descriptor has
been closed if the supplied I/O callbacks return an error or if the I/O
multiplexing call says a side has been closed. It is still possible, from the
framework perspective, to write to a file descriptor whose read side is
closed. However, it is not possible to wait for a response on such a file
descriptor. Conversely, it is possible to read responses from a descriptor
whose write side is closed. It should be stressed that the implementation
underlying the file descriptor might not support this.
The following callbacks can be defined, cf.
puffs_framev_init(), and all are optional. None of them
should block, because this would cause the entire file server to block. One
option is to make the descriptors non-blocking before adding them.
-
-
- rfb
- Read a frame from the file descriptor onto the specified
buffer.
-
-
- wfb
- Write a frame from the specified buffer into the file
descriptor.
-
-
- cmpfb
- Identify if a buffer is the response to the specified
buffer.
-
-
- gotfb
- Called iff no outstanding request matches the incoming
frame. In other words, this is called when we receive a request from a
peer.
-
-
- fdnotfn
- Receive notifications about a change-of-state in a file
descriptor's status.
Better descriptions for each callback are given below.
The buffers of
puffs_framebuf provide automatic memory
management of buffers for the file servers. They provide a cursor to the
current buffer offset. Reading or writing data through the normal routines
will advance that cursor. Additionally, the buffer size is provided to the
user. It represents the maximum offset where data was written.
Generally the write functions will fail if the cannot allocate enough memory to
satisfy the buffer length requirements. Read functions will fail if the amount
of data written to the buffer is not large enough to satisfy the read.
-
-
- puffs_framebuf_make()
- Create a buffer. Return the address of the buffer or
NULL
in case no memory was available.
-
-
- puffs_framebuf_destroy(pufbuf)
- Free memory used by buffer.
-
-
- puffs_framebuf_recycle(pufbuf)
- Reset offsets so that buffer can be reused. Does not free
memory or reallocate memory.
-
-
- puffs_framebuf_reserve_space(pufbuf,
space)
- Make sure that the buffer has space
bytes of available memory starting from the current offset. This is not
strictly necessary, but can be used for optimizations where it is known in
advance how much memory will be required.
-
-
- puffs_framebuf_putdata(pufbuf,
data, dlen)
- Write dlen amount of data from the
address data into the buffer. Moves the offset
cursor forward dlen bytes.
-
-
- puffs_framebuf_putdata_atoff(pufbuf,
offset, data,
dlen)
- Like puffs_framebuf_putdata(), except
writes data at buffer offset offset. It is legal to
write past the current end of the buffer. Does NOT modify the current
offset cursor.
-
-
- puffs_framebuf_getdata(pufbuf,
data, dlen)
- Read dlen bytes of data from the
buffer into data. Advances the offset cursor.
-
-
- puffs_framebuf_getdata_atoff(pufbuf,
offset, data,
dlen)
- Read data from buffer position
offset. Does NOT modify the offset cursor.
-
-
- puffs_framebuf_telloff(pufbuf)
- Return the offset into the buffer.
-
-
- puffs_framebuf_tellsize(pufbuf)
- Return the maximum offset where data has been written, i.e.
buffer size.
-
-
- puffs_framebuf_remaining(pufbuf)
- Distance from current offset to the end of the buffer, i.e.
size-offset.
-
-
- puffs_framebuf_seekset(pufbuf,
offset)
- Set the offset cursor to the position
offset. This does NOT modify the buffer size, but
reserves at least enough memory memory for a write to
offset and will fail if memory cannot be
allocated.
-
-
- puffs_framebuf_getwindow(pufbuf,
offset, winp,
winlen)
- Get a direct memory window into the buffer starting from
offset. The maximum mapped window size will be
winlen bytes, but this routine might return a
smaller window and the caller should always check the actual mapped size
after the call. The window is returned in winp. This
function not modify the buffer offset, but it DOES set the buffer size to
offset + winlen in case that
value is greater than the current size. The window is valid until the next
until the next puffs_framebuf() call operating on the
buffer in question.
-
-
- puffs_framev_enqueue_cc(pcc,
fd, pufbuf,
flags)
- Add the buffer pufbuf to outgoing
queue of descriptor fd and yield with the
continuation pcc. Execution is resumed once a
response is received. Returns 0 if the buffer was successfully enqueued
(not necessarily delivered) and non-zero to signal a non-recoverable
error.
Usually the buffer is placed at the end of the output queue. However, if
flags contains
PUFFS_FBQUEUE_URGENT
, pufbuf
is placed in the front of the queue to be sent immediately after the
current PDU (if any) has been sent.
-
-
- puffs_framev_enqueue_cb(pu,
fd, pufbuf,
fcb, fcb_arg,
flags)
- Enqueue the buffer pufbuf for
outgoing data and immediately return. Once a response arrives, the
callback fcb() will be called with the argument
fcb_arg. The callback function
fcb() is responsible for freeing the buffer. Returns 0
if the buffer was successfully enqueued (not necessarily delivered) and
non-zero to signal a non-recoverable error.
See puffs_framev_enqueue_cc() for
flags.
-
-
- puffs_framev_cb(pu,
pufbuf, arg,
error)
- Callback function. Called when a response to a specific
request arrives from the server. If error is
non-zero, the framework was unable to obtain a response and the function
should not examine the contents of pufbuf, only do
resource cleanup. May not block.
-
-
- puffs_framev_enqueue_justsend(pu,
fd, pufbuf,
waitreply, flags)
- Enqueue the buffer pufbuf for
outgoing traffic and immediately return. The parameter
waitreply can be used to control if the buffer is to
be freed immediately after sending of if a response is expected and the
buffer should be freed only after the response arrives (receiving an
unexpected message from the server is treated as an error). Returns 0 if
the buffer was successfully enqueued (not necessarily delivered) and
non-zero to signal a non-recoverable error.
See puffs_framev_enqueue_cc() for
flags.
-
-
- puffs_framev_enqueue_directsend(pcc,
fd, pufbuf,
flags)
- Acts like puffs_framev_enqueue_justsend()
with the exception that the call yields until the frame has been sent. As
opposed to puffs_framev_enqueue_cc(), the routine does
not wait for input, but returns immediately after sending the frame.
See puffs_framev_enqueue_cc() for
flags.
-
-
- puffs_framev_enqueue_directreceive(pcc,
fd, pufbuf,
flags)
- Receive data into pufbuf. This
routine yields until a complete frame has been read into the buffer by the
readframe routine.
See puffs_framev_enqueue_cc() for
flags.
-
-
- puffs_framev_framebuf_ccpromote(pufbuf,
pcc)
- Promote the framebuffer pufbuf sent
with puffs_framev_enqueue_cb() or
puffs_framev_enqueue_justsend() to a wait using
pcc and yield until the result arrives. The response
from the file server for pufbuf must not yet have
arrived. If sent with puffs_framev_enqueue_justsend(),
the call must be expecting a response.
-
-
- puffs_framev_enqueue_waitevent(pcc,
fd, what)
- Waits for an event in what to happen
on file descriptor fd. The events which happened are
returned back in what. The possible events are
PUFFS_FBIO_READ
,
PUFFS_FBIO_WRITE
, and
PUFFS_FBIO_ERROR
, specifying read, write and error
conditions, respectively. Error is always checked.
This call does not depend on if the events were previously enabled on the
file descriptor - the specified events are always checked regardless.
There is currently no other way to cancel or timeout a call except by
removing the file descriptor in question. This may change in the
future.
-
-
- puffs_framev_readframe_fn(pu,
pufbuf, fd,
done)
- Callback function. Read at most one frame from file
descriptor fd into the buffer
pufbuf. If a complete frame is read, the value
pointed to by done must be set to 1. This function
should return 0 on success (even if a complete frame was not yet read) and
a non-zero
errno
to signal a fatal error. In case
a fatal error is returned, the read side of the file descriptor is marked
closed. This routine will be called with the same buffer argument until a
complete frame has been read. May not block.
-
-
- puffs_framev_writeframe_fn(pu,
pufbuf, fd,
done)
- Write the frame contained in pufbuf
to the file descriptor fd. In case the entire frame
is successfully written, *done should be set to 1.
This function should return 0 on success (even if a complete frame was not
yet written) and a non-zero
errno
to signal a
fatal error. In case a fatal error is returned, the write side of the file
descriptor is marked closed. This routine will be called with the same
buffer argument until the complete frame has been written. May not block.
It is a good idea to make sure that this function can handle a possible
SIGPIPE
caused by a closed connection. For
example, the file server can opt to trap SIGPIPE
or, if writing to a socket, call send() with the flag
MSG_NOSIGNAL
instead of using
write().
-
-
- puffs_framev_cmpframe_fn(pu,
pufbuf_cmp1, pufbuf_cmp2,
notresp)
- Compare the file system internal request tags in
pufbuf_cmp1 and pufbuf_cmp2.
Should return 0 if the tags are equal, 1 if first buffer's tag is greater
than the second and -1 if it is smaller. The definitions
"greater" and "smaller" are used transparently by the
library, e.g. like qsort(3).
If it can be determined from pufbuf_cmp1 that it is
not a response to any outstanding request, notresp
should be set to non-zero. This will cause
puffs_framebuf to skip the test of the buffer against
the rest of the outstanding request. May not block.
-
-
- puffs_framev_gotframe_fn(pu,
pufbuf)
- Called when no outstanding request matches an incoming
frame. The ownership of pufbuf is transferred to the
called function and must be destroyed once processing is over. May not
block.
-
-
- puffs_framev_fdnotify_fn(pu,
fd, what)
- Is called when the read or write side of the file
descriptor fd is closed. It is called once for each
side, the bitmask parameter what specified what is
currently closed:
PUFFS_FBIO_READ
and
PUFFS_FBIO_WRITE
for read and write,
respectively.
-
-
- puffs_framev_init(pu,
rfb, wfb,
cmpfb, gotfb,
fdnotfn)
- Initializes the given callbacks to the system. They will be
used when puffs_mainloop() is called. The framework
provides the routines puffs_framev_removeonclose() and
puffs_framev_unmountonclose(), which can be given as
fdnotfn. The first one removes the file descriptor
once both sides are closed while the second one unmounts the file system
and exits the mainloop.
-
-
- puffs_framev_addfd(pu,
fd, what)
- Add file descriptor fd to be handled
by the framework. It is legal to add a file descriptor either before
calling puffs_mainloop() or at time when running. The
parameter what controls enabling of input and output
events and can be a bitwise combination of
PUFFS_FBIO_READ
and
PUFFS_FBIO_WRITE
. If not specified, the descriptor
will be in a disabled state.
-
-
- puffs_framev_enablefd(pu,
fd, what)
- Enable events of type what for file
descriptor fd.
-
-
- puffs_framev_disablefd(pu,
fd, what)
- Disable events of type what for file
descriptor fd.
-
-
- puffs_framev_removefd(pu,
fd, error)
- Remove file descriptor fd from the
list of descriptors handled by the framework. Removing a file descriptor
causes all operations blocked either on output or input to be released
with the error value error. In case 0 is supplied as
this parameter,
ECONNRESET
is used.
The file system must explicitly remove each fd it has
added. A good place to do this is
puffs_framev_fdnotify_fn() or
puffs_node_reclaim(), depending a little on the
structure of the file system.
-
-
- puffs_framev_unmountonclose(pu,
fd, what)
- This is library provided convenience routine for
puffs_framev_fdnotify_fn(). It unmounts the file system
when both the read and write side are closed. It is useful for file
systems such as
mount_psshfs(8) which
depend on a single connection.
RETURN VALUES
These functions generally return -1 to signal error and set
errno
to indicate the type of error.
CODE REFERENCES
The current users of
puffs_framebuf in the tree are
mount_psshfs(8) and
mount_9p(8). See
src/usr.sbin/puffs/mount_psshfs and
src/usr.sbin/puffs/mount_9p for the respective usage
examples.
SEE ALSO
puffs(3),
puffs_cc(3),
puffs_ops(3)
Antti Kantee, Using
puffs for Implementing Client-Server Distributed File Systems,
Helsinki University of Technology, Tech
Report TKK-TKO-B157, September 2007.
Antti Kantee, Send
and Receive of File System Protocols: Userspace Approach With puffs,
Proceedings of AsiaBSDCon 2008, pp.
55-70, March 2008.