[PREVIOUS CHAPTER] [NEXT CHAPTER]
5 File Operation Functions

5.1	mget options


"mget" command has several options. See help file also.

usage:	mget range [mode] [time_unit]

   range		1-10,12,15-100
			1,2,10
			first  	 (MH linke)
			first:10 (MH linke)
			last:30  (MH linke)
			100-last (MH linke)
   mode			gz tgz ish rfc934 b rfc1153 d unpack uf(default=tgz)
   time_unit		time between sending partial files, 1/N and 2/N ...

   mode variation:	default	is tgz (== tar + gzip).

	tgz	tar + gzip
	uf	PLAINTEXT(UNIX FROM)
	gz	GZIP(UNIX FROM)
	b	RFC934 format 	PLAINTEXT(mh-burst)
	rfc934	
	ish	lha + ish 
	lhaish
	i
	li
	lu	lha + uuencode
	lhauu
	unpack	PLAINTEXT(UNIX FROM)
	uu	UUENCODE
	d	RFC1153 format 	PLAINTEXT
	rfc1153	RFC1153 format 	PLAINTEXT

5.2	mget3 Routine Default


* "mget" command
$MGET_MODE_DEFAULT can change the default file format "mget" command
sends back. The default format is "tgz" on UNIX, 'mp' on Windows NT4.


In libsendfile.pl, 

    local($mode) = $MGET_MODE_DEFAULT || 'tgz'; # default


Available options for $MGET_MODE_DEFAULT is %_fp defined in libfop.pl.
The options are one of them:

	uf
	rfc934
	rfc1153
	mp
	gz
	tgz
	lhaish
	lhauu
	uu

* library commands and "get" files in archive. 

In retrieving files of library or archive directories,

	Text		$MGET_TEXT_MODE_DEFAULT
			(default is 'mp' MIME/Multipart)

	Binary		$MGET_BIN_MODE_DEFAULT
			(default is  'uu', uuencoded file); 


	$mode = -T $opt ? 
	    ($MGET_TEXT_MODE_DEFAULT || 'mp'): 
	($MGET_BIN_MODE_DEFAULT || 'uu'); 


* Subject: of "mget" return files

	$MGET_SUBJECT_TEMPLATE


defines the subject of sending back mails. If it is defined as

   result for mget _DOC_MODE_ _PART_ _ML_FN_


FML replaces each keyword in the following

	   _DOC_MODE_   <=>    [last:10 tar + gzip]
	   _PART_       <=>    (1/4)
	   _ML_FN_      <=>    $ML_FN (here is "(Elena Lolobrigita ML)") 


to result in

   Subject: result for mget [last:3 tar + gzip] (1/1) (Elena Lolobrigita ML)


You can use subject with these options. 


5.3	mget3 search algorithm 


	expand MH like syntax
		e.g. 1,last:30 -> 1,70-100

	if V2 search articles in
		e.g.
		spool
		archive/100.tar.gz 

	elsif search archive directories

	elsif V1 format (obsolete)

	else
		NO MATCH or V1 FORMAT ERROR

5.4	Program paths


File operations uses system programs e.g. tar, gzip ....  "makefml"
searches the OS for them and caches them in config.ph. It is good for
you to check them again by yourself. It is useful to define them in
common definition file "sitedef.ph", "site_init.ph".


	variable			example

	$TAR                           = "/usr/bin/tar cf -";
	$UUENCODE                      = "/usr/bin/uuencode";
	$COMPRESS                      = "/usr/bin/gzip -c";
	$ZCAT                          = "/usr/bin/zcat";
	$LHA                           = "/usr/local/bin/lha";
	$ISH                           = "/usr/local/bin/ish";

5.5	mget3 process


The last stage of "mget3" routine process is sending back requested
files. It is as follows.


   * 	mget3 creates temporary files under locked.
	The temporary files are created in $TMP_DIR.
	After unlock, sending back process runs.


The filename is


	m:"mode":"process_id"return.number(split number)


FML requires a set of (mode pid number) information to ensure
uniqueness since "mget" return processes may run in parallel.  After
temporary files creation, mget routine returns to the main routine.


   * the main routine unlocks itself


   * 	&RunStartHooks calls &mget3_SendingEntry after unlock


mget3_SendingEntry calls mget3 sending back routines with %mget_list.
In this stage plural mget3 sending back processes are runnable in
pararell.


If plural files should be sent back, FML tries to send back one mail
in $MGET_SEND_BACK_SLEEPTIME seconds. 


5.6	Warn a command syntax error ?

e.g. in config.ph


To warn a command syntax error, set

	$USE_WARNING = 1;


But syntax error warnings must be noisy:) since it is so often. 
FML does not warn it in default.

e.g. 

5.7	File size unit of "mget" sending back process

	$MAIL_LENGTH_LIMIT (default is 1000 lines)


is the unit in breaking up files. 1000 lines is about 50 K-bytes.  


5.8	The maximum files for "mget" parameters


The maximum number of files "mget" can provide. The default value is
1000.  When "mget *" matches files > 1000, FML returns the error.

	$MGET_SEND_BACK_FILES_LIMIT

5.9	Mget Format: Unix From Format


We call BSD's /var/mail/$USER style UNIX FROM format.
Example:

	From fukachan Mon Jan 26 09:11:46 1998
	Received: by hikari.fml.org (8.8.8/3.4W6+Hikari-2.1) 
		with ESMTP id JAA02846 for <uja@fml.org>; 
		Mon, 26 Jan 1998 09:11:46 +0900 (JST)
	From: fukachan@sapporo.iij.ad.jp
	Subject: uja


	... mail body ....


5.10	Mget Format: MIME/Multipart BNF representation

   boundary := 0*69<bchars> bcharsnospace

   bchars := bcharsnospace / " "

   bcharsnospace :=    DIGIT / ALPHA / "'" / "(" / ")" / "+" /"_"
                 / "," / "-" / "." / "/" / ":" / "=" / "?"

   Overall, the body of a multipart entity may be specified  as
   follows:

   multipart-body := preamble 1*encapsulation
                  close-delimiter epilogue

   encapsulation := delimiter body-part CRLF

   delimiter := "--" boundary CRLF ; taken from Content-Type field.
                                   ; There must be no space
                                   ; between "--" and boundary.

   close-delimiter := "--" boundary "--" CRLF ; Again, no space
   by "--",

   preamble := discard-text   ;  to  be  ignored upon receipt.

   epilogue := discard-text   ;  to  be  ignored upon receipt.

   discard-text := *(*text CRLF)

   body-part := <"message" as defined in RFC 822,
             with all header fields optional, and with the
             specified delimiter not occurring anywhere in
             the message body, either on a line by itself
             or as a substring anywhere.  Note that the
             semantics of a part differ from the semantics
             of a message, as described in the text.>

5.11	Mget Format: An example of MIME/Multipart

Date: Thu, 18 May 95 22:33:00 JST
From: Elena@phys.titech.ac.jp (Elena Lologrigita)
Subject: P R E S E N T
To: fukachan@phys.titech.ac.jp
MIME-Version: 1.0
Content-type: multipart/mixed;
	boundary="--Thu_18_May_95_22:32:56--"

----Thu_18_May_95_22:32:56--
Content-Type: message/rfc822

Date: Thu, 18 May 95 21:45:17 JST
From: fukachan@phys.titech.ac.jp
Subject: Ahoudori
To: Elena@phys.titech.ac.jp (Elena Lolobrigita ML)
Message-ID: <19950518.01905.Elena.Lolobrigita@Baycity.asia>

Feel so good!

--fukachan

----Thu_18_May_95_22:32:56--
Content-Type: message/rfc822

Date: Thu, 18 May 95 22:00:29 JST
From: Elena@phys.titech.ac.jp
Subject: Ashita Kick Off
To: Elena@phys.titech.ac.jp (Elena Lolobrigita ML)
Message-ID: <19950518.01906.Elena.Lolobrigita@Baycity.asia>

Amino Tetsuro's direction is excellent, who is a representative of
Japanese Animation.

----Thu_18_May_95_22:32:56----

5.12	Mget Format: RFC934

rfc934 "Proposed Standard for Message Encapsulation"


You can this format by MH burst. 

Example:
------- Forwarded Message

From: Anna
Subject: Mama told me

#32 "Ahoudori" (Freekick) is the core of Freekick.

------- Forwarded Message

From: Elena
Subject: Albatros

Ingrid must be a sister of Shun :-)

------- Forwarded Message

5.13	Mget Format: RFC1153 digest 

Example:

Date: Sat, 20 May 95 01:33:50 JST
From: Elena@phys.titech.ac.jp
Subject: Freekick
To: fukachan@phys.titech.ac.jp

GODAI BBS DIGEST	Sat, 20 May 95	Volume 95: Issue 120

Today's Topics:
	Elena's hobby is watering.

------------------------------

Date: Sat, 20 May 95 00:00:31 JST
From: "Ken'ichi Fukamachi" <fukachan>
Subject: Elena's hobby is watering.
X-Mail-Count: 01932

Elena's hobby is watering.

--fukachan 
Albatros follows wind.

------------------------------

End of GODAI BBS Digest V95 Issue #120
********************************

5.14	Mget Format: Customize RFC1153 format

	
$RFC1153_ISSUE		the sequence number of RFC1153 format style.
			It is independent of ML article sequence number
			($SEQUENCE_FILE 's content).


$RFC1153_SEQUENCE_FILE	the file to save $RFC1153_ISSUE
			We cannot resolve how to count 1153 sequence.
			We count the sequence as how many RFC1153
			routine is called.


$RFC1153_LISTNAME 	Title of RFC1153 style digest


$RFC1153_VOL		Volume number e.g. Volume 96


$RFC1153_CUSTOM_HOOK	Hook to run &Rfc1153Custom($mode, *conf)

&Rfc1153Custom($mode, *conf) internal:

    ########## CUSTOMIZE BELOW ##########
    $issue     = $RFC1153_ISSUE    || 1;
    $listname  = $RFC1153_LISTNAME || "UJA";
    $vol       = $RFC1153_VOL      || $year;
    $IssueSeq  = $RFC1153_SEQUENCE_FILE  || 
	"$FP_VARLOG_DIR/IssueSeq"; # file to remember count;

    &GetTime;
    &eval($RFC1153_CUSTOM_HOOK, 'RFC1153 custom:');
    $issue = &Rfc1153GetSeq($IssueSeq);

5.15	Mget Format: lzh + ish option


	mget 200-210 ish


enables "mget" to use lha and ish.

	$LHA = "/usr/share/bin/lha";
	$ISH = "/home/axion/fukachan/bin/ish";


"makefml" automatically set up them in config.ph but it is good for
you to check it again.

5.16	Convert to SJIS (Japanese) (lzh + ish and lhz + uuencode option)


When $USE_SJIS_IN_ISH = 1;, FML appends ^M in the last of each line
and ^Z in EOF. jcode.pl is required in @INC. FML does them by itself
not depending external commands.


[PREVIOUS CHAPTER] [NEXT CHAPTER]