mimedefang-multiplexor - Process pool controller for mail filters.
manages a pool of Perl processes for scanning
e-mail. It is designed to work in conjunction with mimedefang
opens a UNIX-domain socket and listens for
requests for work from mimedefang
. As requests come in,
creates Perl processes as needed to scan mail.
The Perl processes are not killed when scanning is completed, but continue to
run in a loop. Perl processes are re-used for subsequent e-mail messages. This
eliminates the large overhead of starting a new Perl process for each incoming
To avoid memory leaks, the Perl processes are killed after they have handled
some number of scans.
- -U user
- Runs the multiplexor as user rather than
root. This option is mandatory, and must match the -U option
supplied to mimedefang.
- -m minSlaves
- The minimum number of Perl processes to keep running at all
times. The default is zero.
- -x maxSlaves
- The maximum number of Perl processes to run simultaneously.
If a request comes in and all processes are busy, a temporary failure is
signalled to the SMTP peer. The default is 2.
- -r maxRequests
- The maximum number of requests a given process handles
before it is killed and a replacement started. The default is 500.
- -i idleTime
- The idle time in seconds after which to kill of excess Perl
processes. That is, if the process is idle for longer than this time, and
there are more than minSlaves running, the process is killed. Note
that this is implemented as a timer which ticks every idleTime
seconds; therefore, processes may be idle for up to twice this time before
they are killed. The default for idleTime is 300 seconds.
- -b busyTime
- The longest a Perl process is allowed to spend scanning an
e-mail before it is declared hung up and killed. The default is 120
- This option specifies that the multiplexor should accept
and process "status updates" from busy slaves. Note that this
consumes one extra file descriptor per slave, plus a small amount of CPU
time per status update.
- -c cmdTime
- The timeout for communication between
mimedefang-multiplexor and mimedefang, or between
mimedefang-multiplexor and a Perl scanning process. The default is
10 seconds. This timeout should be kept quite short.
- -w waitTime
- When mimedefang-multiplexor starts the initial
slaves, or needs to bring the number of running slaves up to the number
defined by the -m option, it does not start all the slaves at once,
because this could overload your server. Instead, it starts one slave
every waitTime seconds. The default value for waitTime is 3.
- -W waitTime
- If you use this option, mimedefang-multiplexor will
never activate a slave until waitTime seconds have elapsed
since the last slave activation. This could result in mail being
tempfailed if slave activations do not keep pace with incoming mail.
However, it may be preferable to tempfail mail rather than allow the load
on your server to spike up too quickly. The default value for this option
is 0, meaning that mimedefang-multiplexor will start slaves as
quickly as necessary to keep up with incoming mail.
- -z spooldir
- Set the spool directory to spooldir. If this option
is omitted, the spool directory defaults to /var/spool/MIMEDefang.
- -s pathName
- The UNIX-domain socket on which
mimedefang-multiplexor listens for requests. This should be
specified as an absolute pathname. If this option is not supplied, it
defaults to mimedefang-multiplexor.sock under the spool directory.
- -a socket
- A socket for listening for requests. This is similar to the
-s socket, except that a restricted set of requests are processed.
On this socket, the multiplexor will only process requests asking for
status; it will not accept any commands to do scanning or that would
consume a slave. See the SOCKET SPECIFICATION section for the format of
- -p fileName
- Causes mimedefang-multiplexor to write its
process-ID (after becoming a daemon) to the specified file.
- -f filter_path
- Normally, mimedefang-multiplexor executes a Perl
filter script called mimedefang.pl to scan the e-mail. However, you
can have it execute any program you like by specifying the full path to
the program with the -f option. This program must obey the protocol
documented in mimedefang-protocol(7); see that manual page for
Note that the -f option does not specify the
"filter" to use with mimedefang.pl; instead, it specifies
the program for mimedefang-multiplexor to execute. You almost
certainly should not use this option unless you wish to replace
mimedefang.pl with your own program.
- -F rules_path
- Specifies the path to the filter rules. By default,
/etc/mimedefang-filter is used. If you use the -F option,
its value is passed to the underlying Perl filter program using -f.
- Log certain events, including the output of the Perl
slaves' standard-error, using syslog. Normally, the multiplexor does not
log much information.
- Write debugging information about event-handling code in
/var/log/mimedefang-event-debug.log. This is only of use to people
- -R kbytes
- Limits the resident-set size of the slave filter processes
to kbytes kilobytes. This limit is not supported on all operating
systems; it is known to work on Linux.
- -M kbytes
- Limits the total memory space of slave filter processes to
kbytes kilobytes. This limit is supported on all operating systems
which support the setrlimit(2) system call. This should include most
modern UNIX systems.
We recommend that you monitor your slave filter processes and get a feel for
how much memory they use. You should then limit the memory to two or three
times the worst-case that you have observed. This can help mitigate
denial-of-service attacks which use complicated MIME messages to force
mimedefang.pl to consume lots of memory.
- Print usage information and exit.
- -t filename
- Log statistical information to filename. See the
section STATISTICS for more information.
- Log statistical information using syslog(2). You may
use any -t and -T together, in which case statistical
information is logged in a file and using syslog.
- Flush the statistics file after every write. Normally,
mimedefang-multiplexor does not flush the file; this is the best
choice for minimizing disk I/O on a busy mail server. However, if you wish
to watch statistics entries in real-time, you should enable flushing.
- Do not fork into the background and become a daemon.
Instead, stay in the foreground. Useful mainly for debugging or if you
have a supervisory process managing mimedefang-multiplexor.
- -q queue_size
- Normally, if all slaves are busy and
mimedefang-multiplexor receives another request, it fails it with
the error "No free slaves." However, if you use the -q
option, then up to queue_size requests will be queued. As soon as a
slave becomes free, the queued requests will be handed off in FIFO order.
If the queue is full and another request comes in, then the request is
failed with "No free slaves".
- -Q queue_timeout
- Queued requests should not stay on the queue indefinitely.
If a queued request cannot be processed within queue_timeout
(default 30) seconds of being placed on the queue, it is failed with a
"Queued request timed out" message. See the section
"QUEUEING REQUESTS" for more discussion.
- -O sock
- Listen on a notification socket for connections from
listeners. mimedefang-multiplexor can inform external
programs of state changes by sending messages over a notification socket.
The external programs connect to this socket and then listen for
notifications. See the section SOCKET SPECIFICATION for the format of
See the mimedefang-notify(7) man page for details of the notification
- -N map_sock
- Listen on a map socket for Sendmail SOCKETMAP
connections. As of Sendmail 8.13, you can define a Sendmail map type that
talks to a daemon over a socket. mimedefang-multiplexor implements
that protocol; consult the mimedefang-filter(5) man page for detils
(see the SOCKET MAPS section).
See the section SOCKET SPECIFICATION for the format of map_sock.
- -I backlog
- When mimedefang-multiplexor creates a listening
socket, it calculates the "backlog" argument to listen(2)
based on the maximum number of slaves. However, you can explicitly set
this backlog with the -I option. Setting the backlog to a high
value (around 30-50) may help on a very busy server. If you see mail log
messages saying "MXCommand: socket: Connection refused" during
busy periods, then that's an indication you need a higher listen backlog.
- -L interval
- Log the slave status every interval seconds. This
logs a line using syslog; the line looks like this:
Slave status: Stopped=s Idle=i Busy=b Killed=k Queued=q Msgs=m Activations=a
Here, "Stopped" is the number of non-running slaves,
"Idle" is the number of idle slaves, "Busy" is the
number of busy slaves, "Killed" is the number of killed slaves
yet to be reaped, "Queued" is the number of queued requests,
"Msgs" is the total number of messages processed since the
multiplexor began running, and "Activations" is the number of
times a Perl process has been started since the multiplexor began running.
If you supply an interval of 0 (which is the default), no periodic
status logging is performed. If you supply an interval of less than
5 seconds, it is silently reset to 5 seconds.
- -S facility
- Specifies the syslog facility for log messages. The default
is mail. See openlog(3) for a list of valid facilities. You
can use either the short name ("mail") or long name
("LOG_MAIL") for the facility name.
- Specifies that the multiplexor should create an embedded
Perl interpreter. This can improve performance dramatically. But see the
section "EMBEDDING PERL" for more information.
- -X n
- Specifies that the multiplexor should initiate a
"tick" request every n seconds. This causes your
filter_tick function (if defined) to be called. Note that you have
no control over which slave executes filter_tick. If all slaves are
busy when a tick occurs, that tick request is skipped and a warning
message is logged.
- -P n
- Specifies that the multiplexor should run n tick
requests in parallel. Each tick is run as often as specified with the
-X argument. (If you omit the -P option, then the
multiplexor behaves as if -P 1 had been specified.)
If you run parallel ticks, each tick is assigned an integer identifying its
"type". The type ranges from 0 to n-1. While there may be
as many as n tick requests running at a time, only one tick of each
type will be active at any time.
- -Y label
- Sets the tag used in the multiplexor's syslog messages to
label instead of mimedefang-multiplexor.
- Normally, mimedefang-multiplexor uses a umask of 027
when creating listening sockets. If you would like the sockets to be
readable and writeable by the group as well as the owner, supply the
-G option. This causes the umask to be 007 whenever UNIX-domain
sockets are created.
- -y n
- Limits the maximum number of concurrent recipok
checks to n on a per-domain basis. The value of n can range
from 0 (in which case no limit is applied) to maxSlaves, where
maxSlaves is the argument to the -x option. If n is
outside that range, it is ignored (and no limit is applied.)
command ultimately invokes the filter_recipient
function in your filter. If you are doing recipient verification against
servers that may be slow or unreliable, you can use the -y
limit the number of concurrent recipient verifications per domain. That way,
if one domain's server becomes very slow, it won't consume all available
slaves for recipient verification. Instead, its RCPT commands will be
tempfailed and there will be slaves available to handle RCPT commands for
options take a socket as an argument.
This socket can be specified as:
- A UNIX-domain socket
- A TCP socket bound to port portnum, but which
accepts connections only from the IPv4 loopback address (127.0.0.1).
- A TCP socket bound to port portnum which will accept
connections from any address. Use inet_any with caution!
- A TCP socket bound to port portnum listening on the
IPv6 loopback address.
- A TCP socket bound to port portnum listening on the
IPv6 wildcard address.
Normally, if all slaves are busy, any additional requests are failed
immediately. However, the -q
options allow you to queue
requests for a short amount of time. This facility is intended to gracefully
handle a temporary overload; most of the time, your queue should be empty.
checks the number of free slaves when a connection is
opened and fails the connection if there are no free slaves, the intent of the
queue is to allow SMTP transactions that are already underway to continue if
there is a slight overload. Any new connections will be failed if all slaves
are busy, but existing connections are allowed to continue. Queuing requests
may improve throughput on extremely busy servers.
Note that if you supply the -q
option to mimedefang
, then even new
connections are allowed to queue. This may improve throughput by keeping the
slave utilization higher.
option to mimedefang
can be used to reserve a specified
number of slaves for connections from the loopback address. Using the
option has the side-effect of permitting new connections from the
loopback address to queue.
Normally, when mimedefang-multiplexor
activates a slave, it forks and
. However, if the multiplexor was compiled with
embedded Perl support, and you supply the -E
command-line option, the
multiplexor works like this:
- It creates an embedded Perl interpreter, and sources
mimedefang.pl with a special command-line argument telling it to
read the filter, but not to enter the main loop.
- Each time a slave is activated, the multiplexor calls
fork() and runs the mimedefang.pl main loop. This invokes
filter_initialize and then runs the main loop.
On some platforms (for example, Red Hat Linux 7.3 with Perl 5.6.1), it is not
safe to destroy and recreate a Perl interpreter without causing a memory leak.
On those platforms, if you attempt to reread the filter file (by sending the
multiplexor a HUP signal or reread command), the filter will not
re-read, and a message will be logged to syslog. On those platforms, you must
kill and restart MIMEDefang if you change the filter file.
On most platforms, however, a filter reread is accomplished by destroying and
re-creating the embedded interpreter, re-sourcing mimedefang.pl
killing slaves as soon as they are idle.
With the -t
logs certain events to
a file. This file can be post-processed to gather statistics about the
multiplexor. You can use it to tune the number of slaves you run, adjust
timeouts, and so on.
Each line of the file looks like this:
YYYY/MM/DD:HH:MM:SS timestamp event key=val key=val...
Here, YYYY/MM/DD:HH:MM:SS is the local time of day. Timestamp is the number of
seconds since January 1, 1970. Event is the name of an event. The valid events
- A slave process has been started.
- A slave process has been killed.
- A dead slave process has been reaped. It is possible to
have a ReapSlave event without a previous KillSlave event if the slave
process terminated abnormally.
- A slave process has begun filtering an e-mail message.
- A slave process has finished filtering an e-mail message.
The possible keys in the key=value pairs are:
- The slave involved in the event. Every slave is identified
by a small integer.
- The total number of running slaves immediately after the
- The number of busy slaves (slaves which are processing an
e-mail message) immediately after the event happened.
- The reason for a StartSlave or KillSlave event. (Present
only for these events.)
- The number of e-mails processed by the slave. Present only
for an EndFilter event.
If you send the mimedefang-multiplexor
process a SIGHUP signal (kill -1
), it closes and reopens the statistics file. This is useful during
log file rotation.
If you send the mimedefang-multiplexor
process a SIGINT signal (kill -INT
), it terminates all active-but-idle slaves. Also, any
active-and-busy slaves terminate as soon as they finish filtering the current
message. This is useful to force a reread of the filter rules file without
stopping and restarting Sendmail.
If you send the mimedefang-multiplexor
process a SIGTERM signal (kill
), it terminates all slaves and exits immediately.
was written by Dianne Skoll
<email@example.com>. The mimedefang
home page is
mimedefang.pl(8), mimedefang-filter(5), mimedefang(8), mimedefang-protocol(7)