Re: [ILUG] scaling Sendmail

From: kevin lyda (kevin at domain suberic.net)
Date: Fri 17 Sep 1999 - 19:16:36 IST


Nick Hilliard spoke thusly:
>sendmail isn't preforking, which means that every SMTP connection means
>loading up another instance of the program. This leads to bad latency
>problems if there's a memory load or a high disk I/O load on the machine in
>question.

while a process pool is nice, i seem to rember some paper debunking that.
in addition tight memory won't be solved by more processes, and i don't
think there's a disk hit for forking - sendmail's already in ram. the
only way for fork to cause problems is swap, and then you're just back
to not enough ram in your mail server (or not enough mail servers).

in iol's case i only see three mail servers, and one is just a backup.

>When these MTA's try to open up unique filenames, it's partly a hit-and-miss
>thing. You guess a particular filename (using, say, sendmail's paradigm of
>time-of-day + PID), and then if the file doesn't exist, then you're ok. But
>sometimes, you're going to take a performance hit on this (admittedly a
>small one).

as a guess they using standard libc tmpfile routines, but you can use
multiple queues with sendmail.

>More importantly, creating and opening up queue files every time you need to
>write out what you're got from the network is a pretty inefficient thing to
>do. If you run your spool in async mode, then you run the risk of really,

yes, but you save state if the system crashes.

>which give you the advantages of async speeds with sync-style metadata
>safety. Perhaps even better would be Brad Knowle's idea of having a bucket
>of available pre-made spool files.

fs performance is a separate issue. but regardless, files will need to
be created at some time, and i'm not sure a truncate call is any more
efficient then a create/unlink.

>So now, you've got a queue full of data files -- let's say 100,000 of them,
>if a system goes down for a few hours. Filesystem directory structures on

you can create separate queues *and* you can separate the control and
data files.

>In addition, it's difficult to control sendmail when the machine is
>stressed. Ok, you can tell it not to do anything if the load goes above
>whatever, but often by that stage, the machine is pretty crippled anyway,
>and then you start losing out SMTP connections and relying on MX backup
>systems.

and each queue can have it's own policy...

>Also, the controls for delivering mail to remote systems are poor -- you
>can't specify how many concurrent queue run or any other sort of processes
>you want to max out at; you can't specify the max amount of bandwidth you
>want to use, and so forth. It's all a little inflexible.

somehow aol manages to keep sendmail going while handling millions
of messages per hour (at a guess). i suggest you look at your tool
more carefully.

>Anyway, I could go on, but you get the idea. All of the standard mailers
>are deficient in lots of different but infuriating ways. It would have been
>nice to think that starting from a clean slate, that Wietse Venema might
>have created Postfix with a really nice queueing system, but alas....

i really have my doubts on this. i'm not sure ilug is the place to cover
this though. i'll have a read through my bat book this weekend and
get some more detail, but i really think you're not fully utilising
what you've got.

kevin



This archive was generated by hypermail 2.1.6 : Thu 06 Feb 2003 - 13:04:34 GMT