[ILUG] nightly time-savers

[Conor Daly]

On Tue, Jul 03, 2001 at 04:58:39PM +0100 or so it is rumoured hereabouts, 
Brendan Kehoe thought:
> I'm looking into setting up my home server to dial up around 5am and do all of
> these steps:
> 
> * download email for both users (fetchmail cron job?)
> * download articles for a specific set of newsgroups (inn?)
> * download 4 pages deep a list of URLs, with graphics but thru junkbuster
> (wget + junkbuster + squid?)
> 
> Thus when we get up in the morning, we can get our mail from that server, read
> news also from it, and also do our morning website ritual, all without
> actually dialing out.  Thus I can then make the sendmail queue only get run
> every 4 hours, say, to further reduce our phonebills.  (Can you tell we're
> amongst those cut from Surf-No-Limits? :) )
> 
> Have any of you folks done something similar?  I'm happy to put together a
> page of suggestions for the linux.ie site as a result.  (I'm digging around
> and asking friends, but haven't found others who've set something up like
> this.)
> 

Yup, done all that (well, apart from the wget stuff).  I have the following setup:

**** internet firewall / router / dial-on-demand

This box sits and listens for traffic on the internal net destined for the
Internet and then brings up the internet connection.  Once that's up and
the firewall is in place, it telnet's port 67398 on the server to tell it
that the internet is up.  It then watches the internet link and takes it
down once internet traffic dies down.  When it brings down the internet
link, it telnet's the server on port 67399 to tell it that the internet is
down.

**** mail / news / web / print / disc server +local and caching DNS

This box looks after all these services.  mail is handled by fetchmail
with a /root/.fetchmailrc containing lines like

poll <pop.server> proto pop3 user <user.name> with pass <password> is 
<localusername> here forcecr smtpaddress localhost

This gets run as root to fetch the mail.  Outgoing mail is handled by
'sendmail -q'.

News is handled by leafnode (an intelligent local news server.  It gets
the list of groups from the specified server and puts a "placeholder" post
in each on the local server.  When you read the placeholder post, this is
a signal to leafnode to download *this* group.  leafnode then keeps this
group up to date on the local server until you stop reading it for about 1
week.  The upshot of this is that your local news server automatically
keeps those groups that are of interest to the household up to date and
ignores all the others).

Now, how does all this happen?  inetd is yer only man!

[cdaly]$tail /etc/inetd.conf 
ppp-up    stream  tcp     nowait.400      root /etc/ppp/ppp.faenor ppp.faenor up
ppp-down  stream  tcp     nowait.400      root /etc/ppp/ppp.faenor ppp.faenor down

[cdaly]$ grep ppp /etc/services
ppp-up		67398/tcp		#Faenor bringing Internet link up
ppp-down	67399/tcp		#Faenor bringing Internet link down

When the server (hobbiton) receives a telnet connection on 67398,
/etc/ppp/ppp.faenor up is executed.  This does a few tasks like noting the
connection up time in a log but primarily starting the check-mail script.

When hobbiton receives a telnet connection on 67399, 
/etc/ppp/ppp.faenor down is executed.  This notes the connection down time
in the same log and stops the check-mail script (or rather, it asks the
check-mail script to stop).

check-mail is the script that does the real work.  You've realised by now
that check-mail gets executed *every* time the internet connection is up.
check-mail now does the following:

run fetchmail -f /root/.fetchmailrc to get the family's email
run sendmail -q to send any queued email
run fetchnews to do leafnode's update
sleep for 2 minutes and then do it all again until it finds the flag that
says the internet connection is now down whereupon it exits.

So everything is automatic, even down to the cron job that pings some
outside IP address a few times per day to cause a connect to happen
(remember the firewall is listening for traffic destined for the outside).

The telnet / inetd stuff is only necessary since the server and the
firewall are different physical boxes.  Otherwise, the firewall can
directly start the internet up script.  I decided against allowing the
firewall to do rsh stuff on the server but that may be unnecessary
paranoia.  Of course, much of the stuff I did here wasin the way of a
learning exercise and may be needlessly complex for what I need but it
"works for me" (TM).

If you'd like to have a chat on the subject, call me at work at +353 1
8064276

Conor
-- 
Conor Daly <conor.daly at oceanfree.net>

Domestic Sysadmin :-)
---------------------
Faenor.cod.ie
 12:05am  up 19 days, 22 min,  0 users,  load average: 0.08, 0.02, 0.01
Hobbiton.cod.ie
 12:13am  up 19 days, 31 min,  2 users,  load average: 0.00, 0.05, 0.02