On Sun, Jan 11, 2004 at 10:11:35PM +0000, Paul Jakma wrote:
> > - itanium is there, but it's simply too costly for what we're doing
> > right now
>> Curious, is it as expensive as sparc64 and alpha? more expensive
> even? :)
It's about the same; which is very :)
> > We run the numbers, and tweak :) That extra 8Gb of RAM isnt for
> > direct IO exactly, it's for caching so that we never have to hit
> > disk - that's it's major function.
>> Ok, that makes sense. What kind of 'hit ratio' do you get on that
> 8GB<->6TB of cache? Presumably a subset of that 6TB is more popular
> than the rest.
Yep :) At any given time, only about 50% of the requests need to
go to the backend disks, when there's a very big release that
number goes up as a the common files get cached very quickly. An
awful lot of the content gets downloaded very little aswell.
> > The rest is there because even with PAE, it's still *much* better
> > to cache the files in VM than to pull it from disk. So at any given
> > time, we should be able to serve most requests with memory
> > operations, rather than disk.
>> Neat. Actually, apache and rsync use sendfile() dont they? I wonder
> if the kernel is clever enough to be able to avoid a bounce-buffer
> for sendfile? (though, still your 8GB above 4GB has to be
> bounce-buffered).
Actually, I turn sendfile() off, because it breaks TCP checksums
in IPv6 on our hardware, but we use mmap() instead - which does
avoid bounce-buffering :)
> > fast disks and 64bit scalable architechtures, we have 6Tb of IDE
> > disks at the back of it all, with x86 and clever caching in the
> > middle - but it turns out to be a lot better value for money than
> > alternatives :)
>> Yes. Value for money is something the 64bit platform vendors dont
> seem to appreciate. Though, how much would an IBM iSeries ppc64 box
> be? Are they any cheaper than HP or Sun? (And IBM actually support
> Linux on ppc64).
No idea!
> > In it's current configuration, ftp.heanet.ie has peaked at
> > 444Mbit/sec, and our current requests per second threshold is over
> > 1,300. In testing, we've happily saturated it's Gigabit ethernet
> > without complaint.
>> I still bet ya though you could improve greatly on that 444Mbit/s
> with decent 64bit hardware. Whether its worth the cost of the
> inflated prices... :)
That 444Mbit is limited by our users bandwidth, not ours :) We
could go up to 2Gig without any problem, but only for a few
thousand users. Our limits are more in terms of number of users,
we can't scale to tens of thousands of users, which is where
we need to be next :)
--
Colm MacCárthaigh Public Key: colm+pgp at stdlib.net
Maintained by the ILUG website team. The aim of Linux.ie is to
support and help commercial and private users of Linux in Ireland. You can
display ILUG news in your own webpages, read backend
information to find out how. Networking services kindly provided by HEAnet, server kindly donated by
Dell. Linux is a trademark of Linus Torvalds,
used with permission. No penguins were harmed in the production or maintenance
of this highly praised website. Looking for the
Indian Linux Users' Group? Try here. If you've read all this and aren't a lawyer: you should be!
