On Sun, 11 Jan 2004, Liam Bedford wrote:
> Is there an option missing? I think the G5 machines are supported
> in linux now.. possibly even in 64bit mode properly.
That depends on whether the G5 machine has an IO-MMU (ditto for
opteron). A 64bit machine with tonnes of RAM without an IO-MMU has
the same problems as a 32bit PAE machine when it comes to IO. It
seems that at least the IBM iSeries has IO-MMU - no idea about other
Yep. not good either. But that affects translation of virtual
addresses -> physical. Not immediately relevant to IO.
> (maybe that's what you mean by all that technical mumbo jumbo :))
Bounce buffers: PCI has a 4GB address space (well PCI can have a
64bit address space using DAC, but that eats one PCI bus cycle (DAC =
dual address cycle) and AFAIK, isnt useable on 32bit PCs, only on
64bit machines). On PC's (which dont have IO-MMUs), the PCI bus
address space maps directly to physical address space. So, if you
have a process (eg ftpd, httpd) which has data located somewhere
above 4GB in physical ram then you have a problem if you want to tell
a PCI device to DMA that data - as the device has no way to address
that data. Instead, the kernel must first make a temporary copy of
the data to somewhere below 4GB. Ie buffers are kept some where the
device can access and data is bounced via this buffer to/from
process<->device. Having to copy data before/after IO really sucks
Server class 64bit machines have dealt with this problem using
IO-MMUs. Ie ability for the system bus controllers to map bus address
space to arbitrary regions of physical address space. Eg, if data is
located at 6GB physical memory, the kernel programmes the host
controller to map, eg, PCI address 2GB to 6GB physical (for some
length). When the PCI devices DMAs to/from 2GB the controller fetches
to/from 6GB - sort of like NAT for memory :). More sophisticated
IO-MMUs can generally do very fine grained mapping, allowing one
block of PCI address space to be mapped to a whole bunch of
discontigious physical addresses - called scatter-gather. (though,
most PCI devices tend to support this themselves).
Ironically, most /desktop/ PCs actually /do/ have an IO-MMU where
quite a few server PCs do not - the GART is a very basic IO-MMU,
intended for AGP cards to map textures in system RAM. However, the
GART window applies to all of the PCI address space, hence it can be
used to remap memory for any PCI device. It seems that x86-64 uses a
slightly extended version of GART to handle io-mapping (bit icky).
Alpha and USparc machines /definitely/ have good IO-MMU support. IBM
G5 iSeries and Itanic appear to have it too, but no idea how good it
NB: i'm probably completely wrong as usual.
Paul Jakma paul at clubi.iepaul at jakma.org Key ID: 64A2FF6A
warning: do not ever send email to spam at dishone.st
Never invest your money in anything that eats or needs repainting.
-- Billy Rose
Maintained by the ILUG website team. The aim of Linux.ie is to
support and help commercial and private users of Linux in Ireland. You can
display ILUG news in your own webpages, read backend
information to find out how. Networking services kindly provided by HEAnet, server kindly donated by
Dell. Linux is a trademark of Linus Torvalds,
used with permission. No penguins were harmed in the production or maintenance
of this highly praised website. Looking for the
Indian Linux Users' Group? Try here. If you've read all this and aren't a lawyer: you should be!