Hmmm, My frontend box has 512MB, boots off the network, and does not swap. My backend has 256MB, has a hard drive but does not start X, and does not swap. Maybe it's because I don't do any commercial detection or transcoding, but my boxes don't need swapfiles.