What Windows you running. As far as I know Win7 is the only Windows that sees more than 4 gigs of RAM. Vista and lower only see up to 3 gigs.
Any 32bit OS will only see 3GB of RAM. So if you are running 64bit Windows Vista, you can also have more than 3GB.
4GB is a good amount for gaming, though 6GB will give you a bit of an extra edge, though I don't think you will notice much if your video card and processor aren't up to it.
My next system will feature 8GB's. Going big