It's possible that too many other people have set up routers in your vicinity on the same channels, and now you're having interference. Unfortunately, your situation is a side-effect of the unorthodox method that some companies have gone with to get "108" Mbit/s: that of hogging multiple channels. It is not only not polite (you're not the only one on your block who'd like to use wireless) but it also sets you up as a bigger target for interference and problems.
You can confirm the interference question by using a tool like NetStumbler.
I would re-examine your situation to see if you really need 108Mbit (considering that you probably rarely get that througput anyway). Certainly dropping to standard single-channel 54Mbit 802.11g isn't going to slow down your internet connection any. How much data-transfer do you actually do wirelessly computer-to-computer?
Your options are probably limited to:
1) dropping down to single-channel 54Mbit and picking a less-crowded channel (1/6/11 are your only real options).
2) Going with the wired RJ45 LAN adapter and bypassing wireless altogether.