SSH Compression
I still haven't gotten around to testing Dropbear but I did find that SSH has built-in compression that can improve performance over slow links. So using the command:
ssh -X -C user@host
Creates an X tunnel with compression which is much more usable for me. It's still a bit slow but it is slow even on my LAN. I'm still planning to see if Dropbear can squeeze a little more performance out of my old netbook. Ultimately I don't think the Raspberry Pi will be much more powerful (if at all) than my netbook so every bit of performace optimization will help.