"Whether or not you're happy about the iOS-centric changes coming with the release of Mac OS X Lion, you can still benefit from organizing your current OS X dock a little bit more like your homescreen."
I end up filling my Mac dock up pretty quickly and run into the same problem. I usually do what the author had been doing, which is putting the programs I use most often on the left side and the ones I use infrequently on the right. If I find I haven't been using a program or app all that much, it ends up being taken off the dock and I just access it via the finder if I need it again. This article however, will show you how to put your programs into folders just like on your iOS device, if you feel the need to really clean up your dock. I think that is taking it a bit far but whatever, life is full of options isn't it?*
iPhone 4, 1.6 GHz Core 2 Duo MacBook Air, Apple TV, 2.66GHz Intel C2D Mac Mini, 1TB TC, Airport Extreme,several iPods and an iPad, 32GB Wi-Fi version. Follow me on Twitter.
Or you could just use Overflow. Works like a charm, and you don't have multiple folders on your dock. Or, you could use dragthing. I use overflow on the dock, though I usually just launch programs from the keyboard with launchbar.