I agree, it can be very rewarding. I enjoy selecting the parts on my own for a truly customized machine. However, I've run into an issue recently that has changed my PC-building habit.
I moved to a rural area, far from the nearest Fry's Electronics or Microcenter. There are no stores nearby with parts, except some very basic things at a Staples an hour away. So I merrily ordered online at Newegg, etc. When the parts came, I assembled everything as usual, went to to turn it on and had problems. One of the components I ordered had a defect and it took a while to figure out which one (motherboard). I now had to package it up and ship it back. Of course it was a weekend so I had to wait until Monday. And in the meantime, I had disassembled my PC because I was re-using some of the parts. A week later, I got it mostly together only to learn that one of the memory sticks was also faulty. I was able to run the PC with one memory stick (albeit slowly), but again, I had to repackage, return, wait and then crack the case. Now all of this wouldn't have been that big a deal if I could have returned it locally.
So now what I do is drive 3 hours to my local big city, buy the parts and pay someone at Microcenter or similar to assemble and do the basic OS install. If a part fails, they can pull another off the shelf. At least I know when I get it home that day, it will work. The time and money are worth the peace of mind, though I do miss out on some of the fun.