iMovie is designed for regular people, with regular needs. I personally rarely use it, using either Adobe Premier or Final Cut Pro (will move to Final Cut X when its out as that looks....amazing).
From what I could glean via Google - iMovie has a file size limit of 2GB. So perhaps you need to chop up the video into multiple < 2GB file size clips.