This is just due to the heavy CPU power consuming H.264 decoding and the Quicktime wrapper. Same happens when using other editing softwares. You could transcode the Quicktime H.264 to a different video format which does not cosume that much CPU power - such as HDV or Cineform intermediates.