It's totally worth it for me, anyway.

For simply adding little clips to the web, I've been able to shrink file sizes from 17Mb down to 750Kb. Mind you, there's a noticeable drop in quality when you do that, but I'm putting it to rather extreme parameters.

You can easily drop file sizes to about a quarter the size of the original file without a big qualty gap, and if your choice is slightly lower quality vs. not seeing the clips at all, I'd say go for slightly lower quality.