Yepper. Both harmless and a sign of a healthy environment. Fish usually eat them, but shrimp don't.
Not that you want to go go this much into depth, heh, but here's a good link: http://www.harpercollege.edu/ls-hs/bio/dept/guide/gallery/aquatic_worms/thumb.html