I think it depends on the extent of it. If it's just a fleeting "Wonder what he/she is up to these days..." or a random memory, that's not an issue. If you're obsessing and thinking about the person regularly, feeling like you've settled, constantly drawing comparisons, etc... then it is a problem. At least for me, I know that if any of the aforementioned are happening, it means that whoever is currently in the picture is not going to cut it for anything serious. If it feels right, I'm not thinking about anyone else.
If I'm thinking about a certain person and still feel like I'd drop everything if he came back into my life (while I'm hanging out with someone else,) then I know it's dead in the water.