Some carriers provide longevity discounts or even diminish deductibles for their customers the longer they stay with them. When you switch carriers, you lose those. Also, not every insurance agent will quote the exact same coverages, so you may lose coverages and not know what you are giving up.
In addition, every time you quote your insurance, the carrier will do an insurance score inquiry. These will show up on your credit report. Having a high number of inquiries could have a negative impact. This is something you should try and avoid—if at all possible.