Why were mobsters never just thrown in jail?

They were obviously very famous before thrown in jail and people knew about their murders but how come the police never just arrested them? Just wondering because i watch mob wives but can't help to think of it