If you're worried about your holster not being secure then you should probably use one.
Unlike Jeremy there, I have used a sidearm on many occasions. Not just in games where if your main gun is hit you have to sling it and move to a sidearm, but also because I use a high fps gun with a MED, inside of which I use a sidearm. They're also nice at places like sasco where you might use a long gun as a primary, but also want a short gun when you get into a CQB area without carrying a whole other gun.
Anyways, I use a serpa, so I never worry about my pistol falling out. For the people that use passive retention systems or universal velcro types of holsters, a pistol lanyard is great for keeping you from losing the gun.