Aside from a few random acid attacks in Mong Kok and Causeway Bay a few years back, Hong Kong is an incredibly safe place—day and night. The police do a good job maintaining law and order, but there are still a few pickpockets about, especially in Tsim Sha Tsui and Mong Kok. So exercise the same caution you would in any large city: be aware and avoid carrying large amounts of cash or valuables with you, and you should have no problems.
Nearly all consumer dissatisfaction in Hong Kong stems from the electronics retailers in Tsim Sha Tsui. Get some reference prices online before buying, and always check the contents of boxed items before you leave the shop. Have a good idea of what you're looking for before you shop, and keep all receipts.
Distribute your cash, credit cards, IDs, and other valuables between a deep front pocket, an inside jacket or vest pocket, and a hidden money pouch. Don't reach for the money pouch once you're in public.