What are the most important factors for one to consider when starting a business in fashion industry if they are not from business back ground?

I suggest you find someone who understand the fashion industry well and you partner with him. Your role could be a partner / investor. But I don't recommend that you start such a business without having either the fashion experience or the business background. Nothing wrong with being a silent partner, but you need to choose your partner carefully. Also, you need to make a market test to make sure that there is a business opportunity in the fashion industry that you want to join. Maybe it's something you like or want to pursue, but there is not really an opportunity for it in the current market. 

Couple of books I suggest: 1. The E-myth revisted 2. The 10X rule.