Predicting passenger flow is vital for the management, safety and smooth operation of any metro station. Such predictions are highly challenging as it depends on many parameters including travel pattern of the passengers. In this paper, we propose a highly efficient Long Short Term Memory Network [LSTM] which is a specialization of RNN to achieve this task. To do this prediction we employ the historical dataset from the metro containing the count, age and gender category of the passengers. Unlike earlier works, we also take into account the meteorological data of that time period and also the holiday information which includes the local events and public holidays. This accounts for the occasional spikes or fluctuations in the crowd patterns. Also the information about gender and age category of passengers is given emphasis and considered as an important parameter that affects the overall passenger count. Various configurations of the LSTM model are experimented by training the model repeatedly and the ones that yield the best result for this problem are evaluated and analyzed. The results obtained can be used to build an accurate and reliable predictive model to understand beforehand the amount of passenger crowd to expect