Abstract—In recent years, it is in public to use the surveillance cameras for continuous monitoring of public and private spaces because of increasing crime. Most current surveillance systems need a human operator to constantly watch them and are ineffective as the amount of video data is increasing day by day. Surveillance cameras will be more useful tools if instead of passively recording; they generate warnings or real-time actions when unusual activity is detected. But recognizing and classifying human activity as normal or abnormal from a live video stream is a stimulating job in the pitch of CPU vision. There is a need for a smart surveillance system for the automatic identification of abnormal behaviour of humans for a specific-scene. Presentpaperstretches an overview of different machine learning methods used in recent years to develop such a model. It also gives an exposure to the recent works in the field of anomaly detection in surveillance video and its applications.