Abstract: In today’s world, Human machine interaction are used nowadays in many applications. Speech is one of the medium of interaction. Detection of emotion from speech is a main challenge. Communicating with emotions is more affective compared otherwise, since they can be expressed and identified in a better way through facial expressions, speech, gestures etc. If machines understand the emotional content they will start behaving in a friendlier way. Recognition of emotion is always a difficult problem, particularly if the recognition of emotion is done by using speech signal. Significant research has been done on emotion recognition using speech signal. The primary challenges are choosing the emotion recognition corpora i.e. database, identification of different features related to speech and selecting an appropriate choice of classification model. Speech Emotion problem is categorized as: 1) Feature extraction from speech: For this MFCC is used. We use 13 MFCC with 13 velocity and 13 acceleration component as features. 2) Feature classification: The features from MFCC are passed to CNN layer where the classification of above features is done. 3) Emotion detection: Depending on the output from CNN the corresponding emotion is detected i.e. happy, sad, angry, calm, fear. Keywords: CNN-Convolution neural network, MFCC-Mel Frequency Cepstral Coefficient, SAVEE, RAVDESS.