An effective fetal electrocardiogram (FECG) and ultrasound sonography (USG) signals with continues watching is a testing tool utilized by obstetricians to assess the maternal and embryo stage classification method are proposed utilizing a deep 2D convolutional neural network (CNN) which nowadays observe excellent presentation in the field of design identification. Because of the convolution and irregularity, a visible explanation of both Fetal Heart Rate or FHR and Uterine Contractions or UC signals utilizing usual instructions commonly obtained in remarkable individual inter-observer and intraobserver changeability. As a result, automated system depends on modern artificial intelligence (AI) innovation has nowadays been evolved to assist obstetricians in manufacturing targeted medical conclusions. The major object of this research was to make sure a novel, steady, strong, and effective model for maternal and embryo risk detection. Moreover, multiple CNNs is optimized through Genetic Algorithm (GA), overcomes the majority decision drawback in the traditional voting method. Improvement of the suggested classifier incorporate different deep studying methods such as transfer learning, GA initialization, multiple convolutional layers, hybrid optimization SGD with Adam and dropout with softmax were used in the experiments. And then, we differentiated our CNN classifier with four familiar CNN optimized models; such as SGD, Rmsprop, Adam and Adagrad. Depends on the experiment freely available database (CTU-UHB), we got good categorization presentation, after complete investigation, utilizing the proposed Optimized Residual Convolutional Learning Neural Network method with average crossvalidation (10 fold) of the Acc, TPR, TNR, PPV, NPV, HM, Kappa, AUC, PRAUC, Log Loss, DR, Prevalence, DP and BA respectively. Once the proposed Optimized Residual Convolutional Learning Neural Network model with 10 layers is achieved 96.24 % accuracy in average with successfully trained with the 8 different risk factors, the corresponding automated system can be used as a potent device to detect maternal and embryo risk state objectively and accurately.