Crop yield prediction is an important aspect of farming and food-production. Therefore, estimating yield is important for crop breeders, seed-companies, and farmers to make informed real-time financial decisions. In-field soybean (Glycine max L.(Merr.)) yield estimation can be of great value to plant breeders as they screen thousands of plots to identify better yielding genotypes that ultimately will strengthen national food security. Existing soybean yield estimation tools, such as satellite imagery data-based ones, have various limitations such as a lack of real-time and in-field results output. Moreover, due to high-end computation and expertise requirements, plant breeders must rely on third party companies/individuals to perform such tasks. The objective of this study is to apply transfer-learning to train current Machine Learning (ML) frameworks, compare, and suggest the optimum architecture for small-end devices like smart-phones or embedded systems for real-time soybean pod counting and yield estimation. This project is aimed at developing an application to aid soybean breeders to estimate yield from in-field still images or real-time video data collected using smart phone sensors and drones flown at very low altitudes (AGL of ~6 feet). To enhance the dataset, generalize the trained model, and improve predictions, various data augmentation techniques were applied to the collected images. Toward this goal, we trained a variety of streamlined light-weight Deep Learning (DL) based object detection frameworks to find the best architecture by testing and evaluating the model using COCO-evaluation metrics. We used transfer-learning to train existing state-of-the-art DL models (YOLOv5-small, YOLOv3-tiny, EfficientDet-Lite, SSDLite Mobilenet v2) and to compare the performance to propose the best architecture, with the tradeoff between speed and accuracy for mobile and embedded systems. This study demonstrates the potential to significantly reduce the time and effort required to make soybean crop breeding decisions for cultivar development and yield estimation.