Crop yield estimation, an important managerial tool for vineyard managers, plays a crucial role in planning pre/post-harvest operations to achieve desired yield and improve efficiency of various field operations. Although various technological approaches have been developed in the past for automated yield estimation in wine grapes, challenges such as cost and complexity of the technology, need of higher technical expertise for their operation and insufficient accuracy have caused major concerns for growers to practically adopt such technologies. Lag phase is an important phenological stage in wine grape production and accurate prediction/detection of lag phase is vital for crop-estimation and overall vineyard management. The sampling completed in this period can help in obtaining accurate yield prediction due to predictable change in berry weight after lag-phase. In this study, a berry growth tracking system was developed and investigated to properly identify the lag phase in grapes, which will be implemented as a feature in an existing smartphone App being developed at Washington State University. The berries in the cluster were detected with the help of Mask-RCNN with Mean Average Precision value of 0.9. With the help of berry growth trend plot, the lag-phase for the wine grapes was estimated to start on 22 July. Since this model will use cellphone images for estimation, it will be simple and low-cost solution for offering user-friendly and convenient sensing system for lag phase detection in wine grapes which can be used for crop estimation in future.