World climate change and extreme weather conditions can generate uncertainties in crop production by increasing plant diseases and having significant impacts on crop yield loss. To enable precision agriculture technology in Florida’s citrus industry, a machine vision system was developed to identify common citrus production problems such as Huanglongbing (HLB), rust mite and wind scar. Objectives of this article were 1) to develop a simultaneous image acquisition system using multiple cameras on a customized conveyor that rotates citrus fruit in order to allow the imaging hardware to acquire the entire fruit surfaces, 2) to develop a machine vision algorithm with a deep learning technique utilizing a convolutional neural network to accurately inspect the visual characteristics of fruit surface and distinguish HLB-infected citrus from fruit with other common defects, and 3) to simulate real-time video processing utilizing a GPU for faster image processing. A real-time video processing with the state-of-the-art deep learning algorithm was developed and tested using uncompressed RGB video streams recorded from the developed hardware. Accuracy of various defect detection by deep convolutional neural network was 100, 89.7, 94.7, and 88.9 percent for healthy, HLB, rust mite and wind scar classes, respectively. The system can be used in citrus packing houses or developed on a portable conveyor that identifies severity of the diseases in particular locations and enables site-specific crop management in the field.