Site-specific weed management refers to the application of customised control treatments, mainly herbicide, only where weeds are located within the crop-field. In this context, the TOAS project is being developed under the financial support of the European Commission with the main objective of generating georeferenced weed infestation maps of certain herbaceous (corn and sunflower) and permanent woody crops (poplar and olive orchards) by using aerial images collected by an unmanned aerial vehicle (UAV). This article introduces TOAS project and describes the whole research process developed and the main results attained to date. The main tasks are focused on: 1) configuration and use of the UAV and sensors for image acquisition, 2) evaluation of the specifications (sensor type, imagery characteristics, crop-weed phenological stage) required for each type of crop, and 3) development of automatic and robust image analysis procedures for weed mapping and crop assessment by using the captured remote images in order to optimize herbicide applications or other crop-weed management operations. Advances of the TOAS project are regularly updated in http://toasproject.wordpress.com.