Keynote by Victor Lempitsky:
New Single-Stage Approaches to Instance Segmentation
Following on from successful workshops of recent years at ECCV, BMVC, and ICCV this workshop continues to showcase the challenges raised by and extend the state of the art in computer vision for plant phenotyping. The overriding goal is not only to identify key but unsolved problems and expose the current state-of-the-art, but also to broaden the field and the community. Effective plant phenotyping is urgently needed to support the sustainability of our planet and its inhabitants: having new vision scientists enter this field is more crucial than ever.
Background: Plant Phenotyping
Plant phenotyping is the identification of effects on plant structure and function (the phenotype) resulting from genotypic differences (i.e., differences in the genetic code) and the environmental conditions a plant has been exposed to. Knowledge of plant phenotypes is a key ingredient of the knowledge-based bioeconomy, which not only literally helps to feed the world, but is also essential for feed, fibre, and fuel production.
While collection of phenotypic traits was previously manual, image-based methods are now increasingly utilized in non-invasive plant phenotyping and the resulting images need to be analysed in a high throughput, robust, accurate, and reliable manner. The lack of robust automated, image-based phenotyping methods is widely recognised as the major obstacle to ensuring global food security (D. Houle et al. 2010. Phenomics: the next challenge. Nature Review Genetics, 11 (12), pp. 855-866). The problems raised differ from the usual tasks addressed by the computer vision community, due to the requirements posed by this challenging application scenario (M. Minervini et al. 2015. Image Analysis: The New Bottleneck in Plant Phenotyping. IEEE Signal Processing Magazine 32 (4), 126-131).
Plants are complex, self-changing systems whose complexity increases over time. Typical phenotyping problems include measuring the size, shape, 3D surface structure, architecture, growth and other structural traits of plants and their organs (leaves, fruit, roots etc.) at various stages in their lifecycle. Colour and spectral analyses are needed e.g. for investigations of plant health or photochemical status. Many scenarios require quantitative description of plant populations, where core problems include reliable detection and multilabel segmentation of many similar objects, or the reconstruction of specular, almost featureless, and overlapping surfaces. Quantitative description of the growth of these complex, deforming objects is vital, and requires suitable tracking, optical flow and/or scene flow estimation methods. Inherently, the tracked objects change their appearance over time. In some cases images may be acquired under controlled conditions, but they are increasingly likely to be taken in more challenging natural environments like greenhouses, or in the field. Automated image acquisition protocols are highly desirable, generating large numbers of images. Unfortunately, without automated and accurate computer vision to extract the phenotypes, a bottleneck is formed, hampering our understanding of plant biology and limiting our ability to provide the sustainable food supply needed by a rapidly expanding population inhabiting an increasingly unpredictable and hostile environment.
Scope of the Workshop
The key features of the workshop are:
- the unique requirements posed by plant phenotyping (many species imaged several times, high accuracy measurements with high reliability, high computational power typically available and real-time restrictions at relatively low frame rates, but high image resolution)
- dealing with a problem of great socio-economical impact that urgently requires robust computer vision solutions. Examination of the top 10 Emerging Technologies in 2012 according to the world economic forum, shows that those placed at positions 1, 2, 3, and 5 are directly addressed by plant phenotyping research (http://reports.weforum.org/global-agenda-council-2012/councils/emerging-technologies/ ).
- a format that includes dataset sharing, implementation and code sharing, and a practical challenge (permanent on CodaLab)
- a format inviting participants of diverse backgrounds, tailored to broadening participation from the broad CV community, and increasing the potential for synergy.
Some – not all – problems in plant phenotyping are comparable to medical applications. However while medicine focuses on the status of a single species, i.e. humans, plant phenotyping addresses hundreds of different plant species with hundreds to thousands genotypes (mutants) per species. It is often concerned with development over time rather than current status and this under a wide range of environmental conditions. Thus, even within a single application, a wide range of different conditions need to be addressed, to ascertain a robust image-based measurement of the phenotypic trait. Furthermore, computer vision and image analysis in medicine largely acts in an assistive capacity, rendering diagnosis of a single instance (i.e., a patient). In plant phenotyping, especially in high-throughput screening, computer vision will be used to compare several groups of plants, and as such the data size involved is larger, making automated solutions a necessity.