Healthy plants and saving resources: Condition monitoring of ornamental plants with microdisplays and neural networks to optimize fertilization

Dresden /

Conservation of resources and reliable yields are just as essential in plant cultivation as in other branches of agriculture. In order to optimize fertilization processes in ornamental plant cultivation in the future and thus protect the environment, researchers at the Fraunhofer Institute for Organic Electronics, Electron Beam and Plasma Technology FEP together with the Saxon State Office for Environment, Agriculture and Geology (LfULG) have been researching a technology for the early detection of nutrient deficiencies or surpluses in ornamental plants in the ZierSens project (financed by funds from the Free State of Saxony). The use of bidirectional OLED microdisplays in combination with neural networks should make it possible to optimize fertilization processes in the future. Initial results will be presented at the W3+ Fair in Jena from November 29 - 30, 2023, at booth no. C12.

Bidirectional microdisplay for condition monitoring of ornamental plants
© Saxon State Office for Environment, Agriculture and Geology
Bidirectional microdisplay for condition monitoring of ornamental plants

Ornamental plant cultivation is locally anchored in almost all German districts, mainly with small and medium-sized farms. In 2022, the production value of flowers and ornamental plants produced in Germany amounted to around 1.2 billion euros. Sales of cut flowers, for example, amounted to around 3.1 billion euros.1 In order for the plant cultivation sector in Germany to remain competitive, it is dependent on secure and high yields. Decisive factors for this include successful plant breeding, plant protection and optimal fertilization in order to conserve resources and soils.

In the ZierSens project, the Fraunhofer FEP conducted research into a technology that can detect deficits or nutrient surpluses in ornamental plants at an early stage. The institute has decades of experience in the development of application-specific OLED microdisplays for a wide variety of purposes. It is also possible to combine OLED light sources and photodiodes on a silicon chip. So-called bidirectional microdisplays are thus able to emit and simultaneously detect light in a wavelength-specific manner.

This technology enables the measurement and evaluation of small areas for surface inspection in various fields of application. In the project, it was specifically used to detect the condition of an ornamental plant with regard to possible under- or oversupply. With the help of a mechanism a leaf of an ornamental plant is pressed onto the microdisplay, which illuminates the leaf and can detect and evaluate the reflected light. In extensive plant cultivation experiments at the LfULG, plants were prepared in different nutritional states and images of leaves at different stages were taken. These images were used to train artificial neural networks to automatically identify leaf characteristics.

Compared to conventional measuring systems, which require a large number of measurements and use an averaged value within the measuring spot, bidirectional OLED microdisplays offer a location-resolved measurement. This allows several measured values to be generated at one measuring point, which are directly related to each other. This should enable more targeted fertilization and thus a more efficient use of resources, thereby reducing the environmental impact.

Judith Baumgarten, project manager at Fraunhofer FEP explains: "The project was based on the hypothesis that a lack or excess of nutrients can be detected at an early stage by the smallest changes in leaf coloration. By using our bidirectional microdisplay technology, we have achieved various effects that do not disprove this assumption, but also do not yet confirm it 100%. Nevertheless, we were able to further develop the measuring system and lay the foundations for the evaluation of colored camera images. We see great potential for the plant cultivation of the future with the support of smart technologies." Head of the Ornamental Horticulture Department and project manager at the LfULG, Dr. Sven Schubert, adds: "The use of modern technologies is essential for resource-conserving and sustainable horticulture. The data collected in the plant cultivation trials form the basis for the further development of sensor technology."

In addition, the scientists have trained neural networks to identify the plant species, leaf side and leaf age of the ornamental plants used. A further neural network was also developed to automatically segment the images into the areas of leaf area, leaf vein, leaf pattern and defects.

Research into the new technology presented the Fraunhofer FEP with a number of challenges. Firstly, adequate imaging of leaves with a special topology, such as primroses, was required, which necessitated further optimization of the sensor hardware. Secondly, models had to be created for evaluation, which would require a very large neural network due to the different characteristics of the plants. This would mean that calculations could no longer be performed on mobile devices, as is necessary in smart crop production. The Fraunhofer FEP is therefore pursuing a cascaded approach with several smaller networks that require significantly fewer resources.

The Fraunhofer FEP researchers see great potential in this proof-of-concept for plant breeding and ornamental plant cultivation and will present first project results and microdisplay technologies at the W3+ Fair, in Jena, from November 29 – 30, 2023, at booth no. C12.

1 Sources: Statista: and 

About the project

ZierSens – Use of an all-in-one CMOS image sensor with OLED microdisplay for the determination of plant constituents for the purpose of fertilization optimization using the example of ornamental plants
In cooperation with the Saxon State Office for Environment, Agriculture and Geology, funded by the Free State of Saxony.
Project duration: 09/19 – 08/23

Fraunhofer FEP at W3+ Fair and Convention 2023

W3+ Fair Fair & Convention
Sparkassen-Arena Jena, Germany
29 – 30 November 2023
booth no. C12

Further publications on the topic

  • Organic-on-silicon photonic platform for advanced imagers, microdisplays and sensors
    2023; Wartenberg, Philipp et. al
    Smart Photonic and Optoelectronic Integrated Circuits 2023
  • New Ultra Low-Power High Brightness Microdisplays Enabling Broad Applications
    2022; Wartenberg, Philipp et. al
    Society for Information Display. SID Symposium Digest of Technical Papers
  • Organic Microdisplay for Visual Feedback
    2021; Lenk, Simone et. al
    Optical Devices and Materials for Solar Energy and Solid-state lighting 2021