Reservoir Computing with Plants #

The notion of plant intelligence was briefly discussed in Plants as Intelligent and Information Processing Organisms. While this is not the subject of this dissertation, as outlined in secction 1.4, its discussion serves as an interesting general introduction to information processing. To research the computational properties of plants, we propose to use physical reservoir computing (PRC).

In order to study the applicability of the PRC framework to plants, a machine learning foundation was established in Introduction to Machine Learning and Reservoir Computing, and a plant ecophysiology and phenotyping foundation was established in Introduction to Plant Ecophysiology, Phenotyping and Phenomics. These chapters are essential building blocks to achieve a proper experimental design founded in existing research in both domains.

In this chapter, both domains meet, and we discuss how the PRC framework can be mapped to plants based on literature. Moreover, we also discuss the general setup of experiments and their goal.

Mapping the Physical Reservoir Framework to Plants #

We already mentioned that plants are good examples of organisms that perform morphological computations in Morphological Computation. For instance, due to the sessile nature of plants, they need to rely on their ability to grow their body based on environmental queues to explore new regions in search of food. Other organisms such as slime mold have also adopted intelligent strategies of dealing with these limitations (Citation: Mayne, Adamatzky& al., 2015) Mayne, R., Adamatzky, A. & Jones, J. (2015). On the role of the plasmodial cytoskeleton in facilitating intelligent behavior in slime mold Physarum polycephalum. Communicative & Integrative Biology, 8(4). e1059007. https://doi.org/10.1080/19420889.2015.1059007 . However, there is no general framework to understand or quantify morphological computation. Consequently, we need to look into related fields for inspiration. Cellular automata are good candidates (Citation: Adamatzky, 1995) Adamatzky, A. (1995). Identification Of Cellular Automata. CRC Press. https://doi.org/10.1201/9781315274355 , but we preferred to work with reservoir computing because it can be leveraged for more general-purpose problem-solving. As mentioned in Introduction to Machine Learning and Reservoir Computing, reservoir computing is a very general framework in which the dynamics of the reservoir are exploited for computation. Initially, recurrent neural network (RNN)-based reservoirs were used, but in Physical Reservoir Computing we discussed the transfer to physical systems. Yet, the mapping from the RNN to the physical substrate was not discussed in detail.

Therefore, we will discuss this mapping using three examples in figure 4.1. The baseline RNN-form is depicted in (a). Such reservoirs are fully observable. Generally, this is unfeasible for physical implementations and not necessary in practice for PRC as long as the four main requirements listed in Physical Reservoir Computing are met. Nonetheless, these requirements restrict the generalising ability of PRC for morphological computing. Indeed, in case of plants and mold, physical bodies change over time, resulting in violations of the PRC requirements. Yet, due to the lack of a better theoretical foundation, one can limit the time frame and/or impose other constraints such that the requirements are approximately met.

The example physical substrates we consider in figure 4.1 are (b) a silicone arm, (c) a tensegrity robot and (d) a plant. We go over each of the cases and discuss how these criteria are met, with special emphasis on the nonlinear and fading memory properties. Each of these systems can be in an infinite number of transient states, so the high-dimensionality requirement is clearly met. Moreover, the separation property is also fulfilled since none of these systems is generally at the boundary of chaos. As such, slightly different inputs will generate similar signals inside the reservoir.

Figure 4.1b depicts a PRC implementation consisting of a silicone arm that is connected to a motor and submerged in a tank of water, as published by Citation: Nakajima, Hauser& al. (2015) Nakajima, K., Hauser, H., Li, T. & Pfeifer, R. (2015). Information processing via physical soft body. Scientific Reports, 5. 10487. https://doi.org/10.1038/srep10487 . The reservoir is formed by the silicone-water interaction. To read out the reservoir state, they used ten bend sensors that characterised the deformation of the arm. Weighting functions were trained to several nonlinear auto-regressive moving average (NARMA) benchmark tasks. The fading memory requirement is clearly met because the arm always returns to the same resting position for a zero-input due to friction and gravitational forces. Moreover, several physical attributes such as the viscosity of the liquid and arm length determine the memory of the system. The reservoir is built of silicone, which is a highly nonlinear material (Citation: Nakajima, Hauser& al., 2013) Nakajima, K., Hauser, H., Kang, R., Guglielmino, E., Caldwell, D. & Pfeifer, R. (2013). A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm. Frontiers in Computational Neuroscience, 7. https://doi.org/10.3389/fncom.2013.00091 . For instance, we can apply a force that causes part of the material to contract. Doubling this force will not result in doubling this contraction. Due to the nonlinear properties, the contraction will be less and diminish as higher forces are applied.

Another example is depicted in figure 4.1c, based on the work of Citation: Caluwaerts, D’Haene& al. (2013) Caluwaerts, K., D'Haene, M., Verstraeten, D. & Schrauwen, B. (2013). Locomotion without a brain: physical reservoir computing in tensegrity structures. Artificial Life, 19(1). 35–66. https://doi.org/10.1162/ARTL_a_00080 . Here, the reservoir is composed of beams and active and passive springs. Beams are the thick black lines, active springs the dotted lines and passive springs the thin lines. The beams have a fixed length and cannot deform, while the springs can contract and/or extend, either actively or passively. Force sensors in a (sub)set of passive springs measure the state of the reservoir. The nonlinearity in this system arises from the fixed lengths of the beams and spring properties if they have non-zero equilibrium lengths. This last requirement is always met in physical implementations. Memory is also embedded in the structure due to the interconnected nature, and these are also damped due to friction, resulting in fading memory.

In figure 4.1d, we arrive at the most relevant physical reservoir in this work: a plant. It was already mentioned in Are Plants Intelligent? that emergent intelligence could be attributed to plants. One of the requirements of intelligence is the ability to process sensory data in non-trivial ways. In essence, such ability is the same as the ability to compute. As is evident from the previous examples, PRC maps to a wide range of unconventional substrates. As a result, we hypothesise that PRC is a good candidate framework to study the computational properties of plants.

Plants are highly nonlinear. Increasing the air temperature by 3 °C from 27 °C to 30 °C can already result in flower drop in tomato, while a temperature decrees of 3 °C from 27 °C to 24 °C has no adverse effects (Citation: Ozores-Hampton, Kiran& al., 2012) Ozores-Hampton, M., Kiran, F. & Mcavoy, G. (2012). Blossom Drop, Reduced Fruit Set, and Post-Pollination Disorders in Tomato. University of Florida, Institute of Food and Agricultural Sciences, Electronic Data Information Source. . This is a clear example of nonlinear behaviour. Another example of nonlinear behaviour is the oxygen yield of plants in figure 4.2. Initially, the yield increases linearly for low light intensities (green area). However, at higher intensities, it is no longer the case (white area). In conclusion, plants respond nonlinearly to environmental changes. For nearly all environmental factors affecting plants, there are clear plant-specific ranges in which they grow well. However, beyond a certain point, they are subject to stress and can eventually die. Clearly, the fading memory requirement is not met if the plant dies. However, true fading memory is usually not met in a strict sense in physical implementations. Indeed, as the examples above also illustrate, prior conditions can have a lasting effect if they are outside of the allowed interval. Similar behaviour can be found in plants. Nonetheless, partial fading memory can be attributed to plants. For instance, recently, a vaccine was developed for the late blight disease in potato (Citation: Najdabbasi, Mirmajlessi& al., 2021) Najdabbasi, N., Mirmajlessi, S., Dewitte, K., Ameye, M., Mänd, M., Audenaert, K., Landschoot, S. & Haesaert, G. (2021). Green Leaf Volatile Confers Management of Late Blight Disease: A Green Vaccination in Potato. Journal of Fungi, 7(4). 312. https://doi.org/10.3390/jof7040312 . This vaccine offers good initial protection but fades over time.

In Are Plants Intelligent? we already reported on findings that identify memory and learning behaviour in plants. Yet, there is so far no evidence of properties related to fading memory reported in literature. However, it is well established that plants are nonlinear dynamic systems. In modelling plant behaviour, modellers often rely on differential equations to describe relationships within the plant, such as water and nutrient transport in a plant’s vascular system (Citation: De Swaef, Pieters& al., 2022) De Swaef, T., Pieters, O., Appeltans, S., Borra-Serrano, I., Coudron, W., Couvreur, V., Garré, S., Lootens, P., Nicolaï, B., Pols, L., Saint Cast, C., Šalagovič, J., Van Haeverbeke, M., Stock, M. & wyffels, F. (2022). On the pivotal role of water potential to model plant physiological processes. in silico Plants. diab038. https://doi.org/10.1093/insilicoplants/diab038 .

Plants are non-stationary systems: their properties in terms of memory and nonlinearity change over time due to their continued development (Citation: Hilty, Muller& al., 2021) Hilty, J., Muller, B., Pantin, F. & Leuzinger, S. (2021). Plant growth: the What, the How, and the Why. New Phytologist, 232(1). 25–41. https://doi.org/10.1111/nph.17610 . For instance, a leaf can be shed when it is too old or damaged and new plant organs develop. Yet, we suspect that if experiments focus on a sufficiently short timescale, they may be assumed as stationary (reservoirs). While the exact environmental conditions (e.g., 11 °C in place of 12 °C) have diminishing influence the longer ago they occurred, the range can have a lasting influence. For instance, if soy is not subject to a strong nighttime temperature drop, then the plant height increases much faster than when there is a strong temperature drop (Citation: Sionit, Strain& al., 1987) Sionit, N., Strain, B. & Flint, E. (1987). Interaction of temperature and co2 enrichment on soybean: growth and dry matter partitioning. Canadian Journal of Plant Science, 67(1). 59–67. https://doi.org/10.4141/cjps87-007 .

What is missing in the classic description of echo state network (ESN) is a notion of time. In software, the reservoir is stationary. But for physical systems, this is rarely the case. In the first example of PRC by Citation: Fernando & Sojakka (2003) Fernando, C. & Sojakka, S. (2003). Pattern Recognition in a Bucket. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39432-7_63 , the reservoir consisted of a bucket of water. If left untouched, this water will evaporate, making computation impossible. Even before this point, the dynamics would have changed due to a decreasing water level. Clearly, the timescale at which computations are performed and the variability of the reservoir has to be considered. In this case, the changes in the reservoir can be neglected with respect to the timescale at which tasks are performed. Another example of PRC that violates the fading memory requirement can be found in compliant robotics. A series of incorrect actuations of a gait in a quadruped robot can cause the robot to flip. Most quadrupeds cannot recover and are thus stuck. Manual intervention is thus needed to restart the system. Even photonic systems suffer from non-stationary effects due to bias voltage and temperature fluctuations (Citation: Reddy Chittamuru, Thakkar& al., 2017) Reddy Chittamuru, S., Thakkar, I. & Pasricha, S. (2017). Analyzing voltage bias and temperature induced aging effects in photonic interconnects for manycore computing. IEEE. https://doi.org/10.1109/SLIP.2017.7974906 . These fluctuations cause physical properties such as wavelength and output power to change in time. Generally, this is an undesired effect.

In summary, the applicability of PRC on plants needs to be investigated, as well as the (non-)stationary nature of plants. In literature, there are already reports of biological media such as (in vivo) brain cells (Citation: Dominey, 2013 Dominey, P. (2013). Recurrent temporal networks and language acquisition—from corticostriatal neurophysiology to reservoir computing. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00500 ; Citation: Enel, Procyk& al., 2016 Enel, P., Procyk, E., Quilodran, R. & Dominey, P. (2016). Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex. PLOS Computational Biology, 12(6). e1004967. https://doi.org/10.1371/journal.pcbi.1004967 ; Citation: Obien, Deligkaris& al., 2015) Obien, M., Deligkaris, K., Bullmann, T., Bakkum, D. & Frey, U. (2015). Revealing neuronal function through microelectrode array recordings. Frontiers in Neuroscience, 8. 423. https://doi.org/10.3389/fnins.2014.00423 and bacteria cultures of Escherichia Coli (Citation: Jones, Stekel& al., 2007) Jones, B., Stekel, D., Rowe, J. & Fernando, C. (2007). Is there a Liquid State Machine in the Bacterium Escherichia Coli?. https://doi.org/10.1109/ALIFE.2007.367795 that can be used for PRC. These are also non-stationary, and previous events can also have lasting effects. They illustrate that these issues can be overcome, at least to a certain extent.

Not only the time frame in which the dynamics of the reservoir change is important but also the sampling time (time between measurements). A plant is a living organism that continuously senses the environment and optimises its physiology. Observing this variable physiology is, however, only possible in discrete time. Measurements are performed at a certain rate, and any variation that occurs between measurement time points is not captured. As a result, it is important to sample sufficiently fast. Table 3.1 contains some information on relevant timescales for imaging sensors. Table 4.1 is similar, but for other non-imaging sensors. The continuous nature of plants is thus fundamentally different from the discrete-time steps employed in RNN-based ESN reservoirs.

These tables give an indication of the relevant timescale but are only observed after the data was acquired and the sample frequencies were not optimised with respect to the plant’s dynamics. It is also worth mentioning that the employed sensor accuracy and the accuracy of the acquisition system are also key. Changes associated with higher frequencies are inherently smaller due to the time lag caused by underlying physiological processes.

Considering that plants are not designed for computation, it is unlikely that plants can serve as efficient computational devices. Still, plant reservoir computing may have a substantial impact on the plant science community. It can provide a holistic approach to plant modelling, improve plant sensing and bring new insights into plant physiology. This computational framework offers a general basis that can be used to study plant behaviour, where a plant’s state is the result of its information processing of all incoming environmental and internal signals. Such a view can be applied to plant functioning and development in general instead of focusing on certain plant processes as the result of a specific treatment.

Experimental Design #

Evaluating the applicability of PRC to plants requires an adequate experimental design to quantify the nonlinear processing and memory properties of plants. The core of the experimental design is depicted in Figure 4.3. A single plant is placed in a growth chamber and subject to variable light, temperature and relative humidity conditions. These are abiotic factors, which are preferred in this exploratory work because of the ability to replicate the experiment more easily than biotic factors. Moreover, we selected these three factors because they are easy to control in growth chambers and are the main driving forces for short-term physiological responses.

To eliminate disturbances from the observer and environment, we have to automate experiments. This is also necessary to achieve a high temporal resolution.

Details on the Experimental Set-up #

All experiments are conducted in a growth chamber at ILVO (Melle, Belgium). The chamber has a usable volume of 1.45×0.77×1.45m (height×depth×width) (BIOCLIM 1600 US, Weiss Technik, Reiskirchen, Germany). Light intensity, temperature and relative humidity were controlled by a microcontroller board placed outside the growth chamber. The temperature and relative humidity of the growth chamber were controlled using analogue signals and varied randomly between 11 and 33°C, and 31% and 75% respectively.

Because the built-in lights of the growth chamber cannot create spatial variation, a custom-built frame of 1.00×0.70×1.10m (height×depth×width) was inserted into the chamber. Lamps are mounted on this frame and can be turned on and off programmatically.

One key aspect in the experimental design is the ability to distinguish between computation arising due to the dynamics of the environment and those stemming from the plant. If environmental dynamics cannot be eliminated due to the sensor choice, a control experiment should be performed with the same input conditions but without a plant. The performance of both experiments can then be compared to assess the amount of computing power originating from the environment.

The choice of sensor technology employed to monitor the plant’s state is also important. Sensors should have limited to no influence on plant responses. However, as mentioned in Trait Measurement Technologies, different technologies have their own advantages and disadvantages. The exact technologies used are highlighted in Employed Sensor Technologies.

The base sampling period of our experiments is set to 1s, based on literature. Technical limitations of the gas exchange device limit its sample period to 3 s which is later interpolated to match the sample rate of the other sensors. This base period should be sufficiently fast to detect relevant changes in the observable traits (see also table 3.1 and table 3.2).

Plant Species Selection #

When performing experiments, strawberry (Fragaria×ananassa) is the main model species. An additional species of interest is tomato (Solanum lycopersicum).

In Plants as Computational Resource, we discussed several long-term applications of PRC with plants. Most of these are situated in greenhouses since greenhouses offer more opportunities for intervention and optimisation without needing additional infrastructure. Strawberry is selected because it is a commercially interesting crop in Belgian greenhouses (Citation: Lieten, 2013) Lieten, P. (2013). Advances in Strawberry Substrate Culture during the Last Twenty Years in the Netherlands and Belgium. International Journal of Fruit Science, 13(1-2). 84–90. https://doi.org/10.1080/15538362.2012.697024 . Moreover, strawberry is not a seasonal crop. After the fruit and seed production, it does not wilt and die. As a result, a set of plants can be continuously available in the mature growth stage in the greenhouse.

Tomato is also often grown in greenhouses (Citation: Hemming, Zwart& al., 2020) Hemming, S., Zwart, F., Elings, A., Petropoulou, A. & Righini, I. (2020). Cherry Tomato Production in Intelligent Greenhouses—Sensors and AI for Control of Climate, Irrigation, Crop Yield, and Quality. Sensors, 20(22). 6430. https://doi.org/10.3390/s20226430 . This plant is a seasonal crop, characterised by a sigmoid growth curve (Citation: Koning, 1994) Koning, A. (1994). Development and dry matter distribution in glasshouse tomato: a quantitative approach. Wageningen University and Research. (see e.g., figure 2.1) and offers an interesting alternative species that is more actively growing than strawberry in the experiments.

Experiments were conducted in a growth chamber and plants were grown in the greenhouse at ILVO. Thus, selecting typical greenhouse crops is more relevant since the results will be more representative for future applications. Moreover, shock due to the difference in growing conditions between the greenhouse and growth chamber was limited compared to moving plants from the field to the growth chamber (Citation: Poorter, Fiorani& al., 2016) Poorter, H., Fiorani, F., Pieruschka, R., Wojciechowski, T., Putten, W., Kleyer, M., Schurr, U. & Postma, J. (2016). Pampered inside, pestered outside? Differences and similarities between plants growing in controlled conditions and in the field. New Phytologist, 212(4). 838–855. https://doi.org/10.1111/nph.14243 . This is an important practical detail since plants should not experience strong stress after the transfer from the initial growing environment to the growth chamber. Otherwise, responses might be altered, affecting the computational properties. Furthermore, plants can even die if the conditions are too different.

Reservoir Computing Performance Evaluation Metrics #

To evaluate the performance for PRC, we consider a set of benchmark tasks. These tasks fall into three categories: input reconstruction, ecophysiological tasks and benchmarks. These categories and the associated variables are listed in Table 4.2. Estimating the modulated environmental inputs of the reservoir ( I_{\text{PAR}} , T_\text{air} and h ) gives an idea of the information directly available at the output. We estimate the actual environmental conditions that were measured using sensors, not the targets used to set the growth chamber. Since PRC is mainly interesting in a biological context and ecophysiological tasks are highly relevant for future applications and research. Finally, from a more fundamental PRC perspective, benchmark tasks such as NARMA, a delay line and polynomial regression enable us to estimate the computational performance in terms of nonlinearity and memory with literature.

All ecophysiological measurements are captured using the gas exchange measurement device (LI6400XT). Different environmental and ecophysiological parameters were captured, providing a diverse set of target variables.

Error Metric #

There are many error metrics used in literature, such as mean squared error (MSE), mean absolute error (MAE), normalised mean squared error (NMSE) and root normalised mean squared error (RNMSE). MSE and MAE are absolute measures, so different values from targets cannot be compared. NMSE and RNMSE are relative measures. They are normalised in some way such that inter-task comparisons are possible. RNMSE is the root of NMSE Here, we focus exclusively on the NMSE because it is used extensively in PRC. The equation for NMSE is:

\hat{y}(t) is the predicted value from the model, y(t) is the actual value, \text{var}(\cdot) computes the variance and N is the total number of samples considered. This metric has several advantages over a traditional mean squared error comparison. It takes the variability of the target signal into account. This eliminates a possible bias towards slow varying signals. Additionally, interpretability is very straightforward. An NMSE of 1.0 corresponds to the mean prediction for all samples, since the numerator reduces to the variance, while an NMSE of 0.0 corresponds to a perfect prediction. This property makes it very easy to compare and interpret NMSE values.

Machine Learning Model Overview and Data Flow #

The different sensors employed in the experimental setup in figure 4.3 and Details on the Experimental Set-up all generate data. This data is combined using a machine learning model to target a specific task. These tasks can be divided into three categories: environmental tasks, ecophysiological tasks and benchmark tasks.

In this work, we restrict ourselves to regression tasks, which naturally fit well with biological tasks that can be observed using the sensors from table 4.2 and all tasks are evaluated using the NMSE error metric.

Ecophysiological and benchmark tasks are observed using sensors and are a natural output of the PRC system. However, environmental tasks are not. These are the input to the PRC system. Yet, we also target these variables because they can give an insight into how these inputs are represented in the reservoir.

Figure 4.4 visualises how the data flows through the system for PRC with plants. Plant state observations such as leaf thickness and hyperspectral data are the outputs of the plant reservoir (figure 4.3). These form the inputs of the linear model, which targets three types of regression targets previously discussed. When we return to the simulated reservoir from figure 2.14, we note that the red edges are implemented by the model from figure 4.4.

In summary, the inputs of the reservoir (environmental conditions) are not the same variables as the inputs of the machine learning model. The inputs of the machine learning model are the observations of the reservoir. Thus, we can use the inputs of the reservoir as targets of the model, to gain a better understanding of the data retained by the reservoir. This exemplifies that PRC is not another machine learning tool, but rather a shift in paradigm. Our goal is not to present PRC as a tool to analyse phenotyping data such as an support vector machine (SVM) or linear regression. Instead, the goal is the interpret these data in an entirely new way.

Summary #

We mapped plants to PRC using evidence from physiological and PRC literature and reviewed the general experimental design of the subsequent chapters. In these chapters, two sensing technologies are used to assess to evaluate the computational properties of plants. In Reservoir Computing with a Snapshot Hyperspectral Camera, a hyperspectral camera is used, while in Experimental Demonstration of a Plant as Computing Resource for Physical Reservoir Computing leaf thickness clips are used.