Robotic modeling of larval phototaxis in Drosophila melanogaster
by Nigel Jaffe ('18)
ABSTRACT
Complex relationships between sensory input and motor output are central to the decision-making process in simple organisms. As such, the ability to characterize those relationships is immensely important to understanding how the brain responds to its environment. Past analyses of the behavioral patterns of the mobile larvae of Drosophila melanogaster have shown larval behavior to be generally defined by a biased random walk away from unfavorable conditions, basing navigational decisions on perceived changes in their immediate surroundings. In order to test current theories about larval navigation, a robotic car was built, the behavior of which was programmed to be influenced by a light gradient across its environment. The robot’s responses to changes in light intensity are dictated by the same relationships which underlie larval behavior. In this case, light acts as an attractant, and so the robot attempts to find the source of the gradient. The motion of the robot across the light gradient was recorded and tracked via computer vision software. The analysis of the navigational patterns shows that its algorithm—based on that of the larvae—produces a clear bias in the direction of the light.
INTRODUCTION
Drosophila melanogaster larvae determine the relative direction of favorable locations by analyzing sensory input provided by sources including but not limited to chemical, thermal, and photosensory stimulants (Dhariwal et al. 2004; Gershow et al. 2012; Kane et al. 2013). Studying the behavior of this organism, in controlled environments leads to the ability to define movement patterns in terms of a cause (change in stimulus intensity) and effect (resulting movement) relationship (Lahiri et al. 2011; Luo et al. 2010). The use of a novel light system and computer vision tracking program has allowed researchers to track individual larvae as they navigate through gradients of different light intensities (Karagyozov et al. 2017). This has uncovered a set of relationships that define how the larvae perform phototaxis, i.e. movement based on visual stimuli. (Kane et al. 2013; Klein et al. 2014).
Drosophila larvae navigate light-fields on the basis of temporal processing of sensory input (Kane et al. 2013), and have a tendency to move away from areas of higher light intensity. It has been demonstrated that larvae are able to successfully travel down linearly varying light gradients, and the resulting path can best be described as a biased random walk (Kane et al. 2013).
Complex relationships between sensory input and motor output are central to the decision-making process in simple organisms. As such, the ability to characterize those relationships is immensely important to understanding how the brain responds to its environment. Past analyses of the behavioral patterns of the mobile larvae of Drosophila melanogaster have shown larval behavior to be generally defined by a biased random walk away from unfavorable conditions, basing navigational decisions on perceived changes in their immediate surroundings. In order to test current theories about larval navigation, a robotic car was built, the behavior of which was programmed to be influenced by a light gradient across its environment. The robot’s responses to changes in light intensity are dictated by the same relationships which underlie larval behavior. In this case, light acts as an attractant, and so the robot attempts to find the source of the gradient. The motion of the robot across the light gradient was recorded and tracked via computer vision software. The analysis of the navigational patterns shows that its algorithm—based on that of the larvae—produces a clear bias in the direction of the light.
INTRODUCTION
Drosophila melanogaster larvae determine the relative direction of favorable locations by analyzing sensory input provided by sources including but not limited to chemical, thermal, and photosensory stimulants (Dhariwal et al. 2004; Gershow et al. 2012; Kane et al. 2013). Studying the behavior of this organism, in controlled environments leads to the ability to define movement patterns in terms of a cause (change in stimulus intensity) and effect (resulting movement) relationship (Lahiri et al. 2011; Luo et al. 2010). The use of a novel light system and computer vision tracking program has allowed researchers to track individual larvae as they navigate through gradients of different light intensities (Karagyozov et al. 2017). This has uncovered a set of relationships that define how the larvae perform phototaxis, i.e. movement based on visual stimuli. (Kane et al. 2013; Klein et al. 2014).
Drosophila larvae navigate light-fields on the basis of temporal processing of sensory input (Kane et al. 2013), and have a tendency to move away from areas of higher light intensity. It has been demonstrated that larvae are able to successfully travel down linearly varying light gradients, and the resulting path can best be described as a biased random walk (Kane et al. 2013).
Drosophila larvae regulate several factors in their movement to produce the best possible sequence of motor responses as they navigate stimulant gradients (Lahiri et al. 2011). Pulsating peristalsis waves from head to tail drive forward movement, occurring in sporadic stretches, termed “runs” (Lahiri et al. 2011; Luo et al. 2010). Each cessation of forward movement is followed by lateral head sweeps, which measure the intensity of the stimulus so that a change in intensity can be recognized (Luo et al. 2010). The direction of the initial head sweep shows no environmental bias, with the larvae being equally likely to first sweep in a favorable or unfavorable direction (Kane et al. 2013). This initial sweep is either accepted or rejected; if a sweep is rejected, another sweep is undertaken in the opposite direction, with this pattern repeating until one is accepted. There is a high likelihood that a sweep in a favorable direction will be accepted while a sweep in an unfavorable direction will be rejected, but these scenarios are not always the case (Gepner et al. 2015). Once a head sweep is accepted, reorientation maneuvers are taken as needed, termed “turns” (Klein et al. 2014; Lahiri et al. 2011; Luo et al. 2010). The entire navigational system employed by the larvae can be reduced to these two motor programs: forward “runs” and lateral head sweeps which determine the direction of the subsequent “turns” (Klein et al. 2014; Luo et al. 2010). By tracking the muscles of the larvae over time as they navigate stimulant gradients, these distinct motor programs have been characterized effectively (Gepner et al 2015; Luo et al. 2010).
Building a robot that can navigate light fields provides the opportunity to test potential algorithms that larvae may employ to process sensory input. For each algorithm, we can compare the robot and larval motion and determine which algorithm produces the most similarities between the two. Unlike the actual larvae, the behavioral strategies used by the robot are already fully mapped and defined. Using the robot, it is possible to test the analysis software’s accuracy by using it to characterize the behavior of the robot instead of the larvae and then comparing its findings to the actual algorithm used by the robot. There have been previous attempts to build a robot which performed phototaxis using a biased random walk inspired by bacterial motion (Dhariwal et al. 2004). Their goal was to test different behavioral approaches employed by bacteria, which is similar to this project in its exploration of different strategies. The purpose of this experiment, however, was to use a robot to test the efficiency of behavioral algorithms employed by Drosophila melanogaster larvae.
The first phase of the project—building and developing the robot—was begun in February of 2016 and completed during the Spring of 2017. The robot has been continually modified over the course of the last two years. The second phase of the project—testing—was carried out over the last summer and has continued into the fall.
MATERIALS AND METHODS
The robot used to simulate phototaxis was comprised primarily of two DC motor powered wheels, an Adafruit car chassis approximately six inches long, a Raspberry Pi 3 Model B single-board computer, and an Adafruit TSL2561 digital photosensor. Additionally, LED lights were added to aid in tracking the robot via custom computer vision (CV) software. The sensor and LEDs were wired to the Raspberry Pi through a breadboard which was secured on top of the car chassis. The sensor was positioned at the front and oriented normal to the plane of motion, similar to the larval sensory organs.
Building a robot that can navigate light fields provides the opportunity to test potential algorithms that larvae may employ to process sensory input. For each algorithm, we can compare the robot and larval motion and determine which algorithm produces the most similarities between the two. Unlike the actual larvae, the behavioral strategies used by the robot are already fully mapped and defined. Using the robot, it is possible to test the analysis software’s accuracy by using it to characterize the behavior of the robot instead of the larvae and then comparing its findings to the actual algorithm used by the robot. There have been previous attempts to build a robot which performed phototaxis using a biased random walk inspired by bacterial motion (Dhariwal et al. 2004). Their goal was to test different behavioral approaches employed by bacteria, which is similar to this project in its exploration of different strategies. The purpose of this experiment, however, was to use a robot to test the efficiency of behavioral algorithms employed by Drosophila melanogaster larvae.
The first phase of the project—building and developing the robot—was begun in February of 2016 and completed during the Spring of 2017. The robot has been continually modified over the course of the last two years. The second phase of the project—testing—was carried out over the last summer and has continued into the fall.
MATERIALS AND METHODS
The robot used to simulate phototaxis was comprised primarily of two DC motor powered wheels, an Adafruit car chassis approximately six inches long, a Raspberry Pi 3 Model B single-board computer, and an Adafruit TSL2561 digital photosensor. Additionally, LED lights were added to aid in tracking the robot via custom computer vision (CV) software. The sensor and LEDs were wired to the Raspberry Pi through a breadboard which was secured on top of the car chassis. The sensor was positioned at the front and oriented normal to the plane of motion, similar to the larval sensory organs.
During phototaxis runs, the robot continually takes illuminance measurements and stores the most recent one. The algorithm employed is such that the robot moves forward until two consecutive illuminance measurements trigger head sweeps. Head sweeps are triggered when a randomly generated number from a uniform distribution on the interval [0,1] falls below the function
where ∆ is the difference in illuminance measurements, a = −1, and b = 0.8. Head sweeps are accepted or rejected according to the same function and acceptance criteria, but where a = 1, b = 0.25, and ∆ being the difference between the last run illuminance measurement and the measurement after a sweep. If a head sweep is rejected, the robot swings through the center and in the opposite direction, taking a new illuminance measurement at the completion of the head sweep. Head sweep direction and duration is randomly determined. It is necessary to have the robot navigate towards the light, in opposition to what larvae actually do, for having it navigate away introduces too much redundancy in potential endpoints. In order to be able to quantify the robot’s ability to perform phototaxis, the robot needed to have a set area to navigate towards.
The arena consisted of a set boundary approximately three feet in radius, a desk lamp, and a Basler Dart daA1600-60uc USB 3 camera situated approximately five and a half feet above the center of the boundary. The lamp used a bright white Philips 800 lumen LED light bulb and was approximately one foot outside the boundary, two feet off the ground, and at approximately 233˚ from the x-axis measured counterclockwise in the frame of the camera.
The arena consisted of a set boundary approximately three feet in radius, a desk lamp, and a Basler Dart daA1600-60uc USB 3 camera situated approximately five and a half feet above the center of the boundary. The lamp used a bright white Philips 800 lumen LED light bulb and was approximately one foot outside the boundary, two feet off the ground, and at approximately 233˚ from the x-axis measured counterclockwise in the frame of the camera.
Pixels, as captured by the camera, are measured with (x, y) = (0, 0) at the top left corner. To make the environment more suitable for filming, the light was dimmed slightly with a shade. Each trial begins with the robot in a random orientation and ends with the robot crossing the boundary. To automate the process, we wrote a CV program in Python and using the OpenCV library. This program was implemented to record the robot’s movements during phototaxis, terminate the phototaxis code when the robot crossed the boundary, direct the robot back to the center of the arena, and then restart the phototaxis code on the robot when it got within a forty pixel threshold of center. This allowed for trials to be run largely autonomously. A blue LED was wired to the Raspberry Pi and situated at the back of the breadboard. This LED served as the object for the CV software to track. Tracking happens as follows: Once the computer vision detects a blue object in its field of view, a minimum enclosing circle for the blue color is drawn and the center of that circle is calculated, returning a single (x, y) pixel coordinate. This coordinate is used as the robot’s position in each frame, and is used by the CV program differently depending on which stage of motion the robot is in (i.e. phototaxis or get-back-to-center). During phototaxis, the position in each frame is exported to a text file containing all (x, y) position coordinates for that trial.
RESULTS AND ANALYSIS
The robot is able to independently reach the source of stimuli using the navigational patterns incorporated into the algorithm, modeling the strategies employed by the Drosophila larvae. The sequence of runs followed by head sweeps and turns has proven effective in locating the source of a stimulant on a gradient plane. Experimentation of movement based on light is in progress and has been successful thus far. The lux sensor used by the robot is sensitive enough to capture detailed readings of the intensity of the light, while the algorithm defining the likelihood of turning based on those readings guides the robot efficiently to the source. Records of the probability of turning based on the change in light compared to the action taken by the robot show that the assigned value of likelihood for each quantitative change in light intensity was appropriate, turning neither too frequently nor too rarely.
Having the computer vision program supervise the robot allowed us to record almost 200 trials. Several example paths can be seen in Figure 4.
RESULTS AND ANALYSIS
The robot is able to independently reach the source of stimuli using the navigational patterns incorporated into the algorithm, modeling the strategies employed by the Drosophila larvae. The sequence of runs followed by head sweeps and turns has proven effective in locating the source of a stimulant on a gradient plane. Experimentation of movement based on light is in progress and has been successful thus far. The lux sensor used by the robot is sensitive enough to capture detailed readings of the intensity of the light, while the algorithm defining the likelihood of turning based on those readings guides the robot efficiently to the source. Records of the probability of turning based on the change in light compared to the action taken by the robot show that the assigned value of likelihood for each quantitative change in light intensity was appropriate, turning neither too frequently nor too rarely.
Having the computer vision program supervise the robot allowed us to record almost 200 trials. Several example paths can be seen in Figure 4.
For all analysis, the data points are first shifted so that the origin is at the center and then rotated clockwise by 127˚ so that the light source is approximately on the x-axis. To quantify the robot’s navigation in each trial, we calculate the navigational index by taking the mean velocity in the x-direction and dividing it by the mean robot speed. This reduces to dividing the difference in the x component of the first and last position by the total path length. Symbolically, this formula is:
where δx = (xj − xi) and δy = (yj − yi). Thus, for a direct line from center to the light source, the navigational index is 1, while motion opposite this has navigational index -1, and totally unbiased motion has an index of zero (see Figure 5).
After averaging the navigational index over 174 trials, we found the mean to be N ≈ 0.36. We also checked that the navigational index perpendicular to the direction of the light was sufficiently close to zero.
DISCUSSIONS AND CONCLUSIONS
The main goal of this research is for a robot to successfully perform phototaxis in a manner resembling that of the larvae themselves, giving insight into the efficiency of their behavioral patterns as the robot’s motion is analyzed and judged with efficiency in mind. This has been accomplished with the robot. The reason that this is so important is that an algorithm that can successfully model larval behavior can be analyzed in detail, leading to a better understanding of how the larval brain works and makes decisions in response to sensory input from its environment. Analysis of the algorithm that dictates the robot’s behavior reveals how similar the robot’s behavior is to that of the larvae and judges the robot’s navigation efficiency. These assessments allow the robot’s behavior to be improved until it resembles the larval behavior as closely as possible. The greater purpose of modeling larval behavior with the robot is to attain a better idea of how accurate the current understanding of the larval decision-making process is, and a comparison between the robot’s behavior and that of the larvae allows us to address that.
As demonstrated by the navigational index of N ≈ 0.36 for the robot’s motion, the current algorithm shows a definite bias for motion in the direction of higher light intensity. Currently, further analysis is being conducted in order to determine the mean run time between head sweeps, which would offer a more robust comparison between our robot and Drosophila larvae phototaxis. We will then move on to test other phototaxis algorithms. A structure is currently being built using PVC pipes through which the camera could be positioned directly over a gradient from a bar running across. This portable setup will allow the whole arrangement to be transported easily.
ACKNOWLEDGEMENTS
I would like to thank all of the members of the Gershow Lab at the Center for Soft Matter Research in the NYU Department of Physics, and especially Paul McNulty, for their wonderful support, guidance, and advice throughout my time working at NYU. I would like to personally thank my mentor, Dr. Marc Gershow, for his amazing leadership, enthusiasm, and dedication to the incredible community of individuals he has created and the work that we do. He has had a profound influence on my work and will no doubt continue to foster learning and scientific advancement wherever he is. I have had the immense pleasure of working with a number of wonderful people in my research, including Paul McNulty, Ruben Gepner, Doycho Karagyozov, Mirna Mihovilovic, Rui Wu, Amanda Lesar, Jason Wolk, Scott Malley, and Natalie Bernat. Finally, I would like to thank my science research teachers at the Packer Collegiate Institute, Erin Schmitz, Lutz Holzinger, and Tamara Machac, for their incredible devotion to learning and to supporting the scientific community as a whole.
References:
DISCUSSIONS AND CONCLUSIONS
The main goal of this research is for a robot to successfully perform phototaxis in a manner resembling that of the larvae themselves, giving insight into the efficiency of their behavioral patterns as the robot’s motion is analyzed and judged with efficiency in mind. This has been accomplished with the robot. The reason that this is so important is that an algorithm that can successfully model larval behavior can be analyzed in detail, leading to a better understanding of how the larval brain works and makes decisions in response to sensory input from its environment. Analysis of the algorithm that dictates the robot’s behavior reveals how similar the robot’s behavior is to that of the larvae and judges the robot’s navigation efficiency. These assessments allow the robot’s behavior to be improved until it resembles the larval behavior as closely as possible. The greater purpose of modeling larval behavior with the robot is to attain a better idea of how accurate the current understanding of the larval decision-making process is, and a comparison between the robot’s behavior and that of the larvae allows us to address that.
As demonstrated by the navigational index of N ≈ 0.36 for the robot’s motion, the current algorithm shows a definite bias for motion in the direction of higher light intensity. Currently, further analysis is being conducted in order to determine the mean run time between head sweeps, which would offer a more robust comparison between our robot and Drosophila larvae phototaxis. We will then move on to test other phototaxis algorithms. A structure is currently being built using PVC pipes through which the camera could be positioned directly over a gradient from a bar running across. This portable setup will allow the whole arrangement to be transported easily.
ACKNOWLEDGEMENTS
I would like to thank all of the members of the Gershow Lab at the Center for Soft Matter Research in the NYU Department of Physics, and especially Paul McNulty, for their wonderful support, guidance, and advice throughout my time working at NYU. I would like to personally thank my mentor, Dr. Marc Gershow, for his amazing leadership, enthusiasm, and dedication to the incredible community of individuals he has created and the work that we do. He has had a profound influence on my work and will no doubt continue to foster learning and scientific advancement wherever he is. I have had the immense pleasure of working with a number of wonderful people in my research, including Paul McNulty, Ruben Gepner, Doycho Karagyozov, Mirna Mihovilovic, Rui Wu, Amanda Lesar, Jason Wolk, Scott Malley, and Natalie Bernat. Finally, I would like to thank my science research teachers at the Packer Collegiate Institute, Erin Schmitz, Lutz Holzinger, and Tamara Machac, for their incredible devotion to learning and to supporting the scientific community as a whole.
References:
