Automated or semiautomated time-lapse analysis of early stage embryo images during the cleavage stage can give insight into the timing of mitosis, regularity of both division timing and pattern, as well as cell lineage. Simultaneous monitoring of molecular processes enables the study of connections between genetic expression and cell physiology and development. The study of live embryos poses not only new requirements on the hardware and embryo-holding equipment but also indirectly on analytical software and data analysis as four-dimensional video sequencing of embryos easily creates high quantities of data. The ability to continuously film and automatically analyze growing embryos gives new insights into temporal embryo development by studying morphokinetics as well as morphology. Until recently, this was not possible unless by a tedious manual process. In recent years, several methods have been developed that enable this dynamic monitoring of live embryos. Here we describe three methods with variations in hardware and software analysis and give examples of the outcomes. Together, these methods open a window to new information in developmental embryology, as embryo division pattern and lineage are studied in vivo.
- embryo cleavage
- time-lapse analysis
- embryo profiling
- cell lineage
Despite 30 years of practice, the success rate for implantation of embryos into the uterus in in vitro fertilization (IVF) is still only around 30% [1, 2]. Consequently, when transferring embryos from in vitro culture and implanting them, it is critical that only the best embryos are selected. This will not only optimize the chance of live birth but also reduce the need for multiple embryo transfer, with the subsequent risk of twin pregnancy and the neonatal complications and associated maternal pregnancy-related health problems. Though cultivation methods have improved, embryo selection is still largely based on manual evaluation of morphological criteria, and much research has been done in identifying morphological features correlated with embryo health. Other methods such as genetic screening and metabolic profiles of culture media exist, but have not yet proven to increase pregnancy rates [3–9]. There is an ongoing discussion concerning the relevance of embryo morphology in quality assessment , but it is likely that it will continue to play a large part in IVF embryo evaluation also in the future. Traditionally, embryo quality assessment has been performed by manual inspection using light microscopy at intermittent time points during embryo development. Novel technical solutions have recently made it possible to monitor embryos continuously using time-lapse imaging, opening new possibilities for embryo evaluation based on dynamic properties.
It has been shown that the timing of key occurrences within the embryo can vary greatly between embryos that have similar morphological appearance at the end of the recording period and that embryo morphology can change in a matter of hours [11–14], emphasizing the fact that dynamic monitoring is preferred over intermittent monitoring of embryos. An important endpoint for embryo studies is the timing of embryo cleavage, which has been shown to correlate to embryo viability and implantation potential [15–18]. For research purposes, tracking of cell lineage and cell positioning within the early embryo provides important information to understand pluripotency. Embryos are also a good model for the study of developmental biology and three-dimensional cellular interaction. The ability to continuously film and analyze growing embryos gives new insights into temporal embryo development by studying morphokinetics as well as morphology. Until recently, this was only possible by a tedious manual process. Although currently some human IVF laboratories have started to use time-lapse technology to monitor embryo cleavage and growth, further description of the technology and its potential is needed. The focus of this chapter is on the methods used to study living early embryos over time and the possibilities they render as new tools for embryological research and clinical application.
2. The role of live imaging in embryology
Conventional microscopy suffers from several drawbacks, such as requiring sample fixing and only providing static information in an intermittent manner. The complete understanding of cell division and development requires a dynamic perspective on an individual cell level as most information on cell response to environment, dynamic gene expression and timing would be missed in a static analysis. In recent years, the imaging technologies have provided new tools in microscopy, sample handling, and hardware and software for live imaging of individual cells. There are several examples of single cell [19–22] and single molecule monitoring in living cells, using both marker-based and marker-free approaches.
Fluorescent tags enable the tracing of specific proteins and measurement of their characteristics to study gene expression, protein localization, and function and protein-protein interaction. By using several markers simultaneously, it is possible to track several proteins or gene expressions at once. With time-lapse microscopy, intracellular events can be linked to external factors such as cell-cell interaction and ultimate cell fate. These methods give us remarkable new insights into the dynamics of gene expression, cellular interactions, and heterogeneous processes. In fluorescence imaging, a laser is used to excite the fluorophores at a particular wavelength. Full field epifluorescence can then be used to measure the light as the fluorescent tags emit light while returning to their unexcited molecular state . In confocal imaging, a pinhole in combination with focused laser light is introduced to effectively reduce background fluorescence and allows optical sectioning of the sample by mechanical scanning. Varying the pinhole will effectively vary the thickness of the sample being imaged, the image resolution, and the acquisition time.
For some applications, the use of fluorescent tags is not feasible. By continuously filming embryo material some important information on cell outline, position, shape, and texture can be extracted from the time-lapse sequences without the use of fluorescent markers. By matching and tracking, this information can be combined to a timing profile of the dividing embryo, detecting temporal location of division and tracking cell lineage over time. Using computer vision in combination with a noninvasive imaging method makes it possible to continuously study embryo growth with minimal sample interference. Fluorescence imaging allows the noninvasive measurement of gene expression and intracellular characteristics, while marker-free light microscopy allows the tracking of cellular size, shape, and behavior over time in response to molecular changes. This combination gives us the possibility to directly monitor cellular responses and changes in gene expression in response to the environment. The result is a cellular model that can bridge the molecular scale to the cellular, mapping the actual connections between the chemical and the biological world.
3. Noninvasive techniques for embryo imaging
Currently, a set of biotechniques has been successfully applied to mouse and human embryo imaging. This technique includes the addition of a fluorescent marker and marker-free methods. For research purposes, the addition of fluorescent proteins can be considered a noninvasive method, if the protocol used does not significantly disturb embryo growth. For clinical applications in human embryology, no markers of any kind can be used. In this section, we will refer to fluorescent marker methods as noninvasive, and specify the “truly” noninvasive method as “marker free.”
3.1. Fluorescence imaging
Adding fluorescent proteins (FP) is a standard way to selectively study specific intracellular targets . The most common fluorescent tag is the green fluorescent protein (GFP) , derived from the jellyfish
FP can also be used to study the dynamics around the FP binding site by fluorescence recovery after photobleaching (FRAP) . In FRAP, a fluorophore is covalently attached to the molecule of interest. The fluorophore is intentionally photobleached using incident laser light. The diffusion of the molecules can now be quantified by studying the gradual brightening of the photobleached spot, as fresh fluorophores migrate into this area. Three closely related techniques are the fluorescence loss in photobleaching (FLIP), fluorescence decay after photoactivation (FDAP), and fluorescence correlation spectroscopy (FCS) . Fluorescence resonance energy transfer (FRET) (sometimes also called Förster resonance energy transfer) can be used to study protein-protein interactions . In this case, a donor fluorophore is placed in an excited state by incident laser light, and the energy held in the excited molecular state is transferred to an acceptor fluorophore which must be in close proximity (typically less than ten nanometers). When two molecules under study are labelled with the donor and acceptor fluorophores, respectively, the detected light from the acceptor fluorophore indicates that the two molecules are in close proximity.
3.2. Marker-free microscopy
Currently, IVF centers or clinics are using two main techniques for embryo imaging: Hoffman modulation contrast imaging (HMC) (sometimes referred to as white light) [37, 38] and darkfield imaging (DF) . For research purposes, CARS  and light sheet microscopy  are also becoming increasingly common. HMC was standard before time-lapse imaging of IVF embryos came in use and is still used in manual microscopy set ups. Consequently, images from time-lapse sequencing resemble the microscopy images to which embryologists are accustomed, an advantage when annotating images and comparing manual and computational approaches. HMC is best suited for imaging internal cell detail. On the other hand, Darkfield gives better detail to edge structures such as cell membranes, and more accurately to detect and track cell outlines.
Darkfield imaging is an imaging method that excludes any unscattered light, causing the samples to appear brighter on a darker background and enhancing the contrast of the imaged and unstained sample . It is a simple yet effective method to noninvasively enhance sample contrast but has the disadvantage of low light levels available for collection. To compensate, the sample must be strongly illuminated and the heavy light exposure can cause sample damage. However, the low light level also means the image is almost entirely free from optical artifacts. Darkfield microscopy is most useful for studying boundary structures with a high difference in refractive index and imaging cell membranes is, for instance, more effective than internal cell structures. It is best suited for thin samples with high differences in refractive index (such as for sharp edges) and for thick samples, artifacts may occur.
HMC Imaging was invented by Hoffman in 1975 . Today, it is a common technique for noninvasive contrast enhancement of biological samples. Its advantages include good contrast, low light exposure, excellent resolution, and a short depth of field, with the opportunity of focal sectioning at a resolution controllable by the numerical aperture of the objective. The ability to section is also influenced by the sample homogeneity. The disadvantages include strong optical artifacts and image appearance unsuitable for computerized image processing. HMC is commonly been used for embryology studies and has been included in a number of commercial products.
4. Challenges in live embryo imaging
Although advances have been achieved in techniques for live single-cell imaging in recent years, several challenges still exist for wider implementation. An experimental design for long-term imaging and analysis must ensure not only high-quality imaging but also long-term support for sample vitality and appropriate computational methods for the analysis. Observing embryo in vitro requires an incubator environment to provide optimal living conditions or the sample during the imaging period. Temperature changes can affect the function of physiological processes as well as reaction kinetics and the challenge will increase with the length of the study sequence. One solution is the installation of an incubation flow chamber on the microscope, reducing the amount of gas and liquid to sustain the sample to a small volume, but suffering from drawbacks such as the risk of introducing condensation on the incubator chamber surfaces. Another approach is to integrate the microscopy optics in an incubator chamber, posing demands on the microscope optics and electronics to function in a humid, temperate atmosphere. A limited number of commercial solutions exist, which combine incubation capabilities with imaging hardware. With any of these solutions, the embryo medium and container must not introduce imaging artifacts such as light reflecting surfaces, auto-fluorescence, or excessive medium volumes in the light path. Another challenge is the loading and retrieval of cells from the mounting chamber, a process that may cause loss of cell identification. For IVF, several combinations of incubators and microscopes exists , either as integrated solutions or in the form of a microscope designed for use inside an incubator. So far, no difference has yet been seen in growth and implantation rates of embryos grown in the standard intermittent incubator system and a time-lapse incubator system [45–47]. One study found a higher rate of miscarriage for the time-lapse group, indicating there are reasons for caution. However, the same study noted no effect on pregnancy rates or embryo health prior to implantation .
In nonhuman IVF, phase contrast microscopy is commonly utilized instead of HMC. Phase contrast microscopy is similar to HMC in that it gives high level of image detail at the expense of image artifacts in the form of halos around sample objects. The varying appearance of embryos of different species will affect the decision of which optical system to use. Some species have dark, dense-appearing embryos (e.g., pig), while others are more translucent (e.g., mouse). As a consequence the optimal optical system for a given embryo species vary, and any appropriate analytical software must be chosen accordingly. Single-cell studies using darkfield imaging is limited by the hardware to the 4–6 cell stage. Using focal sectioning in HMC, it is possible to image the entire embryo from zygote to blastocyst stage, but any automated analysis becomes increasingly difficult with increasing cell number as the out-of-focus image details cannot be removed, despite the sectioning. In humans, the compaction at the 9–16-cell stage involves a reduction in visibility of cell boundaries and may represent a feasible stage for automated detection beyond the 8-cell stage. The cavitation and blastocyst formation stages also offer opportunities for automated analysis of images, covering expansion and collapse events.
In fluorescence time-lapse imaging of nonhuman mammalian embryos, the life time of the fluorophores is limited, an effect referred to as
There is a trade-off between information gathered and potentially harmful sample exposure, and the frequency of image capture must be carefully chosen depending on the study end-point and the expected frequency of the dynamics under study. In the case of simultaneous monitoring of multiple samples, two solutions exist. In
For two-dimensional imaging, full-field techniques are the most efficient as they capture the entire field of view in one single exposure. However, the stability of the system becomes critical as the time-lapse sequence length increases.
Even with moderate capture frequency, the amount of data from time-lapse studies can quickly build up to terabytes or more, especially if data is recorded simultaneously in multiple dimensions and imaging modalities. Consequently, both
No single set of experimental conditions for long-term imaging can be used universally. Each biological question and model requires its own specific combination of hardware and software tools and must often be customized. Solutions to these challenges will enable important discoveries in embryology in the future. Kang et al.  and Turksen  provide useful summaries of protocols for fluorescent labelling and the imaging and tracking of stem cell, respectively. The following three sections exemplify successful time-lapse imaging methodologies for both human and nonhuman embryos with solutions to the experimental challenges using three very different approaches.
5. Method 1: three-dimensional mouse embryo morphology using fluorescent markers
To understand compaction, cell lineage, cell rearrangement and dynamic behavior of embryonic cells during the cleavage phase, and dynamic imaging is necessary. This project studied the role of filopodia formation in compaction, apical constriction, pluripotent cell internalization, and cell positioning prior to embryo compaction, which is believed to be important for pluripotent development of embryonic cells. In addition, intracellular processes are monitored using a variety of targeted fluorescently tagged proteins and transcription factors.
With fluorescence microscopy, we can selectively excite and visualize fluorescent proteins as a marker in living tissue. The discovery of genetically encoded fluorescent proteins (FPs) permits the quantitative analysis of most cellular proteins including monitoring of their distribution and dynamics . Fluorescence imaging is a technique that perfectly addresses problems in embryonic development, because of the need to study embryos in vivo. In confocal microscopy, in contrast to widefield fluorescence imaging, the detector pinhole blocks fluorescence from areas that lie out of focus . This allows confocal imaging to reduce some of the scattering effects elicited by widefield fluorescence microscopy. However, scanning a single section implies the excitation and, therefore, damaging off-focus areas above and below the focal plane. In addition, the pinhole will also exclude scattered signal photons emitted from the focal plane as they travel away from the specimen. Therefore, widefield and confocal imaging are methods best suited for thin samples of less than ~40 μm. To study the events occurring deeper in the mouse embryo, which is about 100 μm in diameter (70 m of cellular portion plus the zona pellucida), requires the use of two-photon excitation fluorescence microscopy.
Two-photon excitation (2PE) fluorescence microscopy is a way to limit phototoxicity in the sample and to extend the imaging time and depth at high resolution and contrast . In 2PE, two photons of half the excitation energy are needed to place the FP in the excited state. A focused laser is used in 2PE to generate higher intensity localized in the area of the focal plane, which results in excitation limited to a very small focal volume (typically of ~0.1 μm3). A combination of confocal and two-photon excitation (2PE) fluorescence microscopy can be used to follow and characterize different morphogenetic changes in developing embryos such as cell division, polarity, filopodia formation and dynamics, compaction, and blastocyst cavitation (Figure 1). For this aim, specific fluorescently tagged proteins or peptides are used to label nuclear, cytoplasmic, or membrane constituents and optimized confocal and 2PE fluorescence imaging methods [29, 31, 65]. These imaging conditions allow the scan of a single embryo at intervals down to less than 60 s and reconstruction of 3D embryo morphology using Imaris (Bitplane AG) or ZEN (Zeiss) software. For long-term imaging sessions positioning software (Zeiss Zen) is used to image 20–30 embryos cultured next to each other (Figure 1). Thanks to the high-sensitive detectors of confocal and 2PE fluorescence microscopes, it is possible to perform long-term imaging sessions lasting more than 24 h, without this affecting the health and integrity of the mouse embryos. Thus it is possible to follow in an overnight imaging session cell dynamics in 20–30 embryos. Images are captured at intervals of 40 min from eight-cell stage to blastocyst (an interval of about 36 h). Capturing fluorescent imaging together with brightfield optics makes it possible to monitor simultaneously cell and molecular dynamics (Figure 1D).
For the simultaneous subcellular study of proteins at different stages of development, it is possible to study the dynamics of subcellular markers from zygote to blastocyst stage. For this purpose, DNA constructs in the pCS2+ expression vector  and synthesized capped RNA (using the Ambion mMessage mMachine SP6 transcription kit) are used. Capped marker-GFP RNA is injected into one-cell stage embryos. For nulcei, H2B-RFP is commonly used as marker, whereas memb-mCherry, Ecad-RFP, Ecad-GFP, or Ezrin-RFP can be used for membrane monitoring (Figure 1) [32, 34, 65]. Figure 1C shows an example of using the nuclear marker H2B-GFP and the membrane marker Ecad-GFP. Polarity events can be studied using Ezrin-GFP. Ezrin is expressed homogeneously in all cells before it becomes polarized during embryonic compaction  (Figure 1B). Hence, colocalization with Ezrin-GFP is an excellent way to study the dynamics and distribution of any protein of interest during compaction and cell polarity.
6. Method 2: cell lineage studies of human embryos using machine learning
This method focuses on automated monitoring of human embryonic cells in dark field time-lapse microscopy images of embryos with the goal to develop methods to segment, detect, localize the embryonic cells at each time step, and perform cell lineage analysis on a complete sequence. The result is a helpful tool for embryologists and IVF clinicians to understand the development of human embryo and more accurately select viable embryos.
In contrast to other cells (e.g., stem cells and embryonic cells of other species), automated analysis of nonstained human embryonic cells is challenged by complex development patterns such as compact growth and overlapping cells. These challenges are further complicated by the limitations of the single plane imaging limitations imposed by the dark field imaging mode, causing intensity variance and loss of depth information.
An important and first step in automated analysis is being able to efficiently and reliably segment the embryo from background noise. To this end, a framework to segment the developing embryo by estimating the contour around the embryo was developed by defining segmentation as an energy minimization problem and solving it via graph cuts . Second, cells are spatially localized and divisions subsequently detected. For localization purposes, cells are modeled as ellipses fitted to the segmented outlines for each time step (Figure 2).
Predicting the number of cells is a fundamental task in cell biology analysis, and an indirect way to temporally locate embryo cleavage events. In the context of human embryonic cells, cell number is of prime importance as current embryo viability biomarkers require accurate cells counts. The prediction of cell numbers can either be performed directly from the microscopy images  or by detecting (localizing) cells [70, 71]. Both approaches can also be used in combination. In this method, a framework that combines both approaches in a conditional random field (CRF)  is used. The result is a model of the cell division ancestry by recording cell associations between adjacent frames, resulting in a complete lineage tree for the time-lapse sequence. Cell lineage analysis is vital in understanding dynamics of developing embryos and is a fundamental step in cell biology analysis. The cell lineage tree and segmented shapes can now be studied for various attributes of the growing embryo such as timing of cell cleavage, abnormal division patterns, and cell symmetry (Figure 3).
7. Method 3: human embryo profiling using video image processing
HMC imaging is superior when it comes to image detail of human embryos. However, optical artifacts introduced by the optical modulation causes edge structures to appear with multiple gradients. Objects in focus commonly appear clearly, but at the same time, superimposed light from out-of-focus objects will often introduce “shadows” in the image. The result is an image where it is inherently difficult to segment cell outlines, but with a high degree of detail in internal cell structures, despite the fact that the technique is completely marker free. Attempting to segment such an image is possible, but since subsequent analysis is often dependent on the resulting segmented outline, it is easy to introduce cumulative errors. This method focuses on the detection of developmentally relevant events in the embryo such as compaction, blastocoel formation, nuclei localization, cell cleavage, and embryo fragmentation without the need for complete segmentation.
Raw images are spatially filtered to embryo location and a set of image features are extracted from the embryo interior [73, 74]. As the embryo grows, characteristics of the image will change also the image features, making it possible to profile embryo development without the complete image data . One example is shown in Figure 4, where the gray-level variance of the image of the embryo interior is used to plot a development sequence of the embryo in two dimensions. The gray-level variance is a measure of the contrast in the image and will increase for each cell division, as each division introduces a new cell, and thus a new set of darker cell membrane into the image, contributing to a rise in image variance. As a consequence, each cell division can be detected as a sudden steep gradient in the variance profile. The compaction is detected as a massive loss of variance as the cell membranes becomes less apparent, followed by a new increase in variance as the embryo forms a blastocoel. The ideal development of an embryo follows a predictable pattern over time, where events such as cleavage can be more easily and automatically detected than using images directly (Figure 4) and abnormal development will differ clearly (Figure 5).
Simultaneously to feature detection, segmentation of intracellular structures such as nuclei and pronuclei is possible due to the high level of image detail (Figure 6). The segmentation is constricted in shape and size, ensuring the located structures are of the predefined biological shape. A slight disturbance is introduced in the form of a rotation and serves to effectively average out the located structures and preventing the detection of false positives . The result is a framework where the entire development from zygote to blastocyst can be profiled and combined with the visibility of relevant intracellular compartments such as nuclei, without the need for any fluorescent markers.
It has been shown that embryos can grow outside the womb for longer than 14 days, a limitation set by legal requirements . This period of early embryo development has yet been little studied, due to technical constraints. New combinations of software analysis, imaging, and incubator technologies will soon make it possible to study embryo development from a whole new set of perspectives.
Using specific FP-tagged protein markers for the nucleus and plasma membrane it is possible to follow the dynamics of important morphogenetic changes during mammalian embryo development, including cell division, cell polarity, and cavitation during blastocyst formation. The quantitative analysis of these developmental hallmarks pave the way for the design of functional and phenotypical studies such as silencing (knocking down), overexpressing, or blocking using inhibitors of selected genes of interest. These method combinations can lead to the crucial understanding of developmental function and disease.
Methods for automated or semiautomated label-free analysis of embryos in vivo make it possible to study embryo development over longer times than previously possible—opening up a new set of insights into especially early human development, where ethical considerations are important for the choice of study method. By time-lapse sequence studies of routinely growing embryos in IVF, the research data can be gathered in a clinical context, and methods can simultaneously contribute to better IVF embryo monitoring.
In conclusion, these noninvasive methods open a window to increase the understanding of general developmental embryology as well as specific medical questions such as embryo division patterns, lineage, and the reasons behind the low human fertility rates.
Anna Leida Mölder is grateful for the help and support of Manchester Metropolitan University. Juan Carlos Fierro-González is grateful for the support received from the Swedish Society for Medical Research.