Alzheimer’s disease (AD) is the most common form of dementia affecting millions of people worldwide and currently, the only possible treatment is the use of symptomatic drugs. Therefore, there is a need for new and disease-modifying approaches. Among the numbers of biological targets which are today explored in order to prevent or limit the progression of AD, the modulation of serotonin receptors the subtype 4 and 6 receptors (5-HT4R and 5-HT6R) has received increasing attention and has become a promising target for improving cognition and limit the amyloid pathology through modulation of the neurotransmitter system. A large number of publications describing the development of ligands for these serotonin receptors have emerged, and their pharmaceutical potential is now quite evident. However, 5-HT4R and 5-HT6R functionality is much more complex than initially defined. This chapter describes recent advances in the understanding of this modulation as well as the medicinal chemistry efforts towards development of selective 5-HT4R or 5-HT6R ligands.
- serotonin pathways
- 5-HT4R and 5-HT6R modulators
- structure–activity relationship
Alzheimer’s disease (AD) is a devastating but poorly treated disease. Therefore, there is an urgent need for new and efficient treatment strategies, emphasized by recent statistics from WHO predicting that AD will become the second-most prevalent cause of death within 20 years.
Mounting evidence accumulated over the past years indicates that the neurotransmitter serotonin plays a significant role in cognition and memory. The intimate anatomical and neurochemical association of the serotonergic system and brain areas affected in AD have inspired researchers to focus on this system as a major therapeutic drug target.
Based on the current knowledge of mechanisms involved in serotonin regulation, we here provide structural insight into chemical compounds that have been developed for targeting of the serotonin receptors the subtype 4 and 6 receptors (5-HT4R and 5-HT6R) processes as potential treatments in AD.
2. The serotonergic system in Alzheimer’s disease
Serotonin is a small molecule that functions both as a hormone in the periphery, and as neurotransmitter and neuromodulator in the central nervous system (CNS) . In the CNS, it is produced by a small cluster of neurons located in the raphe nuclei of the midbrain. Through innervation of numerous brain regions, serotonin (5-hydroxytryptamine, 5-HT) modulates various physiological functions including circadian rhythms, mood, sleep, appetite, and learning and memory. The areas of the brain involved in learning and memory show high concentrations of 5-HT1AR, 5-HT4R, 5-HT6R and 5-HT7R, why modulation of these is of particular interest in for reversing the cognitive impairment associated with AD .
AD has been linked to a decrease of serotonergic neurons in the raphe nuclei, seemingly due to the accumulation of hyperphosphorylated Tau as well as deposits of amyloid beta in the projection sites of serotonergic neurons, causing retrograde degeneration of the neurons . Furthermore, a significant decrease in the number of serotonin transporter (SERT) have also been reported . Overall, this leads to a decrease in serotonin neurotransmission, suggesting that increasing serotonin level in the raphe nuclei can improve cognitive performance in AD patients. This is supported by studies showing that administration of selective serotonin reuptake inhibitors (SSRI) to mouse models of AD, reduced the production of toxic amyloid beta plaques [5, 6]. However, recent clinical trials concluded that treatment with amyloid beta lowering agents should be administered in the very early stages of the disease progression to have any impact on AD, limiting their use until better pre-symptomatic AD diagnostics have been developed.
Several 5-HT receptors (5-HTR) have been shown to influence processing of the amyloid protein precursor (APP), including 5-HT2AR, 5-HT2CR, and 5-HT4R [7, 8]. Among them, the 5-HT4R and 5-HT6R receptors have been of most interest. The 5-HT4R was identified as a most promising target, since activation of this receptor shift the equilibrium of APP cleavage towards formation of the soluble non-amyloidogenic form (sAPPα) fragment possessing neurotrophic and neuroprotective properties , while the 5-HT6R has caused much interest for potential roles in AD due to its modulatory effects on gamma-aminobytyric acid (GABA) and glutamate levels,  which facilitate the secondary release of other neurotransmitters including dopamine, noradrenaline and acetylcholine, all of which are compromised in AD. In addition, 5-HT6R are exclusively found in the CNS, indicating the possibility of selective CNS targeting to limit off-target toxic effects.
3. Serotonin subtype 4 receptor
Among the large family of 5-HTR, the 5-HT4R’s are postsynaptic receptors. Although widely expressed throughout the body, the highest density is observed in the brain (olfactory tubercles, basal ganglia, substantia nigra, superior colliculi, hippocampus, and cortex). These are all CNS structures that are extensively involved in cognitive functions, suggesting that the 5-HT4R could be a therapeutic target for improving memory performance and hereby slowing memory deficits, such as those that occur in AD . Moreover, it has been shown that 5-HT4R expression is reduced in AD patients. Furthermore, it has been shown that activation of these receptors enhances the release of acetylcholine in the frontal cortex and hippocampus , increases long term potentiation in the hippocampus  and induces a rapid and sustained increase in basal firing of 5-HT cells in the dorsal raphe nucleus [13, 14]. 5-HT4R activation also stimulates hippocampal expression of plasticity/learning-related proteins such as brain-derived-neurotrophic-factor, AKT, CREB, as well as neurogenesis in the dentate gyrus .
In addition, and a major advantage of using 5-HT4R agonists in treatment of AD, is their ability to shift the equilibrium of APP processing pathway from production of the neurotoxic amyloid-beta-peptide towards formation of the sAPPα . In contrast to amyloid-beta-peptide, the soluble form has putative neurotropic and neuroprotective properties, see Maillet et al.  for a review. The ability of 5-HT4R agonists to stimulate the amyloidogenic pathway leading to release of soluble form of APP has been demonstrated in various cell-based animal models .
In early years, 5-HT4R agonists attracted much attention as potential gastrointestinal drugs used in the therapy of functional bowel illnesses such as constipation, irritable bowel syndrome, gastroparesis, and gastroesophageal reflux disease . The first generations of 5-HT4R agonists used in clinical medicine were Tegaserod (
Another concern is the risk that prolonged or repeated exposure of the 5-HT4R to an agonist, may lead to receptor desensitization. The 5-HT4Rs are G-protein-coupled receptors (GPCRs), which can be desensitized following activation by agonists . Agonist-induced desensitization of GPCRs is less common for partial agonists than strong agonists and therefore, most focus has been given to developing highly selective partial 5-HT4R agonists for treatment of AD. In order to being therapeutic useful in treatment of AD, the compound must therefore fulfill several requirements. In addition to being a selective partial 5-HT4R agonist, the target molecule must show good brain barrier penetration, which was also limited in the early generation agonists .
However, the potential for 5-HT4R partial agonists to offer clinical benefit for the treatment of AD has indeed been demonstrated. Data from a small Phase 2 study in patients with mild to moderate AD with the selective partial 5-HT4R agonist, PRX-03140 (
This has stimulated much research aiming at designing and developing more selective 5-HT4R partial agonists.
3.1 Pharmacophore of the 5-HT4R ligand
In accordance with the natural ligand, 5-HT (
Suitable aromatic systems  include 4- amino-5-chloro-2-methoxy benzoic acid, indole, imidazopyridines and N-alkyl benzimidazolodonone among others (see Figure 1C). Several basic amines with voluminous substituents such as piperizines , and piperidines  have been used.
3.2 Agonists of 5-HT4R
Over the last years, a broad range of structural varied 5-HT4R agonists have been developed, which all share the common structural features presented by the pharmacophore described above. Below we have grouped them into 4 main groups and discuss the structural features for each of these groups.
3.2.1 Group 1: benzisoxazole, oxindole and benzimidazolodonone core
Much research aiming at developing new 5-HT4R agonists has focused on compounds possessing a benzyl ring linked to a 5 membered heterocycle, analogous to the indole ring in the natural 5-HTR ligand.
Inspired from early generations of 5-HT4R agonists (
For benzisoaxazoles (
Orjales and coworkers  provided a structure–activity-relationship (SAR) study of 2,3-dihydro-2-oxo-1H-benzimidazole-1-carboxamide derivatives bearing a piperazine moiety (
A similar trend with benzimidazolone 5-HT4R ligands was reported by Langlois et al.  While the DAU-6215 ligand (
3.2.2 Group 2: chloro-aniline core
The parent compound of this class is metoclopramide [36, 37] (
Russo et al.  synthesized and studied a library of structures based on the 5-HT4R partial agonist, ML10302 (
In 2019, Lanthier et al.  generated a MTDL targeting both activation of the 5-HT4R while also bearing antioxidant activities; hereby being able to both control Ab protein accumulation and prevent toxicity of reactive oxygen species (ROS) in neuronal cells .
The chloro-aniline core connected via a chemical spacer to a basic piperidine ring (structure
A similar approach was investigated by Yahiaoui et al., who reported the design of the dual compound
To further elaborate on these studies, Hatat et al.  designing an antiamnesic MTDL with balanced 5-HT4R agonist, 5-HT6R antagonist and AChE inhibitory activities. Starting from the dual MTDL
3.2.3 Group 3: imidazo[1,2-a]pyridine, imidazo[1,5-a]pyridine, imidazo[4,5-b]pyridine and imidazo[4,5-c]pyridine core
Compounds based on the imidazo[1,5-a]pyridine [63, 64] core were initially reported as dual mediators of 5-HT3R and 5-HT4R , see Figure 10 (
3.2.4 Group 4: quinoline core
Compounds containing the quinoline bicyclic aromatic core have attracted much attention in the design of 5-HT4R ligands [67, 68, 69, 70]. To provide medicinal chemistry understanding of the quinoline 5-HT receptor ligands, Castriconi et al.  synthesized and studied binding affinity of potential 5-HT4R agonists with reference to the 5-HT4R ligands ML10302 (
In later efforts, Cappelli and coworkers published a more comprehensive SAR study of receptor ligands with 2-methoxyquinoline as the aromatic system . A series of piperidine containing functionalities were investigated, demonstrating N-butyl-4-piperidinylmethyl (also present in RS67333 (
4. Serotonin subtype 6 receptor
The 5-HT6R was discovered in 1993 by Monsma et al. [73, 74, 75, 76]. These receptors are GPCRs, which are located postsynaptically to serotonergic neurons . Since its identification, significant efforts have led to a better understanding of the biology of this receptor. The 5-HT6 receptors are present in regions of the brain regions responsible for learning and memory, making them of high interest in AD research. Furthermore, blockade of 5-HT6R function was shown to increase acetylcholine- and glutamate-related neurotransmission, which enhances learning and memory [78, 79]. Evidence indicates that blockade of this receptor improve both cholinergic and glutamatergic system . Furthermore, blockade of 5-HT6R alleviates memory deficits, such as age-related decline in cholinergic or glutamatergic neurons, [79, 80]. Studies conducted by Kotańska et al.  revealed that antagonism of the 5-HT6R enhanced neuroplasticity, helped maintain neurite growth and provided a neuroprotective effect against amyloid beta neurotoxicity [81, 82].
This has led to a high interest in this receptor in treatment of the cognitive decline associated with AD [75, 78, 79, 83]. In addition, the receptor is exclusively expressed in the CNS, primarily in the striatal, hippocampal and cortical areas of the brain  and therefore, could potentially provide therapeutics with limited peripheral side-effects .
Among the first reported selective antagonists are Ro-04-6790 (
4.1 Pharmacophore of the 5-HT6R ligand
From 1998 until today a fair number of studies have been conducted on 5-HT6R antagonists, which led to a good understanding of the pharmacophore. There are four main features responsible for interaction with the receptor: a polar positively ionizable (PI) group, a hydrogen bond acceptor (HBA), an aromatic area (AR) and a hydrophobic site (HYD) [75, 79] (see Figure 14).
In 2017, González-Vera and coworkers published  a SAR study regarding the hydrophobic moiety (HYD). In total 18 compounds were synthesized, all containing a sulfonamide as the HBA moiety. This study revealed that aromatic halogens in the HYD part of the structure increased affinity.
Based on this pharmacophore framework, a broad range of substances has been investigated as ligands for the 5-HT6R, aiming at introducing selectivity.
4.2 Antagonists for 5-HT6R
A comprehensive review was published by López-Rodríguez and colleagues in 2014 , which discussed the structural key features of 5-HT6R antagonists. In this review, they grouped them into 4 overall groups of structures that had been investigated for antagonism of 5-HT6R (see Figure 15). They made some general conclusions, which are summarized in Figure 15.
For more details about specific compounds, please refer to . This chapter will focus on studies made since 2014, while the reader is referred to the following excellent reviews [75, 79] for investigations before 2014.
A SAR study published by Zajdel et al. in 2016  studied analogues of the natural substrate 5-HT (
The two compounds were further tested for their functional activity
In 2019 Hogendorf et al.  published a comprehensive SAR study involving more than 50 compounds, all containing an aminoimidazole moiety as a new bioisoster of the classical PI amino-group. These were found to form strong hydrogen bonds with electronegative acceptors, such as carbonyls, in the enzyme active site. In total six different series were investigated, but only the two series that were found to be most important for activity are included in Figure 17.
Based on affinity, physiochemical properties (ClogP, pKa, topological polar surface area, water solubility, etc.), metabolic stability, mutagenic/toxicity and BBB permeation, compound
AD is more than just an imbalance in the cholinergic or glutamatergic systems and therefore, it was speculated whether better therapeutics could be achieved by using MTDLs affecting several neurotransmitter pathways . In recent years, various MTDLs have been studied .
Marcinkowska et al.  designed MTDLs combining 5-HT6R antagonism, inhibitory effect against butyrylcholinesterase (BuChE)  and antioxidant properties , aiming at achieving both cognition-enhancing properties and neuroprotective activity. Both series were based on typical the 5-HT6R antagonist scaffold with a N-substituted 4-(piperazin-1-yl)-1H-indole core . The indole 1-position was occupied with a benzyl group for series 1 and a 2-chloro benzene moiety for series 2 (marked with blue in Figure 18). The piperazine was N-substituted with an alkyl-phthalimide moiety with variation of the length of the alkyl chain (see Figure 18). The phthalimide group (marked with yellow in Figure 18) is known for interacting with the hydrophobic pocket in BuChE . The indole moiety (marked with red in Figure 18), in addition to being important for 5-HT6R antagonism, had antioxidant properties: Indoles are known for their ability to capture free radicals and protect biological systems against peroxidation . From their screen they found that longer alkyl chains, connecting the phthalimide to the piperazine ring, decreased affinity for the 5-HT6R, while substituents on the benzyl group did not alter affinity significantly.
However, counteracting requirements within the scaffold complicated the design. From the BuChE inhibitor (BuChEI) screen they concluded that generally the unsubstituted benzyl analogues showed highest activity, with compound
Zajdel et al. recently  investigated MTDLs combining 5-HT6R inverse agonist activities and a monoamine oxidase B (MAO-B) inhibitory effect. The design included a 5-HT6R antagonist scaffold and fragments of either a reversible or an irreversible MAO-B inhibitor (see Figure 19) attached through an alkyl spacer of different lengths.
Their results indicate that the sulfone group was crucial for affinity to the 5-HT6R, while non-substituted phenyl groups (R = H) seemed to result in the best 5-HT6R binding. However, it is relevant to mention that no compounds containing the sulfone group together with the chloride-substituent were tested, making it difficult to make an overall conclusion. Among the 18 synthesized compounds, the very promising lead compound
In 2016 Grychowska et al.  published a study for a new core design based on a scaffold-hopping approach, with swapping of carbon and nitrogen atom in the indole ring , starting from the SSRI 6-nitroquipazine (
From a GPCR signaling assay it was determined that compound
In an attempt to obtain the desired MTDL, Grychowska et al.  designed a series of 11 compounds combining structural elements of the 5-HT6R antagonist
For several of the antagonists developed over the years, clinical trials seemed promising until the late studies. One example is the well-known non-sulfonyl compound Idalopirdine or Lu AE58054 (
4.3 Neutral antagonists and inverse agonists for the 5-HT6R
An important feature of the 5-HT6R is its ability to exist in different conformational states depending on the ligand bound, which can lead to initiation of different signal transduction pathways. The engagement of the 5-HT6R in several pathways has now been demonstrated. In addition to the canonical Gs adenylyl cyclase signaling pathway implicated in the control of neuronal migration , the 5-HT6R is also involved in pathways engaged in brain development of synaptic plasticity, more specifically the rapamycin  and cyclin-dependent kinase 5 (Cdk5) signaling pathways . Another relevant feature is the high level of constitutive activity of the 5-HT6R. The 5-HT6R has different pathways that can be activated upon different antagonistic and agonistic approaches, and the above-mentioned problems during clinical trials, stimulated interest for investigating other mechanistic approaches against the receptor.
It is worth having in mind that Cdk5-dependent neurite growth has been found to involve the 5-HT6Rs  and being agonist dependent. An inverse agonist of this signaling system, like SB-258585 (
Utilizing a scaffold-hopping approach based on swapping one carbon with a nitrogen atom in the indole ring, Vanda et al.  synthesized 33 compounds varying in both the position of the nitrogen, alkyl-substituents on the C2 position (R-group, Figure 23) and substituents on the benzyl group and based on biological testing, they made some general conclusions. Localization of the nitrogen was crucial for 5-HT6R affinity and compounds with the imidazole[4,5-b]pyridine fragment were in general the best binders. Elongation of the benzyl to a phenethyl group decreased affinity. Furthermore, while bulky and aromatic substituents were not tolerated in the C2 position, small alkyl substituents was in general accepted, with the ethyl-group being the most favored. Furthermore, substituents in the benzyl 3-position were generally preferred, while substituents in the 2- and 4-position lowered affinity compared to the non-substituted analogue. The studies resulted in identification of compound
4.4 Agonists for the 5-HT6R
Interestingly, it has been suggested that not only 5-HT6R antagonists but also 5-HT6R agonists may have pro-cognitive activities . The 5-HT6R agonist WAY-181187 (
Alzheimer’s disease is increasingly being recognized as one of the most challenging medical and social challenging health concerns in older people. To date, only treatments offering symptomatic relief to patients exist for this disease, limiting benefit to patients. As there is no curable medical treatment available, much effort has been focusing on identifying novel potential targets for drug development. The rich involvement of serotonin (5-HT) in both cognition and memory; some of the most symptomatic areas being affected in AD, has directed current drug discovery programs to focus on this system as a major therapeutic drug target.
Thus, serotonin receptor modulators offer an attractive option for a future treatment of AD patients and modulation of 5-HT4R has indeed demonstrated to improve neurotransmission and enhance the release of acetylcholine resulting in the memory formation. Furthermore, in various cell based and animal models, partial 5-HT4R agonists were demonstrated to promote the release of sAPPα and block the release of amyloid beta peptide. Remarkably, 5-HT4R agonists were also reported to induce neurogenesis in hippocampus as well as enteric system through the activation of cyclic AMP response element binding protein in rodents.
During the past 20 years, also the 5-HT6R has received increasing attention and is now a promising target for improving cognition. However, 5-HT6R functionality is much more complex. Several studies with structurally different compounds have shown that not only antagonists but also 5-HT6R agonists improve learning and memory in animal models. This paradoxical effect may explain why several compounds that reached phase III clinical trials failed to replicate the positive impact on cognition [76, 94]. Therefore, even though preclinical and clinical trials show that the 5-HT6R is a promising target for treatment of neurodegenerative diseases such as AD, there is an urgent need for a better understanding of the pathways involved in modulation of the receptor. However, there is hope that with the recent advances in molecular biological techniques, including improved cloning and sequencing methods, strategies for the development of in silico GPCR models, will advance our understanding of the molecular mechanisms underlying the impact of serotoninergic signaling in AD to provide beneficial treatments for AD.
Taken together, 5-HT4R and 5-HT6R modulators address all major facets of AD. However, although important progress has been made with developing relevant modulators to improve both cognition and memory, crucial challenges still need to be overcome before a promising cure to AD has been found. Most importantly, an in depth understanding of the pathways involved in modulation of the serotonin receptors is urgently needed. Also, to limit side-effects the identification of CNS specific molecules is crucial.
|ADME||absorption, distribution, metabolism, and excretion|
|APP||amyloid protein precursor|
|cAMP||cyclic adenosinde monophosphate|
|Cdk5||cyclin dependent kinase 5|
|ClogP||calculated logarithm of octanol/water partition coefficient|
|CNS||central nervous system|
|D3R||dopamine subtype 3 receptor|
|GPCR||G protein-coupled receptor|
|HBA||hydrogen bond acceptor|
|MAO-B||monoamine oxidase B|
|ROS||reactive oxygen species|
|sAPPα||soluble non-amyloidogenic form|
|SSRI||selective serotonin reuptake inhibitors|
|5-HT4R||5-hydroxytryptamine receptor 4|
|5-HT6R||5-hydroxytryptamine receptor 6|