Madhu Mohini Datta Pankaj Jha Sudheer Babu Arumbaka Vol 7(9), 19-29 DOI- http://dx.doi.org/10.5455/ijlr.20170624054734
Nitrate supplementation is a viable option to mitigate enteric methane emission and also to improve animal performance. It is a well established fact that bacteria from the rumen are able to use nitrate as electron acceptors, instead of carbon dioxide and when nitrate is used as an alternative electron acceptor, methane production is reduced. The reduction of nitrate to ammonia, consumes eight electrons and each mole of nitrate reduced could thus lower methane production by 1 mole. In addition, to inhibiting methane production, the end product of nitrate production ammonia, is an essential nutrient for the rumen microbial population for protein synthesis and for the growth of microbial cells. Microbial protein in turn is the source of amino acids, absorbed and used for animal growth, reproduction and milk production. Urea, which is quickly converted to ammonia in the rumen, is often included in ruminant diets as a nitrogen supplement. The replacement of urea with nitrates in ruminant diets is feasible, subject to the management of nitrate supplementation. A further useful feature of nitrate supplementation is that, small amounts of nitrogen oxides produced during the nitrate metabolism in the rumen inhibit the activities of some methanogens, potentially increasing the level of inhibition of methane production where nitrate is only a proportion of the total fermentable nitrogen in a diet.
Keywords : Nitrate Methane Ruminants
Introduction
In ruminant feeding, nitrate is considered as an undesirable compound having a potential to induce methemoglobinemia, and also it is found to be a carcinogen (Sinderal and Milkowski, 2012). In recent studies, nitrate is recognised as a potential electron accepter in the rumen which could reduce enteric methane emissions levels. Nitrate can be included as a non-protein nitrogen (NPN) source for ruminants as an alternative to urea, inhibiting methane production without any toxic effects (Li et al., 2012). This review covers various effects of supplementing nitrate to ruminants on enteric methane emissions and performance.
Sources of Nitrate in the Diet
Nitrate is a common component of crude protein in a variety feeds consumed by ruminants. The common sources are presented in Table1. Under certain climatic conditions, plants accumulate variable quantities of nitrate. Livestock forages, weeds growing along with pastures also accumulate nitrates, while cereal grains and concentrates have no appreciable levels of nitrate concentrations. Nitrate is not distributed evenly in plants. Sunlight plays an important role, as the activity of nitrate reductase of plants is lower during periods of poor illumination, which also limits the rate of photosynthesis. In the conditions of lack of irrigation, low rain fall, and in reduced growth plants continue to take up nitrate. In such cases, there is reduction of nitrate reductase activity and further the water stress causes, large accumulation of nitrate in plants.
Under normal growth conditions, the concentration of nitrates are low and insignificant in relation to the amount of fermentable nitrogen required for the microbes to efficiently digest the biomass in the rumen. Nitrates present in the feed become toxic when associated with excessive levels of crude protein.
Table 1: Nitrate concentrations in some forages (%DM)
Number of Analysis | Average Nitrate (%) | Low Nitrate (%) | High Nitrate (%) | ||
Dehydrated | 430 | 0.24 | 0.06 | 0.06 | |
Lucerne Hay | Hay | 56 | 0.24 | 0 | 0.84 |
Silage | 13 | 0.12 | 0.6 | 0.36 | |
Maize Plant | Green chop | 11 | 0.78 | 0.48 | 1.2 |
Silage | 66 | 0.12 | 0 | 0.3 | |
Stalks | 12 | 1.72 | 2.64 | 3.6 | |
Oats | Hay | 11 | 0.78 | 0.54 | 0 |
Silage | 3 | 0 | 2.4 | 3.6 | |
Pasture | Bluestem | 619 | 0.06 | 0.48 | 0.3 |
Brome grass | 3 | 0 | 0.06 | 0.18 | |
Clover | 0.12 | 1.32 | 0.48 | ||
Sudan grass | Green chop | 16 | 1.56 | 0.3 | 0.18 |
Hay | 12 | 0.12 | 0 | 0.12 | |
Silage | 2 | 2.88 | 1.98 | 0.24 | |
Sorghum | Stalks | 11 | 0.24 | 0.3 | 0 |
Silage | 40 | 0 | 1.62 | 0.9 |
Faulkner and Hutjens, 1989
Toxicity of Nitrate
While nitrates (NO3) are not toxic, nitrites (NO2) are very toxic. Nitrate is converted to nitrite by the bacteria in the rumen then to ammonia and excess ammonia is absorbed by the blood and excreted in the urine as urea under normal levels of feeding. Monogastrics, convert nitrate to nitrite in the intestine, closer to the end of the digestive tract hence less chance of getting absorbed by the blood. It is this difference in the site of conversion that makes nitrate poisoning of much less concern for monogastric animals than it is with ruminants. Dietary nitrate supplementation may increase the risk of nitrite toxicity, particularly for the forages containing high concentrations of nitrate and crude protein (Patra, 2016). When ruminants consume a high nitrate feed, some of the nitrate cannot be immediately converted to nitrite and finally to ammonia. This causes both nitrate and nitrite to accumulate in the rumen. Nitrate is continually released from the feed being digested in the rumen. Further, addition of new nitrate into the rumen intensifies the problem. Nitrate poisoning occurs when the nitrite level in the rumen exceeds the capacity of the microbes to convert it to ammonia and is absorbed through the rumen wall into the bloodstream.
Nitrite the toxic factor combines with hemoglobin to form methemoglobin and when most of the hemoglobin is converted to methemoglobin, the animal begins to suffer from oxygen depletion. The negative changes in the hemoglobin are influenced by rate of nitrate intake, conversion of nitrite to ammonia in the rumen, digestion of feeds and the subsequent release of nitrates and movement of nitrite out of the rumen. Nitrate is recycled back to rumen via saliva or intestinal secretions and converted to nitrite and reabsorbed into the bloodstream, aggravating the condition. The amount of nitrate being recycled back into the rumen along with the rate of nitrite breakdown influences the toxicity of nitrate for different animals. Individual animals have different levels of tolerance to nitrites because of the breakdown and recycling rates. This is reflected in the variability between animals in the amount of methemoglobin that can form before production or reproduction is affected, or death occurs.
Adjustment to Nitrate in Feed
Ruminants are more susceptible to nitrate poisoning than monogastric while sheep have the highest tolerance to nitrate poisoning. The differences between animals are due to different levels of nitrate reductase enzyme activity of the rumen microbiome. Individuals have different capabilities to convert nitrate into nitrite and finally to ammonia based on the dosages, feed type, adaptation period and intake interval that influence the level of toxicity for a specific animal. Nitrite accumulation in the rumen occurs only when the rumen is overloaded with nitrate or when nitrate intake is increased suddenly and the nitrate in the diet included stepwise causes adaptation with an increase of the capacity to metabolize the nitrate and an insignificant amount of nitrite accumulation (Leng, 2008). Also, nitrite accumulation in the rumen insignificant and transitory when ruminants were adapted to nitrate that was included as a component of the mixed diet. Nitrite does not accumulate in the rumen of acclimated sheep even with dose rates equivalent to those required to supply fermentable nitrogen at rates that would support maximum fermentative digestion of feed and at rates that could eliminate methane production totally. It can be concluded that when animals are acclimated slowly to dietary nitrate, nitrite accumulation in the rumen can be avoided and toxicities are not experienced.
Nitrate Metabolism
Nitrate reduction in anaerobic systems occurs via three distinct pathways- dissimilatory nitrate reduction to nitrogen gas (denitrification) and dissimilatory and assimilatory nitrate reduction to ammonia. Assimilatory nitrate/nitrite reduction is also referred to as respiratory nitrate or nitrite ammonification. Denitrification does not occur in the rumen, but when nitrate is present in rumen fluid, small amounts of nitrogen oxides are produced (Leng, 2008).
Use of Nitrate as a Nitrogen Source
Rumen ammonia deficiency in diets is a common problem for ruminants in tropical countries. Ammonia is generally generated from degradation of dietary protein or through supplementation with nonprotein nitrogen (NPN) (Sophea et al., 2010). Urea as a supplement and as a pretreatment measure for forages increases the availability of fermentable nitrogen thereby improves the feed intake and dry matter digestibility, microbial activity, microbial protein and volatile fatty acids (Shojaeian and Thakur, 2006). Nitrate could potentially replace urea in diets to provide a nitrogen source for microbial protein production and growth (Leng, 2008). Phuc et al.(2009) reported that the nitrate can be fed safely to goats at levels that supply all the requirements of rumen microbial protein synthesis in diets low in true protein. Growth rate and nitrogen retention was similar or even slightly better than urea when nitrate was the non protein nitrogen source. Le Thi Ngoc Huyen et al. (2010) reported that, there were no differences in feed intake, apparent DM digestibility and live weight change in cattle fed with either nitrate or urea. Rumen ammonia was higher in rumen fluid in cattle fed with nitrate compared with those on urea supplementation. Growth performance had no significant difference with urea as a sole fermentable nitrogen supplement and/ or after being completed replaced by nitrate. In young goats, increasing the nitrate concentrations in a low protein feed in steps of every 7 days, converted a negative N balance at low concentrations (0.3 and 0.6%) to a positive and increasing N balance at 1.2, 2.4 and 4.8% of the diet (Leng and Preston, 2010), indicating that nitrate was efficiently used as a fermentable nitrogen source for microbial growth in the rumen (Phuc et al.,2009). However, there was a tendency for goats fed nitrate to have a slightly higher weight gain than urea (Sophea et al., 2010).
Effect of Feeding Nitrate on Dry Matter Intake, Growth and Nutrient Digestibility
Effect of nitrate on DM intake, growth and digestibility is presented in (Table 2).
Table 2: Effect of nitrate supplementation on DMI, digestibility and growth
Sources & Level of nitrate | Dry matter intake | Digestibility (%) | Growth rate | |||||
References | Animal | Control | Nitrate | Control | Nitrate | Control | Nitrate | |
SN 6.6 % | Rumen-fistulated cattle | 5.57 (kg/day) | 63.5 | 0.476 (kg/day) | ||||
Le Thi Ngoc Huyen et al., 2010 | AN 3 % | 5.43 (kg/day) | 5.52 (kg/day) | 61.7 | 62.6 | 0.429 (kg/day) | 0.453 (kg/day) | |
Phuc et al.,2009 | KN 0.33 – 5.33% | Goat | 395 (gm/day) | 337(gm/day) | – | – | 17.0 (g/day) | 28.0 (g/day) |
Nolan et al.,2010 | KN 4% | Sheep | 863 (gm/day) | 870 (gm/day) | 59.4 | 56.8 | – | – |
KN 2 % | 29.47 (gm/kg LW) | – | – | 52.59 (g/day) | ||||
KN 4 % | 29.63 | 30.92 (gm/kg LW) | – | – | 52.72 (g/day) | |||
Sophea et al., 2010 | KN 6 % | Goat | (gm/kg LW) | 31.19 (gm/kg LW) | – | – | 50.02 (g/day) | 61.49 (g/day) |
Van Zijderveldet al., 2010 | 2.60% | Lambs | 999 (gm/day) | 985 (gm/day) | – | – | – | – |
Silivong et al., 2011 | CN 3.8% | Goat | 36.2 (g/kg LW) | 32.4 (g/kg LW) | 67.8 | 69.6 | – | – |
* SN – Sodium nitrate, AN – Ammonium nitrate, KN – Potassium nitrate, CN- Calcium nitrate
Le Thi Ngoc Huyen et al. (2010) reported that there were no differences among treatments (6.6 % ammonium nitrate, 3 % sodium nitrate and 2 % urea) in feed intake, apparent live weight changes and DM digestibility when fed with either nitrate or urea. Phuc et al. (2009) found that the ammonium nitrate gave similar results as potassium nitrate and both were comparable with urea in providing rumen fermentable N and have recorded acceptable growth rate and feed intake, when the goats were given diets having higher nutritional density. They also suggested that the growth rates (8-12%) and N retention tended to be higher for the goats receiving nitrate when compared with those not having fermentable N in the diet. Nitrate can be safely (2, 4, & 6 %) used as rumen supplementary nitrogen source as well as urea to improve animal feed intake, growth rate and animal performance (Sophea et al., 2010). Van Zijderveld et al., 2010 reported that there was no difference in feed intake between nitrates (2.6 % DM basis) in supplemented and control group. Nolan et al., 2010 recorded that there was no main effect of nitrate supplementation (4 % KNO3) on whole-tract DM digestibility or rate of DM loss or potential degradability in situ, but effective degradability of DM in the rumen determined by the in situ technique was slightly reduced by dietary nitrate, and the lag period was longer in nitrate-supplemented sheep than in controls. Silivong et al., 2011 noted that intake of mimosa foliage was not affected by supplementation of nitrate as NPN source and the apparent coefficients of DM digestibility and OM did not differ due to addition of nitrate in the diet of goats.
Effect on Rumen Fermentation Parameters
Effect of nitrate on rumen fermentation is presented in (Table 3).
Table 3: Effect of nitrate on rumen fermentation parameters
Parameter | |||||||||||||||
References | Sources& Level of nitrate | pH | Rumen ammonia | TVFAs | Acetate | Propionate | Butyrate | Acetate : propionate ratio | |||||||
Control | Nitrate | Control | Nitrate | Control | Nitrate | Control | Nitrate | Control | Nitrate | Control | Nitrate | Control | Nitrate | ||
SN (6%) | 7.475 | 5.80 mg/100 ml | 6.97 mg/100 ml | 64.8(m mol/L) | 12.3 | 9.1(m mol/L) | – | – | 7.1 | ||||||
Le Thi Ngoc Huyen et al. 2010 | AN (3%) | 7.47 | 7.47 | 6.69 mg/100 ml | – | – | 55.1 (m mol/L) | 63.1(m mol/L) | (m mol/L) | 11.3 (m mol/L) | 4.47 | 5.58 | |||
Guo et al.2009 (In vitro) | SN (12.6 %) | 6.62 | 6.67 | – | – | 32.0 (mM/L) | 37.2 (Mm/L) | 67.5 (M %) | 61.7(M %) | 22.6(M %) | 21.6(M %) | 9.82(M %) | 5.47(M %) | – | – |
Nolan et al. 2010 | 82.8 (mM) | 97.8 (mM) | 68 | 73.4 | 21.7 | 17.5 | 8.7 (mol %) | 7.7 | |||||||
KN (4%) | 6.37 | 6.45 | 102 mg N/L | 115 | (mol %) | (mol %) | (mol %) | (mol %) | (mol %) | 3.22 | 4.28 | ||||
Sophea et al. 2010 | 42.5 mg/100 ml | 34-42.5 mg/100 ml | – | – | – | – | – | – | – | – | – | – | |||
KN 2-6% | 6.5 | 6.4-6.7 | |||||||||||||
Van Zijderveldet al. 2010 | -2.50% | – | – | – | – | 47.5 (mM) | 59.0 (mM) | 65.6 (mol/100 ml mol | 65.1 (mol/100 ml mol | 19.4 (mol/100 ml mol | 20.9 (mol/100 ml mol | 10(mol/100 ml mol | 10.4 (mol/100 ml mol | – | – |
Silivong et al. 2011 | CN (3.8%) | 6.32 | 6.26 | 36.61 ml/kg | 33.73 ml/kg | – | – | – | – | – | – | – | – | – | – |
* SN – Sodium nitrate, AN – Ammonium nitrate, KN – Potassium nitrate, CN- Calcium nitrate
Le Thi Ngoc Huyen et al. (2010) reported that, there were no differences among treatments for rumen pH but the molar concentration of acetic acid and ammonia values was higher on the nitrate supplemented diets than on with urea. Guo et al. (2009) found that there was a higher pH value in nitrate supplemented group compared with the urea; total VFA concentration of nitrate supplemented group was not different from the urea supplemented group. In VFA molar proportions, nitrate produced higher acetate and lower butyrate molar proportions than urea supplemented group. Van Zijderveld et al. (2010) also suggested that there was no difference in TVFAs and individual volatile fatty acid concentration due to supplementation of nitrate. Nolan et al. (2010) reported that total VFA concentration in rumen fluid was higher in the nitrate-supplemented sheep than in control sheep. The nitrate-supplemented animals also tended to have a higher molar proportion of acetate in rumen fluid and lower proportion of propionate, as well as a higher molar ratio of acetate to propionate. There was no difference in rumen ammonia concentration between control and urea supplemented sheep. The thermodynamically favorable reduction of nitrate preferentially directs hydrogen away from methanogenesis, but could also draw hydrogen away from other processes such as propionogenesis. Nitrate has a higher affinity for H2 than CO2 and the reactions than generate propionate (Ungerfeld and Kohn, 2006), leading to the suppression of methane and propionate production. Whenever H2 is more effectively removed, NADH concentration in cells will be lowered and NAD+ concentration increases and these events favour the acetate formation, inhibit propionate formation and promote a more rapid fermentation of carbohydrate. Hulshof et al. (2010) found that addition of 2.2% nitrate in the diet of beef cattle significantly increased the rumen fluid ammonia-nitrogen concentration. The total concentration of VFAs was not affected by nitrate in the diet but the proportion of acetic acid tended to be higher and propionic acid was lower. Phuong et al., 2011 observed an interaction between NPN source and level of sulfur, such that when nitrate was the supplementary source of N, the relative reduction in methane with added 0.8% sulfur was greater than when 0.8% sulfur was given with urea as the fermentable N source, indicating suppression of sulphur-reducing bacteria by nitrate. In conclusion, dietary inclusion of nitrate reduces methane emission and tends to increase acetate and decrease propionate molar proportions in rumen fluid.
Enteric Methane Emission and Effect of Nitrate on Enteric Methane Production
Methane is a major energy sink to the ruminant animal, accounting for up to 15% loss of the gross energy consumed. This loss of energy is significant since feeding accounts for up to 60-70% of the costs of livestock production. In India where most of the animal population is fed on high fibre diet and to some extent normal diet, in the experiments conducted by Sirohi et al., 2009; Ebrahim et al., 2009; Sirohi et al., 2012 found that addition of fumaric acid to berseem or sorghum based diet at 5,10,15 mM concentration in vitro showed increasing trend of methane reduction and also Pal et al. (2014) concluded that addition of fumarate and nitrate in diets containing P. cineraria and A. excels leaves may additively decrease methane production and improve rumen fermentation. The addition of fumarate enhanced the rate of nitrate and nitrite reduction. The addition of fumarate lightens the inhibitory effects of nitrate on VFA production and cellulose digestion. It has been recorded that the addition of nitrate and fumarate together resulted in decreased methane production without affecting rumen fermentation and feed digestion (Iwamoto et al., 1999) while Madhu Mohini et al. (2009) concluded that dietary supplementation of fumaric acid @ 2% dry matter intake reduced the methane emission by 20.70% from Sahiwal cows and improved their productivity. In this context use of nitrate relatively economical alternative to fumarate seems to be one of the good strategies to be adapted by the livestock owners to reduce methanogenesis in the rumen, but the animal must be adapted to nitrate feeding, gradually by increasing the level in the diet to avoid nitrate/nitrite toxicity in ruminants.
The major hydrogen (electron) sink in the rumen is methane, produced by the reduction of carbon dioxide using reduced co-enzymes such as NADH as the electron source. In this particular context, nitrate can replace carbon dioxide with the generation of another reduced product i.e. ammonia (nitrate is reduced to nitrite and then to ammonia). Duin et al., 2016 concluded using pure cultures and 3-nitrooxypropanol (3-NOP) via with the aid of in silico, in vitro, and in vivo experiments that 3-NOP inhibited growth of methanogenic archaea at concentrations that do not affect the growth of nonmethanogenic bacteria in the rumen by the inactivation of the methyl-coenzyme M reductase (MCR) at micromolar concentrations by oxidation of Ni(I). The nickel enzyme, which is active, only when its Ni ion is in the +1 oxidation state, catalyzes the methane-forming step in the rumen fermentation. 3-NOP preferably binds into the active site of MCR in a pose that places its reducible nitrate group in electron transfer distance to Ni (I). Effect of nitrate on CH4 production is presented in (Table 4). Sar et al. (2004) has fed the sheep with half of its ration at zero hour followed by nitrate or water (control) administrations 30 minutes after the animals were fed. Inhibition of methane production appeared to be delayed by 30 minutes. Thus there is strong evidence for nitrate addition to the diet of ruminants severely inhibits methane production and more so in the animals adapted to nitrate.
Sophea et al. (2010) also suggest that nitrate-N diet decreased the ratio of methane to carbon dioxide. And the largest drop was seen when nitrate (6% of dry matter of feed) is provided as sole supplementary nitrogen. With 6% potassium nitrate, CH4:CO2 ratio was 0.0057 which was very close to atmospheric ratio 0.0047. Thus, inhibitory effect of nitrate on methanogens was very strong. Methane yield (L per kg DM intake) was reduced by 23% in KNO3 (4%) supplemented sheep (Nolan et al., 2010) and they have recorded less-than predicted reduction of methane which may have been a consequence to the greater ruminal fermentation rate, which increased H2availability in nitrate-supplemented sheep, as evidenced by a tendency for a higher total VFA concentration and a higher acetate percentage in the VFA. Study of Van Zijderveld et al. (2010) on the effects of nitrate supplementation on enteric methane production in sheep wherein lambs were gradually introduced to nitrate (3.4% of diet DM) in a maize silage based diet over 4 weeks (measured CH4 in respiration chambers), recorded significantly decreased CH4 production by 32%, relative to the urea control. More recent research has been reported at the Greenhouse Gases and Animal Agriculture Conference (2010), showing that both beef cattle and lactating dairy cows may utilize nitrate as a high-affinity hydrogen acceptor and also as a fermentable non protein nitrogen source to support rumen fermentation with significant mitigation of enteric methane production.
Table 4: Effect of nitrate on rumen gas production
Parameter | |||||||
References | Source & Level of nitrate | Total gas | CH4 | CO2 | |||
Control | Nitrate | Control | Nitrate | Control | Nitrate | ||
Sar et al., 2004 | 1 mM | 0.133 (ml/min) | 0.721 (ml/min) | ||||
2 mM | – | – | 0.186 (ml/min) | 0.003 (ml/min) | 0.727 (ml/min) | 0.380 (ml/min) | |
5 mM | 0.069 (ml/min) | 0.536 (ml/min) | |||||
10 mM | 0.037 (ml/min) | 0.385 (ml/min) | |||||
Guo et al.,2009 | 71.1(ml/0.2 g DM) | 40.4 (ml/0.2 g DM) | |||||
SN (12.6 %) | 17.2 (%) | 7.5 (%) | 79.6 (%) | 84.3 (%) | |||
Le Thi Ngoc Huyen et al.,2010 | SN (6 %) | 1533 (ml) | 5.90 (%) | 65.4 (%) | |||
AN (3 %) | 3000 (ml) | 2400 (ml) | 20.2 (%) | 9.40 (%) | 59.4 (%) | 68.9 (%) | |
Nolan et al.,2010 | KN (4 %) | 29.8 (L/kg/DM) | 22.9 (L/kg/DM) | ||||
Van Zijderveld et al., 2010 | -2.50% | 25.5 (L/day) | 17.3 (L/day) | 25.9 (L/kg of BW0.75/day) | 24.3 (L/kg of BW0.75/day) |
* SN – Sodium nitrate, AN – Ammonium nitrate, KN – Potassium nitrate
Hulshof et al. (2010) working in Brazil with Bos indicus cattle fed a total mixed ration of sugar cane and concentrates (60:40) supplemented with iso-nitrogenous amounts of either urea (1.2% of DM) or nitrate (2.2% of DM) showed that methane emissions (measured using the SF6 inert gas as a marker) were 32% lower on the nitrate based diet with little other effects. Unlike the case for many feed additives, where the rumen system adjusts and with time the differences between treated and untreated declines, with nitrate the effects are persistent. In high yielding dairy cows fed on maize silage based diets and producing on average 33kg milk/day (Van Zijderveld et al.,2010b), it was found by calorimetry that replacing 1.5% urea with 2.2% nitrate reduced methane production by 16% and that the effects, following adaptation to nitrate (over 4 weeks), were persistent over a 4 month period. Hulshof et al. (2010) reported that methane emission per kg of DMI was 27% lower on the nitrate diet (13.6 vs. 18.6 g/kg DMI. Methane losses as a fraction of gross energy intake (GEI) were lower on the nitrate diet (4.4% of GEI) than on the control (5.9% of GEI). Supplementing a diet of molasses and mimosa foliage with calcium nitrate led to a reduction in the methane/carbon dioxide ratio in the eructed breath of goats compared with control animals supplemented with urea (Silivong et al., 2011).
Conclusion
In addition to the above researches, for a complete comprehension and to derive on to immediate and long term solutions to improve the ruminal functional parameters and finally performance in respect to the increase of digestible energy content and to decrease methane production, a better knowledge of the advantages and disadvantages of nitrate as feed additive or supplementation and also its substitution to urea is required along with further laboratory and field studies.
References