Int J Adv Life Sci Res, Vol 1, Issue 1, 53-66Research Article


Effectiveness of Pharmacists’ Intervention in Improving Patients Knowledge and Attitude Towards Antibiotic Usage in Klinik Kesihatan Seremban, Malaysia

Payal Choudhury* , Zulawati A, Lailatul Munirah I, Ong KS, Shahnaz B

Pharmacy Department, Seremban Health Clinic, Seremban Health District Office,
Negeri Sembilan State Health Department, Malaysia
zw262@yahoo.com; kingsien90@gmail.com; nahs1978@gmail.com
*Correspondence : Email: payalpc@gmail.com; Phone: +067679267 ext 141

Abstract

Introduction: Antibiotics are an important tool in the battle against infections. Antimicrobial resistance is an increasing problem; hence the appropriate use of antibiotics is significant in the primary healthcare system in Malaysia.

Objective: 1) To understand community’s knowledge and attitude towards antibiotic use.

2) To determine the effectiveness of pharmacist intervention in proper antibiotic use.

Method: A cross-sectional, prospective study with convenient sampling using a pre-validated guided questionnaire was conducted in September 2016 for pre-data collection. Implementations of remedial measures were carried out in October 2016.

Results: Nearly 54.3% of the respondents had moderate knowledge, 36.8% had good knowledge before and 95.53% after intervention. 99.47% could identify that antibiotics are indicated for the treatment of bacterial infections compared to before intervention (55.79%). However, 67.4% incorrectly thought that antibiotics are used to treat viral infections which reduced to 4.74% after intervention. Generally, respondents had a negative attitude before intervention with mean 5.57 compared to after intervention (7.22). Paired T-test obtained shows that pharmacy intervention had a positive impact in improving patient’s knowledge, the mean score of 11.19 compared to pre-intervention 6.74 with significance (P-value = 0.000).

CONCLUSION: Pharmacist intervention in improving patient’s knowledge and attitude was found to be successful towards antibiotic use.

KEYWORDS: antibiotic; public knowledge; attitudes; Malaysia

NMRR ID: NMRR-16-1344-31362

Introduction

Antibiotics are the cornerstone of treatment for bacterial infections but antibiotic resistance is a growing problem which is the biggest threat globally. Antibiotic resistance occurs when bacteria changes in response to the use of these medicines. Bacteria and not humans, develop antibiotic resistance. Humans that are infected by these bacteria will be tougher to treat than non-resistant bacteria. A prospective survey of antibiotic prescribing patterns in six Ministry of Health general hospitals in Malaysia was done (Hassan et al., 1994). Information on antibiotic prescriptions including the type of antibiotics used and the purpose for prescription was obtained through questionnaires (Hassan et al., 1994). A total of 1,918 antibiotic prescriptions were available for analysis (Hassan et al., 1994). About two-thirds of prescriptions were for therapeutic purposes5. The most common infections treated were lower respiratory infection (31%), followed by skin and soft tissue infection (17%), and urinary tract infection (8%) (Hassan et al., 1994). Only 20% of the therapeutic prescriptions were based on microbiological test results (Hassan et al., 1994). Antibiotics are becoming less effective due to resistance as a growing number of infections such as pneumonia, tuberculosis, and gonorrhea are becoming difficult to treat. Antibiotic resistance leads to greater medical costs, increased mortality and prolonged hospital stays. Antibiotic resistance is mounting dangerously worldwide. Globally new resistance mechanisms emerge daily and the widespread threatens our capacity to treat common infections. Misuse of antibiotics by the general community is documented to have an important role in increasing antibiotic resistance. Therefore, the appropriate use of antibiotics is significant in the primary healthcare system in Malaysia.

Materials & Methods

This study was conducted by the pharmacist in the pharmacy unit of Klinik Kesihatan Seremban (Seremban Health Clinic), PKD Seremban. The population of this study consists of patients visiting the pharmacy of Klinik Kesihatan Seremban (Seremban Health Clinic). An average patient per month that visits the pharmacy is approximately 17678.

A self- administered pre and post-experimental were used for this study. This research used a pre-validated guided questionnaire to obtain the data and information. The duration of this study was from September 2016 to November 2016. Only questionnaires with complete responses were analysed.

Approval from the Malaysian Research Ethics Committee (MREC) and Clinical Research Centre through National Medical Research Register (NMRR) was sought prior to the study. The Registered NMRR number is NMRR-16-1344-31362. All information from the data collection is kept private and confidential.

A sample size of the study was determined using the Raosoft sample size calculator which is approximately 380 patients that participated. Participants were recruited by our research team and a random convenience sampling method was adopted for this study, regardless of the acquisition at the time of visit or antibiotic use at any time of life.

The inclusion criteria are 1) all patients’ visiting pharmacy Klinik Kesihatan Seremban (Seremban Health Clinic), during working hours (8 am – 5 pm); 2) Aware of the term antibiotic; 3) Patient age 10 years and above; and 4) Consented patients. The exclusion criteria consist of 1) Integrated Drug Dispensing System (SPUB); 2) Dental patients; and 4) Healthcare professionals.

Data was collected by distributing questionnaires to randomly selected people with convenience sampling method during the patients' visit to the pharmacy. In detail, each selected patient was given a questionnaire to be answered. Data was collected for the pre-data collection and education through self-designed pamphlets and counseling was given to those patients. Then, the same questionnaires were given to the same sample after some time (one month) for the post-data collection after the intervention is done. Minimum time of one month is selected for patient convenience (same time as their medication collection).

Descriptive statistics, including percentages, means, and standard deviations will be used to summarize the responses to all variables as well as paired t-test for analysis of variance. The 95% confidence interval was set for the test whereby the result is significant if p ≤ 0.05. Data will be analysed by using SPSS.

Results

Reliability analysis is commonly done to test the reliability, stability and the consistency of results in representing the actual characteristics with the help of the measurement tools of the questionnaire. The ideal test of reliability analysis in SPSS analytical software is Cronbach’s Alpha as it is widely used and discussed by researchers. There are so many studies discussing the minimal threshold of Cronbach’s Alpha (Zikmund,2013). However, the widely accepted threshold is at 0.7. The value of Cronbach’s Alpha is 0.7 or more can confirm that the questionnaire construct is good and reliable (Badeloder,2013). As per Table 1, the overall Cronbach’s Alpha is 0.787 (>0.7) is good.

Table 1: Reliability Statistics

Cronbach's Alpha N of Items
0.787 12

Table 2 shows the demographic characteristics of 380 patients that fit the inclusion criteria of the study. The range of patients’ age is from 7 to 85 years. Majority of patients were females (N=220; 57.9%) compared to males (N=160; 42.1%).

There was 328 patients (86.3%) aged < 65 years whereas 52 patients (13.7%) aged ≥ 65 years, which were categorized as elderly and the highest respondent from the age group of 18 to 30 years old which accounted for 62.9%. The ethnicities of the patients were: Malay (45.3%), Chinese (17.6%), Indian (34.2%) and others (1.6%).

Normality test was performed with p=0.07 which is greater than 0.05, hence the data was normally distributed.

Table 2: Demographic data

Demography No. of patients – N: 380 (100%) P-Value*
Age (Mean: 33.19, SD: 1.690)
7-19 years 76 (20.0)
20-29 years 74 (19.5)
30-39 years 89 (23.4) 0.203
40-49 years 35 (9.2)
50-59 years 54 (14.2)
≥ 60 years 52 (13.7)
Gender
Males 160 (42.1) 0.043
Females 220 (57.9)
Ethnicity
Malay 172 (45.3)
Chinese 67 (17.6)
Indian 130 (34.2) 0.020
Bumiputera 5 (1.3)
Others 6 (1.6)
Marital Status
Married 212 (55.8)
Single 152 (40.0) <0.001
Divorcee/Widowed 16 (4.2)
Highest Education Level
No Formal Education 6 (1.6)
Primary Education 33 (8.7) 0.170
Secondary Education 184 (48.4)
Tertiary Education 157(41.3)
Occupation Sector
Government 67 (17.6)
Private 85 (22.4)
Self-employed 45 (11.8) <0.001
Unemployed 28 (7.4)
Student 83 (21.8)
Retired 28 (7.4)
Housewife 34 (8.9)
Others 10 (2.6)
*Chi-square test

Knowledge of Antibiotic

The majority of respondents, 54.3% (n=206) had a moderate knowledge on antibiotic usage followed by 36.8 (n=140) that had good knowledge of antibiotics with a median score of 7 (SD =2.42) (Table 3).

Table 3: Level of Knowledge (Pre)

Level of knowledge Total score Number (%)
Poor 0-3 34 (8.9)
Moderate 4-7 206 (54.3)
Good 8-12 140 (36.8)

Post-test results show an increase in the level of knowledge where good 95.53% (n=363) (Table 4), moderate 3.68% (n=3.68) and poor 0.79(n=3) which is reduced by 81%. Statistically, a significant difference in knowledge level was noted between gender (p= 0.04), race/ ethnicities (p=0.02), marital status and occupational sector (p<0.001). There was no significant difference in knowledge with the

age of respondents (p=0.207) and level of education (p=0170) (Table 2).

Table 4: Level of Knowledge (Pre)

Level of knowledge Total score Number (%)
Poor 0-3 3 (0.79)
Moderate 4-7 14 (3.68)
Good 8-12 363 (95.53)

Based on the statement given in the survey, most of the respondents answer correctly on the role of antibiotic is to kill bacteria, 55.79% (n=212), the post-test result shows an increase of almost 100% (n=378). However, 67.4% (n=256) of the respondents also answered pre-test and 95.26 (n=362) answered that antibiotic is used to kill viral. Nearly half of the respondents 61.3% (n=233) knows that antibiotic can kill bacteria that normally live in the gut and skin and 51.1% (n=194) respondents know that these bacteria are good for health. In terms of the adverse effect of antibiotics, 58.4% (n=222) respondents answer that antibiotic can cause allergic reactions (Table 5 & Table 6).

However, less than 50% of respondents answer antibiotic does not cause side effects and overuse of antibiotic does not cause antibiotic to lose its effectiveness during pre-test and increased to 937% (n=356) and 96.3% (n=366) respectively.

In regards to the knowledge of administration, only 52.9% (n=201) answered it was not appropriate to stop taking antibiotics when symptoms were improving, and 61.1% (n=232) respondents answered no for the statement “taking less antibiotic than prescribed is healthier than taking full course prescribed”.

Table 5: Percentage of the answer based on knowledge statement (Pre)

Role of Antibiotic Correct answer n (%) Incorrect answer n (%) Not sure n (%)
1. Antibiotic are medicine that can kill bacteria 212 (55.79) 61 (16.05) 107 (28.16)
2. Antibiotic can be used to treat viral infection 113 (29.7) 256 (67.4) 11 (2.9)
3. Antibiotic work on most cold and cough 237 (62.4) 118 (31.1) 25 (6.6)
Good Bacteria
4. Antibiotic can kill bacteria that normally live on skin and gut 233 (61.3) 69 (18.2) 78 (20.5)
5. Bacteria that normally live on the skin and in the gut are good for your health 194 (51.1) 141 (37.1) 45 (11.8)
Identification of antibiotic
6. Penicillin is an antibiotic 214 (56.3) 101 (26.6) 65 (17.1)
7. Antibiotic are the medication used to relieve pain and fever 184 (48.4) 142( 37.4) 54 (14.2)
Adverse effect
8. Antibiotic may cause allergic reactions 222 (58.4) 108 (28.4) 50 (13.2)
9. Antibiotic does not cause side effects 183 (48.16) 170 (44.7) 27 (7.14)
10. Overuse of antibiotics can cause antibiotic to lose effectiveness in long term 167 (43.95) 132 (34.7) 81 (21.35)
Administration of antibiotic
11. It is okay to stop taking antibiotic when symptoms are improving 201 (52.9) 172 (45.3) 7 (1.8)
12. Taking less antibiotic than prescribed is more healthy than taking the full course prescribed 232 (61.1) 137 (36.05) 11 (2.85)

Table 6: Percentage of the answer based on knowledge statement (Post)

Role of Antibiotic Correct answer n (%) Incorrect answer n (%) Not sure n (%)
1. Antibiotic are medicine that can kill bacteria 378 (99.47) 2 (0.53) 0
2. Antibiotic can be used to treat viral infection 362 (95.26) 18 (4.74) 0
3. Antibiotic work on most cold and cough 322 (84.7) 58 (15.3) 0
Good Bacteria
4. Antibiotic can kill bacteria that normally live on skin and gut 372 (97.9) 8 (2.1) 0
5. Bacteria that normally live on the skin and in the gut are good for your health 355 (93.4) 25 (6.6) 0
Identification of antibiotic
6. Penicillin is an antibiotic 347 (91.3) 33 (8.7) 0
7. Antibiotic are the medication used to relieve pain and fever 32.8 (86.3) 52 (13.7) 0
Adverse effect
8. Antibiotic may cause allergic reactions 364 (95.8) 16 (4.2) 0
9. Antibiotic does not cause side effects 356 (93.7) 24 (6.3) 0
10. Overuse of antibiotics can cause antibiotic to lose effectiveness in long term 366 (96.3) 14 (3.7) 0
Administration of antibiotic
11. It is okay to stop taking antibiotic when symptoms are improving 338 (88.9) 42.(11.1) 0
12. Taking less antibiotic than prescribed is more healthy than taking the full course prescribed 349 (91.8) 31(8.2) 0

Attitude towards antibiotic usage

The positive attitude was observed based on the percentage of the correct answer on attitude statement for “I will take antibiotic according to the instruction on the label and I normally will look at the expiry date of antibiotic before taking it” in Table 7. Statement 1 to 6 showed negative attitude by respondents. The post-test result (Table 8) shows a positive attitude towards proper antibiotic use. Upon statistical analysis, there is no correlation between the knowledge statements on the administration of antibiotic with attitude statement, which state that it is okay to stop taking antibiotic once symptoms are resolved (r=0.223, n=380, p<0.001).

Table 7: Percentage of Answer Based on Attitude (Pre)

Statement Agree n (%) Disagree n (%) Not sure n (%)
1.When I get a cold, I will take antibiotics to help me get better more quickly 195 (51.3) 178 (46.8) 7 (1.8)
2.I expect antibiotics to be prescribed by my doctor if I suffer from common cold symptoms 194 (51.1) 180 (47.4) 6 (1.6)
3.I normally stop taking an antibiotic when I start feeling better 184 (48.4) 191 (50.3) 5 (1.3)
4.If my family member is sick I usually will give my antibiotic to them 281 (73.9) 97(25.5) 2 (0.6)
5.I normally keep antibiotic stock at home in case of emergency 282 (74.2) 97 (25.5) 1 (0.3)
6.I will use leftover antibiotics for a respiratory illness 297 (78.2) 81 (21.3) 2 (0.5)
7.I will take antibiotic according to the instruction on the label 346 (91.1) 33 (8.7) 1 (0.3)
8.I normally will look at the expiry date of antibiotic before taking it. 340 (89.5) 37 (9.7) 3 (0.8)

Table 8: Percentage of Answer Based on Attitude (Post)

Statement Agree n (%) Disagree n (%) Not sure n (%)
1.When I get a cold, I will take antibiotics to help me get better more quickly 42 (11.1) 338 (88.9) 0
2.I expect antibiotics to be prescribed by my doctor if I suffer from common cold symptoms 51 (13.42) 329 (86.58) 0
3.I normally stop taking an antibiotic when I start feeling better 76 (20) 304 (80) 0
4.If my family member is sick I usually will give my antibiotic to them 13 (3.4) 367 (96.6) 0
5.I normally keep antibiotic stock at home in case of emergency 49 (12.9) 331 (87.1) 0
6.I will use leftover antibiotics for a respiratory illness 21 (5.5) 359 (94.5) 0
7.I will take antibiotic according to the instruction on the label 349 (91.8) 31(8.2) 0
8.I normally will look at the expiry date of antibiotic before taking it. 363 (95.5) 17 (4.5) 0

Paired T-test shows that mean for patients’ knowledge from this study obtained is 4.45 with a standard deviation 2.066 (p < 0.0001). Mean value for knowledge obtained for pre-test is 6.74 (SD=2.424) and post-test 11.19 (SD=1.069). Mean for paired T-test for patients’ attitude towards antibiotic use is 1.648 with standard deviation 1.740 (p < 0.001). Mean value for attitude obtained for pre-test is 5.57 (SD=1.915) and post-test 7.22 (SD=1.059). Paired T-test obtained shows that pharmacy intervention had a positive impact in improving patient’s knowledge with significance (P-value = 0.000).

Discussion

The study shows that the population has a misperception in relation to the indication of antibiotics, which may lead to the risk of antibiotic-resistant. The level of knowledge of the study population is moderate to good with 54.3% to 36.8% compared to a study in Penang which has a moderate level of knowledge, 54.7% (Andre et al., 2010) and Shah Alam 37.6% (Davies et al., 2010).

In this study, most of the respondents knew that antibiotic was indicated for bacterial infections, 55.79%, which is lower than reports in Sweden, 77.2% (Qamar et al., 2014), Shah Alam 80.3% (Davies et al., 2010) and in Penang, 76.7% (Andre et al., 2010). In another knowledge statement, 67.4% of the respondents had a misconception that antibiotic can be used to eradicate viral infections which are relatively high as compared to study done in Sweden which is 26.8% (Qamar et al., 2014), Shah Alam 40.5% (Davies et al., 2010) and Penang 67.2% (Andre et al., 2010) but is lower than Putrajaya 80.3% (Azevedo et al., 2013). The probable reason for the lack of awareness in this area can be due to the terminology “germs” that are generally used in the delivery of medical advice or counseling to the patients instead of using the scientific terminology “bacteria” or “virus” (You et al., 2008). A total of 62.4% of respondents in this study failed to identify that antibiotic does not work for a cough and cold compared to 60.5% in Shah Alam (Davies et al., 2010). Studies in Penang (Andre et al., 2010) and Putrajaya (Azevedo et al., 2013) both have more than 80% of respondents failing to identify this usage of antibiotic; however, only 53.0% was reported in UK studies (McNulty et al., 2007)

Knowledge statements 6, 11 and 12 shows substantial correlation with that the knowledge gap may not be totally accidental. Respondents’ might have been misguided that antibiotics are equivalent to painkillers or antipyretics, leading them to assume that discontinuing antibiotics is safe, as they would do with painkillers and antipyretics with symptom improvement.

Respondents’ knowledge of appropriate antibiotic usage was found to correlate positively with attitude. A Strong association was also observed between some of the knowledge and attitude statements. This was consistent with a study in Hong Kong (Teng et al., 2004), where adequate knowledge of antibiotics was shown to be a predictor for appropriate attitudes toward antibiotics and their use where participants with adequate knowledge were 1.52 times more likely to demonstrate appropriate attitude.

In this research, it was observed that majority of respondents‟ answered correctly on the attitude statement. However, regarding the two statements from the survey, the respondents responded incorrectly. The statements are “when I get a cold, I will take antibiotics to help me get better quickly” with 51.3% (n=195) and “I expect antibiotic to be prescribed by my doctor if I suffer from common cold symptoms”, 51.1% (n=194). However, the percentage of incorrect responses for both the statements are slightly lower as compared to a study done in Shah Alam (Davies et al., 2010) and Putrajaya (Azevedo et al., 2013) for the second statement but higher for the first statement in Shah Alam.

For the first statement, a percentage of incorrect answers was 61.8% for Putrajaya (Azevedo et al., 2013) and 47.6% for Shah Alam (Davies et al., 2010) while the second statement was 73.8% for Putrajaya (Azevedo et al., 2013) and 57.4% for Shah Alam (Davies et al., 2010). In Penang, incorrect answers for the first statement were 46.8% (Andre et al., 2010) and the second statement was 57.8%. 38.2% of the respondents responded incorrectly regarding the attitude statement, “if my family member is sick, I usually give my antibiotic to them‟ which is comparatively high with a study done in Penang 11.8% (Andre et al., 2010) and Putrajaya 17.0% (Azevedo et al., 2013) and Shah Alam 32.1% (Davies et al., 2010). This indicates good knowledge of antibiotic use doesn’t necessarily need a good attitude towards antibiotic usage. Around 51.1% of respondents agreed that they normally keep antibiotics more than what is needed at home in case of emergencies. The figure in Penang (80.1%) (Andre et al., 2010), Shah Alam (62.6%) (Davies et al., 2010) and Putrajaya (83%) (Azevedo et al., 2013) is reported to be higher. This study also shows that 26.6% of the respondents who answered that they will use leftover of antibiotics for respiratory illness were higher compared to Penang (Andre et al., 2010), Shah Alam (Davies et al., 2010) and Putrajaya (Azevedo et al., 2013). In this study, 28.9% respondents answered incorrectly and in Penang (Andre et al., 2010) and Shah Alam (Davies et al., 2010) it was 11.3%. Putrajaya also stated a lower figure with 14.7% (Azevedo et al., 2013). In the current study, 48.4% of respondents claimed to take antibiotics according to the instruction on the label while Penang with 93.1% (Andre et al., 2010), Shah Alam 87.6% (Davies et al., 2010), and Putrajaya 96.5% (Azevedo et al., 2013). However, for this analysis, 14.2% answered that they were not sure if they took the antibiotic correctly. Therefore, good knowledge of antibiotic doesn’t mean good attitude in consuming is practiced.

This study is assimilating the role of a pharmacist in patient care on proper antibiotic use to reduce antibiotic resistance. All pharmacists’ in Klinik Kesihatan Seremban (Seremban Health Clinic) are involved in increasing patients’ knowledge towards proper antibiotic use. Education through self-designed pamphlets was carried out for patients who have a language barrier, health literacy and insufficient knowledge on the appropriate use of antibiotics for better communication between the pharmacist and patients. Counseling sessions, exhibitions, and talks were also held for better understanding. Self-designed pamphlets, a newly designed pamphlet available in 4 main languages which is Malay, English, Chinese and Tamil was provided to patients starting September 2016. Pharmacists would discuss with patients about their understanding and attitude towards antibiotics for every patient that were prescribed for it. An antibiotic campaign was launched in November 2015 in Malaysia targeting the public for improving their knowledge and awareness on antibiotic use.

Alongside with Know Your Medicine Campaign which was launched jointly by the Ministry of Health and Consumers Association of Malaysia in 2007, the Antibiotic Awareness week is a yearly affair to health care providers. It was a positive start and established willingness of the policy-makers and also provides a ground stage to promote wise medicine usage among the public. Percentage of patients with good knowledge towards antibiotic use increased from 22.7% to 77.1%. Patient’s adherence to medications, health literacy, and attitude towards antibiotic use also improved significantly. Misuse of antibiotics by the general community reduced which played an important role in reducing antibiotic resistance. This will be most beneficial to the patients as this proved that pharmacist intervention had significantly increased the percentage of good knowledge and positive attitude amongst patients towards antibiotic use, with acceptance by both patients and prescribers. Patient-oriented programs and clear communication strategies to promote appropriate use of antibiotics and patients’ understanding of antibiotic are essential in maximizing the treatment and minimizing the occurrence of resistance. A major, essential role can actually be played by a pharmacist in order to ensure the appropriate antibiotic use in patients by collaborating with prescriber to make proper interventions.

Education through self-designed pamphlets in English, Malay, Chinese, and Tamil was to be carried out for patients every day if necessary. Exhibitions and talks are planned to be held twice a year along with Know Your Medicine for better patient understanding. Remedial measures need to be reviewed and interventions ensured at least once a year in order to meet the need and objectives of patients in understanding antibiotic use.

Limitations of this study consist of the use of convenience sampling and data collected based on patients’ memory, hence it may be subjective. However, it did assist in the conduct of the study but may minimise the generalisability of the results. Additional survey in multiple zones of the state or country with larger populations will help to endorse these results.

Conclusion

Assimilation of pharmacist-based intervention in improving patients’ knowledge and attitude towards antibiotic use has managed to help increase the percentage of patients’ with good knowledge and attitude towards antibiotic use in Klinik Kesihatan Seremban (Seremban Health Clinic). There is a future plan to standardize the remedial measures of increasing knowledge and attitude towards appropriate antibiotic use in Government Health Clinics in Seremban District. Continuous educations are essential for proper antibiotic use. In conclusion, the approach used in this study seems favorable. To stimulate long term preservation of knowledge, our hope and believe is that pharmacist’s intervention and education can be simulated in other regions, accompanied by new activities towards appropriate use of antibiotic. Therefore, the appropriate use of antibiotics is significant in the primary healthcare system in Malaysia.

Conflict of Interest

The authors declare no conflict of interest.

Acknowledgements

We would like to thank the Director General of Health Malaysia for the permission to publish this manuscript. We would like to express our gratitude to our colleagues from the Pharmacy Outpatient Pharmacy Department of Klinik Kesihatan Seremban (Seremban Health Clinic) for their cooperation and assistance throughout the study’s data collection process. We appreciatively acknowledge the study respondents without whom this study would not have been possible.

References