QUALITY, RELIABILITY, AND READABILITY OF TURKISH DIZZINESS INFORMATION: WEB SOURCES VERSUS GENERATIVE ARTIFICIAL INTELLIGENCE
Summary
Background: Patients increasingly seek information about dizziness from both web-based sources and generative artificial intelligence (AI) tools. However, the quality, reliability, and readability of these sources may vary. This study aimed to compare Turkish web-based dizziness information with responses generated by ChatGPT and Gemini in terms of quality, reliability, readability, clinical accuracy, and safety-related variables.Methods: This cross-sectional matched digital content analysis included 10 patient-oriented questions related to dizziness. For each question, one Turkish web-based response, one ChatGPT response, and one Gemini response were obtained, yielding 30 texts. Google searches were performed between March 14 and 17, 2026, and AI responses were collected on March 17, 2026. Readability was assessed using the Ateşman and Çetinkaya-Uzun formulas. Quality and reliability were evaluated with the DISCERN instrument. A structured 5-item accuracy/safety form was also used.
Results: Mean DISCERN scores were 60.2±2.2 for ChatGPT, 56.1±4.9 for Gemini, and 51.7±8.1 for web sources, with a significant overall difference among groups (p=0.001). Mean accuracy scores were 8.7±0.9 for ChatGPT, 8.2±1.5 for Gemini, and 7.8±1.9 for web sources; however, this difference was not statistically significant (p=0.212). Çetinkaya-Uzun readability scores differed significantly among groups (p=0.007), whereas no significant difference was found for Ateşman readability scores (p=0.122). Risky advice was identified only in one web-based text, whereas overly definitive language was observed only in one Gemini response.
Conclusion: Generative AI responses showed higher quality/reliability scores and more favorable readability values in some comparisons; however, no statistically significant differences were found in clinical accuracy or safety-related variables. Therefore, higher DISCERN or readability scores should not be interpreted as evidence of clinical superiority or patient-safety advantage. Both AI-generated and web-based sources should be interpreted cautiously, and clinicians should guide patients in critically evaluating digital health information.
Introduction
Dizziness is a common complaint encountered in primary care and otorhinolaryngology practice; however, rather than representing a single disease entity, it is a heterogeneous symptom that may reflect vestibular, cardiovascular, neurological, or psychogenic causes [1]. Given its broad etiology and substantial impact on quality of life, patients frequently seek information regarding its possible causes, warning signs, and treatment options [1].The internet has become one of the most easily accessible sources in this information-seeking process. Although online health information may help patients prepare for physician visits, formulate questions, and shape self-care decisions, the accuracy, timeliness, reliability, and readability of such content for the intended audience may vary considerably [2]. Among individuals with limited health literacy, complex or ambiguous texts may increase the risk of misunderstanding, unnecessary anxiety, and inappropriate healthcare decisions[3].
Previous studies on Turkish-language online health materials have also supported this concern. In a study evaluating online patient information materials on tinnitus, the texts were reported to be generally written above the educational level of the target audience [4]. Regarding dizziness-related internet-based patient information texts, Tahir and Kent assessed the readability of the first 100 Google search results [5]. Subsequently, Çoluk examined both readability and content quality using the DISCERN instrument in Turkish websites identified through the keywords “baş dönmesi,” “vertigo,” and “denge bozukluğu” [6]. Similarly, in Turkish online informational materials on tympanostomy tubes, the use of references, relevant specialty attribution, and academic affiliation were shown to be associated with higher DISCERN scores[7]. Taken together, these studies indicate that, in online health content, not only accessibility but also quality and reliability warrant careful evaluation In recent years, generative artificial intelligence-based conversational systems have also become part of the online health information ecosystem. Patients now use not only search engines but also systems such as ChatGPT and Gemini to obtain direct explanations about symptoms, diagnosis, and treatment. Studies conducted in different medical fields have shown that these tools are often capable of producing fluent and well-structured responses; however, their quality, accuracy, and readability may vary depending on the model and prompting strategy used[8,9]. Therefore, comparing generative AI responses with conventional web-based sources using the same evaluation criteria has become increasingly important in terms of digital health literacy and patient safety [8,9].
The aim of this study was to compare Turkish-language web-based sources and generative artificial intelligence responses regarding dizziness in terms of quality, reliability, readability, clinical accuracy, and safety-related variables.
Methods
Study DesignThis study was designed as a preliminary cross-sectional, matched digital content analysis to compare Turkish-language web-based sources and generative artificial intelligence responses regarding dizziness in terms of quality, reliability, readability, clinical accuracy, and safety-related variables. A set of 10 fixed questions phrased in patient-friendly language was used in the analysis. For each question, one web-based response, one ChatGPT response, and one Gemini response were obtained, resulting in a total of 30 texts for evaluation. The questions were selected to cover the domains most commonly searched by patients in relation to dizziness, including definition, causes, alarm symptoms, the appropriate medical specialty for consultation, treatment approaches, recommendations applicable at home, common vestibular causes, and situations requiring urgent medical attention. Technical terminology was avoided whenever possible, and patient-friendly wording was used instead. The English question pool and the exact Turkish query/prompt used for Google, ChatGPT, and Gemini are presented in Supplementary Table 1.
){kind=link}
Ethical Considerations
As this study was based on the secondary analysis of open-access web texts and artificial intelligence-generated outputs, did not involve any intervention, and contained no personal data, ethics committee approval was not obtained.
Selection of Web-Based Content
For the web-based arm, searches were performed using the Google search engine between March 14 and March 17, 2026, in incognito mode. To minimize the influence of personalized search results, browser history, cookies, and cache were cleared before the searches. For each question, the first eligible organic Turkish-language result that was freely accessible, text-based, intended for patient information, and directly relevant to the query was included in the study. Sponsored links, advertisement-focused pages, video-only content, forum, dictionary, and social media content, PDF files, duplicate links, non-Turkish pages, access-restricted content, pages with insufficient textual content, and pages primarily intended for institutional promotion rather than patient education were excluded. When the same website appeared again for a different question, the next eligible organic result was selected in order to increase source diversity. For each question, the patient-oriented question text was used as the search query. The full characteristics of the included sources, including the recorded URLs, source titles, and website/source types, are provided in Supplementary Table 3. The original Turkish texts used in the analysis across all three source groups, including the extracted web texts and AI-generated responses, together with their English translations, are provided in Supplementary Table 4. Because web-based content is dynamic and may be updated or removed over time, the recorded source information reflects the status of the included web pages during the data collection period.
){kind=link}
){kind=link}
Acquisition of Artificial Intelligence Responses
Responses from ChatGPT and Gemini were obtained on March 17, 2026, in separate sessions. ChatGPT was accessed through a ChatGPT Plus subscription via the standard web interface, and Gemini was accessed using a Gemini Plus subscription via the standard web interface. A new chat session was opened for each question, and all prompts were entered independently to prevent prior conversation context from influencing subsequent responses. All prompts consisted solely of the exact Turkish patient-oriented question text; no additional role assignment, tone instruction, length restriction, formatting instruction, or follow-up prompt was used. For each question, the first response generated by the system was included in the analysis, and no regeneration was performed. This approach was intended to preserve the response format most similar to a real-world user query. All AI responses were collected on the same day and in Turkish. The exact underlying model version of Gemini Plus was not explicitly displayed in the interface and was therefore noted as a methodological limitation. Reporting considerations for chatbot-based health advice studies were taken into account during the design and reporting of the study[10]. The exact Turkish prompts used for data collection are listed in Supplementary Table 1. The original Turkish texts used in the analysis across all three source groups, including the extracted web texts and the raw AI-generated responses, together with their English translations, are provided in Supplementary Table 4.
Data Extraction and Coding
All texts were transferred to Microsoft Excel for data extraction and coding. For web pages, only the main patient-information body text was included in the analysis; advertisements, menus, navigational links, image captions, and institutional promotional statements were excluded. For each web-based source, the source name, recorded URL, website/source type, question text, and extracted response text were recorded. For AI-generated responses, the platform, subscription/interface information, response date, exact Turkish prompt, and generated response text were recorded. The original Turkish texts used in the analysis across all three source groups are provided in Supplementary Table 4 together with their English translations.
Readability, Quality, and Accuracy Assessment
Readability was assessed using the Ateşman and Çetinkaya-Uzun formulas [11,13]. After the texts had been transferred to Excel, the total number of syllables, words, and sentences in each text was calculated, and readability scores were derived using both formulas [11,13]. The quality and reliability of the texts were evaluated using the DISCERN instrument [12].
In addition, a structured researcher-developed 5-item accuracy/safety form was used to assess clinical accuracy and patient safety more directly. The form was developed based on general principles of patient-oriented health information, clinical assessment principles of dizziness, red-flag symptoms requiring urgent evaluation, and expert judgement in audiology/vestibular assessment. It evaluated the accuracy of the core information, the adequacy of major causes or explanations provided, the reporting of alarm findings when appropriate, the absence of incorrect or potentially harmful recommendations, and the appropriateness of healthcare referral guidance. Each item was scored from 0 to 2, and the total accuracy score was calculated on a scale ranging from 0 to 10. The individual items of this form and their detailed scoring criteria are provided in Supplementary Table 2. In addition, the variables “presence of alarm findings,” “presence of incorrect/risky recommendations,” and “presence of overly definitive statements” were coded separately for each text.
){kind=link}
Rating Procedure and Inter-Rater Agreement
DISCERN and accuracy/safety scoring were performed independently by two raters. For the main analysis, the mean of the two raters’ total DISCERN scores was used; therefore, some DISCERN values were reported as half points. Inter-rater agreement was assessed using the intraclass correlation coefficient (ICC) for total DISCERN and total accuracy/safety scores, and linearly weighted kappa coefficients for ordinal ratings. Linearly weighted kappa coefficients were also calculated for the individual items of the accuracy/safety form. In cases of disagreement in item-based evaluations, consensus was first sought; when consensus could not be achieved, the opinion of a third researcher was obtained and the final decision was made accordingly.
Statistical Analysis
Descriptive data were presented as number, percentage, mean, standard deviation, median, minimum, and maximum values. Because three different responses corresponding to the same 10 questions were compared, analyses were conducted using a matched-measures approach. Non-parametric matched-measures tests were selected given the ordinal nature of the DISCERN and accuracy/safety assessments and the small sample size (n=10). The Friedman test was used for comparisons of DISCERN, accuracy, and readability scores across the three groups. When a significant difference was identified, pairwise comparisons were performed using the Wilcoxon signed-rank test. Since three pairwise comparisons were conducted, Bonferroni correction was applied, and the threshold for statistical significance in pairwise analyses was set at p<0.0167. Effect sizes were calculated to quantify the magnitude of observed differences: Kendall’s W was reported for Friedman test results, and r, calculated as Z divided by the square root of N, was reported for statistically significant Wilcoxon signed-rank pairwise comparisons. The Cochran’s Q test was used for the comparison of binary safety variables across groups.
All analyses were performed using Microsoft Excel and IBM SPSS Statistics version 27.0. A two-sided p value <0.05 was considered statistically significant.
Results
Responses to 10 fixed questions related to dizziness were analyzed in this study. For each question, one web-based source, one ChatGPT response, and one Gemini response were evaluated, yielding a total of 30 texts for analysis. The overall score distributions of the groups are presented in Table 1. Mean DISCERN scores were highest in the ChatGPT group, followed by the Gemini and web-based source groups. The mean accuracy score was also descriptively highest in the ChatGPT group; however, this finding should be interpreted together with the non-significant between-group comparison reported below. The highest mean Ateşman and Çetinkaya-Uzun readability scores were likewise observed in the ChatGPT group (Table 1).In the between-group comparisons, a statistically significant difference was found in DISCERN scores (Friedman test, p=0.001; Kendall’s W=0.67, indicating a large effect). In pairwise comparisons, the ChatGPT group had higher DISCERN scores than both the web-based source group (r=0.89) and the Gemini group (r=0.83). In contrast, no significant difference was observed among the groups in terms of accuracy scores (p=0.212; W=0.16). While no significant difference was found among the groups for Ateşman scores (p=0.122; W=0.21), a significant difference was observed for Çetinkaya-Uzun scores (p=0.007;
W=0.49;
significant in the ChatGPT vs (Table 2).
Gemini pairwise comparison, p=0.014, r=0.76) Inter-rater agreement was good for the total DISCERN score (ICC[2,1]=0.912; linearly weighted κ=0.764). Agreement for the total accuracy/safety score was moderate-to-good (ICC[2,1]=0.874; linearly weighted κ=0.701). For the individual items of the accuracy/safety form, linearly weighted κ values ranged from 0.69 to 0.78. In addition, a small systematic scoring difference was observed between the two raters for DISCERN scores.
Table 2: Between-Group Comparisons of DISCERN, Accuracy, and Readability Scores
When safety-related variables were examined, the number of responses containing alarm findings was similar across the three groups. Incorrect or potentially risky recommendations were identified in only one text in the web-based source group. Overly definitive statements were observed in only one text in the Gemini group. However, no statistically significant differences were found among the groups for the binary safety variables (Cochran’s Q test; all p>0.05) (Table 3).
When readability classifications were evaluated, a substantial proportion of texts in the web-based source and Gemini groups were classified as “difficult” or “very difficult” according to the Ateşman formula. In the ChatGPT group, the proportion of texts classified as “moderately difficult” and “easy” was higher than in the other groups. According to the Çetinkaya-Uzun classification, the number of texts at the “independent reading” level was greater in the ChatGPT group than in the web-based source and Gemini groups (Table 4). Overall, the readability distribution indicated that the ChatGPT group had a higher proportion of texts at the independent reading level and a lower proportion of very difficult texts.
Discussion
In this study, Turkish-language web-based sources and generative artificial intelligence responses regarding dizziness were compared in terms of quality, reliability, readability, clinical accuracy, and safety-related variables. The results showed significant differences among groups in DISCERN quality/reliability scores and Çetinkaya-Uzun readability scores.Table 3: Distribution of safety-related variables across groups
Table 4: Distribution of readability levels across groups
However, no statistically significant differences were found among the groups in terms of clinical accuracy scores, Ateşman readability scores, or safety-related variables. Although the ChatGPT group showed higher DISCERN scores and descriptively higher accuracy and readability values, these findings should be interpreted cautiously, particularly because the differences in clinical accuracy and safety-related variables were not statistically significant. In addition, the significant Çetinkaya-Uzun finding was supported by the ChatGPT versus Gemini pairwise comparison, whereas no significant pairwise superiority was demonstrated across all source types. With respect to safety-related variables, the number of responses containing alarm findings was similar across the groups, while incorrect or potentially risky advice was identified in only one web-based text and overly definitive language was observed in only one Gemini response. Overall, these findings suggest that higher quality/reliability or readability scores should not be interpreted as evidence of superiority in clinical accuracy or patient safety.
Because dizziness is a heterogeneous symptom that may involve multiple body systems rather than a single disease entity, the accuracy and balanced presentation of patient education materials are of particular importance[1]. Online health information is known to influence individual health decisions, and complex or audience-inappropriate content may lead to misunderstanding, unnecessary anxiety, and inappropriate healthcare choices[2,3]. In this respect, our findings suggest that not only access to information, but also its quality and readability, should be carefully evaluated. This is especially important for symptoms such as dizziness, which may sometimes be associated with alarm findings and serious underlying conditions[1-3].
The lower DISCERN scores observed for web-based sources in our study are consistent with the existing literature on Turkish online health information. Previous studies evaluating Turkish online patient information materials on tinnitus and dizziness have highlighted important limitations in readability and content quality[4-6]). Similarly, higher DISCERN scores have been associated with the presence of references, specialty attribution, and academic affiliation in Turkish online materials on tympanostomy tubes[7]. Taken together, these data suggest that the problems of quality and readability in vestibular health information are not unique to Turkish-language resources, but rather reflect a broader structural issue in online health content [4-7,14].
The relatively stronger performance of AI-generated responses, particularly with respect to DISCERN scores, is noteworthy. One possible explanation is that these systems tend to generate more organized, structured, and directly responsive texts. Musheyev et al. showed that both readability and information quality in chatbot-generated cancer information may vary according to the model used[8]. Similarly, Sidhu and Selvamogan reported that AI chatbot responses to frequently asked dermatologic questions demonstrated an assessable level of quality and readability[9]. In a comparative study, Olszewski et al. also found substantial variability among chatbots in terms of length, quality, and readability, while emphasizing that even apparently reliable responses should still be verified [16]. The finding that ChatGPT achieved the highest mean quality and readability scores in our study is in line with this literature[8,9,16]. Nevertheless, high readability or a high DISCERN score should not be interpreted as evidence of flawless clinical performance. The overly definitive wording identified in one Gemini response suggests that fluency in AI-generated texts may at times shift toward categorical and context-insensitive statements. This insensitivity to clinical context has direct patient safety implications: a patient reading an overly definitive AI response may forego necessary clinical evaluation, delay presentation for alarm symptoms, or self-manage a condition that warrants urgent care. From a clinical cross-check perspective, both AI-generated and web-based content should be considered adjuncts to, not substitutes for, professional evaluation, particularly for a symptom as heterogeneous as dizziness.
Another important finding was that, although accuracy scores were descriptively higher in the ChatGPT group, no statistically significant difference was found among the groups. This may be explained by several factors. First, the sample was limited to only 10 matched question sets, which may have reduced the ability to detect statistically significant differences. Second, both web-based and AI-generated responses may have provided a similar acceptable core of clinical information in certain key areas, such as alarm findings, situations requiring emergency referral, and the distinction between vertigo and dizziness. The similar number of responses containing alarm findings across all groups supports this interpretation. In addition, Ayers et al. showed that chatbot responses to physician-developed open-ended medical questions may achieve a certain level of accuracy and completeness, while important limitations still remain[15]. Therefore, our findings suggest that AI systems may have advantages in terms of information presentation and readability, but larger studies are needed before concluding that they clearly outperform web-based sources in terms of accuracy in all contexts[15,16].
The readability findings are also noteworthy. Although no statistically significant difference was found in Ateşman scores, the mean values and categorical distributions pointed to better readability in the ChatGPT group. In contrast, Çetinkaya-Uzun scores showed a significant overall difference, with the significant pairwise comparison observed between ChatGPT and Gemini. Previous studies have already demonstrated that Turkish patient information materials are often written above the reading level of the intended audience [4-7]. In our study, the higher proportion of texts classified as “difficult” or “very difficult” in the web-based and Gemini groups suggests that this problem persists. The greater number of texts at the “independent reading” level in the ChatGPT group indicates that AI-generated responses may be closer to patient-friendly language and may offer a more fluent reading experience. However, readability alone is not sufficient; a text may be easy to read yet still be inaccurate or overly definitive, thereby posing clinical risks[2,3]. For this reason, readability should not be evaluated independently of quality and reliability.
One of the strengths of this study is that the same question pool was applied to three different sources of information. This made it possible to compare sources directly on the basis of the same patient information needs, independent of variations in topic wording or keyword selection. In addition, the use of Turkish readability formulas and the DISCERN instrument ensured consistency with the existing Turkish literature [4-7,11,12].
These findings have several clinical and practical implications. Patients seeking information about a complex symptom such as dizziness no longer rely solely on websites, but increasingly consult generative AI tools as well. Our results suggest that AI tools may offer advantages, particularly in presenting more organized and readable content. Nevertheless, the heterogeneity observed in web-based sources and the tendency toward overly definitive language that may appear in AI-generated responses indicate that both source types should be interpreted critically. Clinicians should therefore be encouraged to produce clear, up-to-date, and reliable Turkish digital content addressing the most common patient questions about dizziness [2,3,7,10]. Future studies using larger question pools, additional AI models, and more detailed accuracy standards may further clarify this area.
In conclusion, the present study demonstrated differences in the quality (DISCERN scores) and readability of Turkish-language sources of dizziness information. The ChatGPT group showed the highest mean DISCERN and readability scores, the Gemini group generally produced acceptable but occasionally more definitive responses, and web-based sources displayed a more heterogeneous structure. Importantly, no statistically significant differences were found in clinical accuracy or safety-related variables across groups; therefore, these results should not be interpreted as evidence that ChatGPT is more accurate or safer than other sources. These findings indicate that in digital patient education on dizziness, not only access to information, but also the quality, readability, and safety of that information should be evaluated together [2,4-10,12,14-16].
Limitations of the Study
This study has several limitations. First, given that only 10 questions and 10 texts per group were analysed, the present work should be regarded as a pilot or preliminary evaluation. Accordingly, the analysis was conducted using only 10 fixed questions, and the small sample size may have reduced statistical power and the ability to detect statistically significant differences, particularly for accuracy and safety-related variables. Second, web searches were performed only using the Google search engine and within a specific time frame, and only the first eligible organic Turkish-language result for each question was included in the study. Therefore, different results might be obtained using different search engines, at different times, or under different user conditions. Third, both artificial intelligence systems and web-based content are dynamic in nature; the same questions may generate different responses on different dates. Although the exact prompts, verbatim raw AI outputs, recorded URLs, and extracted web texts are provided in the Supplementary Tables, identical queries may nevertheless yield different responses if repeated at a later date. Fourth, accuracy was assessed using a structured form developed by the researchers, and no standardized international instrument was used for this purpose. Fifth, the exact underlying model version of Gemini Plus at the time of data collection (March 17, 2026) was not explicitly displayed in the interface; this represents a definitive methodological constraint on reproducibility, as results may differ across model versions. Sixth, the study included only Turkish-language written texts and did not evaluate video-based, social media-based, or non-Turkish sources. Therefore, the findings should not be generalized to all digital health information resources.
Reference
1) Bösner S, Schwarm S, Grevenrath P, Schmidt L, Hörner K, Beidatsch D, Bergmann M, Viniol A, Becker A, Haasenritter J. Prevalence, aetiologies and prognosis of the symptom dizziness in primary care: a systematic review. BMC Fam Pract. 2018;19:33.
2) Thapa DK, Visentin DC, Kornhaber R, West S, Cleary M. The influence of online health information on health decisions: a systematic review. Patient Educ Couns. 2021;104(4):770-84.
3) Weiss BD. Health literacy and patient safety: help patients understand. Manual for clinicians. 2nd ed. Chicago (IL): American Medical Association Foundation; 2007.
4) Saldırım HB, Eren M, Kurtuluş N, Kırlaroğlu SN, Şerbetçioğlu MB. Tinnitus ile ilgili çevrim içi hasta bilgilendirme materyallerinin okunabilirliğinin değerlendirilmesi. Balkan Sağlık Bilimleri Dergisi. 2023;2(1):1-6.
5) Tahir E, Kent AE. Baş dönmesi ile ilgili internet kaynaklı hasta bilgilendirme metinlerinin okunabilirlik düzeyleri. KBB-Forum. 2021;20(2):163-70.
6) Çoluk Y. Vertigo ile ilgili Türkçe internet kaynaklı bilgilerin kalite ve okunabilirlik düzeylerinin değerlendirilmesi. KBB-Forum. 2023;22(4):260-270.
7) Ersin K, Saldırım HB, Eren M. Kulak tüpü hakkında çevrim içi bilgilendirme metinlerinin okunabilirlik, kalite ve güvenilirliği. Mersin Univ Sağlık Bilim Derg. 2026;19(1):69-79.
8) Musheyev D, Pan A, Gross P, Kamyab D, Kaplinsky P, Spivak M, Bragg MA, Loeb S, Kabarriti AE. Readability and information quality in cancer information from a free vs paid chatbot. JAMA Netw Open. 2024;7(7):e2422275.
9) Sidhu RS, Selvamogan A. Assessing the quality and readability of AI chatbot responses to frequently asked questions about basal cell carcinoma. Clin Surg Oncol. 2025;4(3):100094.
10) The CHART Collaborative. Reporting guideline for chatbot health advice studies: the CHART statement. JAMA Netw Open. 2025;8(8):e2530220.
11) Ateşman E. Türkçede okunabilirliğin ölçülmesi. Dil Dergisi. 1997;(58):71-74.
12) Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-11.
13) Çetinkaya G. Türkçe metinlerin okunabilirlik düzeylerinin tanımlanması ve sınıflandırılması. Ankara: Ankara Üniversitesi Sosyal Bilimler Enstitüsü; 2010.
14) Felipe L, Beukes EW, Fox BA, Manchaiah V. Quality and readability of English-language Internet information for vestibular disorders. J Vestib Res. 2020;30(2):63-72.