Measuring Similarity, Credibility and Value of Information Content of Google and Generative AI Platforms
Abstract
This study examines the similarity, credibility, and value of information content generated by Google and various Generative AI (GAI) platforms, such as ChatGPT, Qwen, Perplexity, and Poe. Using the Jaccard Similarity Index, the study measures content overlap between Google search results and AI-generated responses, resulting in two clusters. The first one is the low correlation between the Google and AI outputs. Second is the consistency among AI-based models in terms of similarity. Thus, AI platforms reflect higher internal similarity, suggesting a high degree of homogeneity among them. The research also evaluates credibility, finding Google to have higher average credibility scores (0.82) compared to Perplexity (0.59) in reference analysis, with GAI platforms often lacking reliable citations and traceability. Content credibility scores, assessed using criteria such as source credibility, content accuracy, completeness, objectivity, and verifiability, indicate that Google excels the AI platforms. Google scored 6.54, whereas Perplexity, Qwen, ChatGPT, and Poe scored 4.72, 3.75, 2.81, and 2.26, respectively, in user evaluation. In terms of credibility, Google surpasses generative AI platforms; while AI tools provide useful summaries, they lack the depth, evidence base, and scholarly context found in Google search results. The study concludes that while AI enhances communication efficiency, it raises concerns about reliability, knowledge foraging, and scientific integrity. Future research aims to analyze content volume and length as measures of value, emphasizing the need for transparency and human-centered AI development.
References
[1] Lindsey Witmer Collins. How AI is undermining online authenticity. https://www.fastcompany.com/ 91364892/how-ai-is-undermining-online-authenticity
[2] Bozkurt, A., Sharma, R. C. (2024). Trust, credibility and transparency in human–AI interaction: Why we need explainable and trustworthy AI and why we need it now. Asian Journal of Distance Education, 19(2). http://www.asianjde.com/ojs/index.php/AsianJDE/article/view/819 [3] Luna, Javier Canales. (2023). ChatGPT vs Google Bard: A comparative guide to AI chatbots. DataCamp Blog. https://www.datacamp.com/blog/bard-vs-chat-gpt [4] Borji, Ali. (2023, April 3). A categorical archive of ChatGPT failures (arXiv:2302.03494v8) [Preprint]. arXiv. https://arxiv.org/abs/2302.03494v8 [5] Guo, Biyang., Zhang, Xin., Wang, Ziyuan., Jiang, Minqi., Nie, Jinran., Ding, Yuxuan., Yue, J., Wu, Yupeng. (2023). How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection (arXiv:2301.07597v1). https://arxiv.org/abs/2301.07597v1 [6] Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., Hovy, E., Ji, H., Menczer, F., Miguez, R., Nakov, P., Scheufele, D., Sharma, S., Zagni, G. (2024). Factuality challenges in the era of large language models and opportunities for fact checking. Nature Machine Intelligence, 6(8), 852–863. [7] Zhang, Peng., Boulos, M. (2023). Generative AI in medicine and healthcare: Promises, opportunities and challenges. Future Internet, 2023. [8] Li, Alice., Sinnamon, L. (2024). Generative AI search engines as arbiters of public knowledge: An audit of bias and authorit. https://arxiv.org/abs/2405.14034 [9] Pichappan, P., Krishnamurthy, M., Vijayakumar, P. (2023). Analysis of ChatGPT as a question-answering tool. Journal of Digital Information Management, 21(2), 50–60. [10] Ou, M., Zheng, H., Zeng, Y., Hansen, P. (2024). Trust it or not: understanding users’ motivations and strategies for assessing the credibility of AI generated information. New Media & Society, November. [11] Zhang, He., Xie, Jingyi., Wu, Chuhao., Cai, Jie., Kim, Chanmin., Carroll, John. M. (2024). The future of learning: Large language models through the lens of students, Association for Computing Machinery, New York, NY, USA. [12] Huschens, Martin., Briesch, Martin., Sobania, Dominik., Rothlauf, Franz. (2023, September 5). Do you trust ChatGPT? Perceived credibility of human and AI generated content (arXiv:2309.02524v1)[cs.HC]. [13] Knight, Samsun., Bart, Yakov. (2023). Generative AI and user-generated content: Evidence from online reviews. Social Science Research Network. [14] Zhou,Tao., Lu, Hailin. (2025). The effect of trust on user adoption of AI generated content. The Electronic Library, 43(1), 61–76. [15] Cheng, S., Tsai, S., Bai, Y., Ko, C., Hsu, C., Yang, F., et al. (2023, December 25). Comparisons of quality,correctness, and similarity between ChatGPT generated and human written abstracts for basic research: Cross sectional study. Journal of Med Internet Res, 25, e51229. [16] Kim, H. J., Yang, J. H., Chang, D., Lenke, L. G., Pizones, J., Castelein, R., et. al. (2024, June 26). Assessing the reproducibility of structured abstracts generated by ChatGPT and Bard compared to human written abstracts in spine surgery: Comparative analysis. Journal of Med Internet Res, 26, e52001. [17] Jang., Lukasiewicz. (2023). Consistency analysis of ChatGPT (arXiv:2303.06273) [Preprint]. arXiv. 2303.06273 [18] Shen, X., Chen, Z., Backes, M., Zhang, Y. (2023). In ChatGPT we trust? Measuring and characterising the reliability of ChatGPT, arXiv [Preprint]. arXiv: 2304.08979 [19] Xu, Danni., Fan, Shaojing., Kankanhalli, Shaojing. (2023, October 29–November 3). Combating misinformation in the era of generative AI models. In: Proceedings of the 31st ACM International Conference on Multimedia (MM ’23) (pp. 1–8). Ottawa, ON, Canada: ACM. https://doi.org/10.1145/3581783.3612704 [20] Wahid, Risqo M., Mero, J., Ritala, P. (2023). Editorial: Written by ChatGPT, illustrated by Midjourney: Generative AI for content marketing. Asia Pacific Journal of Marketing and Logistics. [21] Tai, Yuehong Cassandra., Patni, Khushi., Hemauer, N. D., Desmarais, B. A., Lin, Y.-R. (2025). GenAI vs. human fact checkers: Accurate ratings, flawed rationales. In Proceedings of the Web Science Conference (pp. 1–12). [22] Hanimann, Anina., Heimann, Andri., Hellmueller, Lea., Trilling, D. (2023). Believing in credibility measures: Reviewing credibility measures in media research from 1951 to 2018. International Journal of Communication, 17(2023), 214–235. [23] Guo, Biyang., Zhang, Xin., Wang, Ziyuan., Jiang, M., Nie, J., Ding, Y., Yue, J., Wu, Y. (2023). How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection (arXiv:2301.07597v1). [24] Travieso, Gonzalo., Benatti, Alexandre., Costa, Luciano da F. (2024). An analytical approach to the Jaccard similarity index (arXiv:2410.16436v1) [25] Talaver, Oleh V., Vakaliuk, Tetiana A. (2025, May 14). A model for improving the accuracy of educational content created by generative AI. In: 7th International Workshop on Augmented Reality in Education (AREdu 2024) (pp. 149–158). Kryvyi Rih, Ukraine. [26] Finstad, Kraig. (2010). Response interpolation and scale sensitivity: Evidence against 5 point scales. Journal of User Experience, 5(3), 104–110. [27] Farquhar, S., Kossen, J., Kuhn, L., et. al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630, 625–630.[28] Xiao, Y., Wang, W. Y. (2021). On hallucination and predictive uncertainty in conditional language generation. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (pp. 2734–2744). PA: Association for Computational Linguistics. [29] Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., Saenko, K. (2018). Object hallucination in image captioning. In Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (Eds.), In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 4035–4045). Association for Computational Linguistics. [30] Blau, Wolfgang., Cerf, Vinton. G., Enriquez, Juan., Francisco, Joseph. S., Gasser, Urs., Gray, Mary. L., Greaves, Mark., Grosz, Barbara J., Jamieson, Kathleen Hall., Haugk, Gerald H., Hennessy, John L., Horvitz, Eric., Kaiser, David I., Londono, A. J., Lovell-Badge, R., McNutt, M. K., Minow, M., Mitchell, T. M., Ness, Susan ., Parthasarathy, Shobita., Perlmutter, Saul., Press, William H., Wing, Jeannette M., Witherell, Michael. (2024). Protecting scientific integrity in an age of generative AI. Proceedings of the National Academy of Sciences of the United States of America, 121(22), e2407886121. https://doi.org/10.1073/pnas.2407886121, pp 1-3
[2] Bozkurt, A., Sharma, R. C. (2024). Trust, credibility and transparency in human–AI interaction: Why we need explainable and trustworthy AI and why we need it now. Asian Journal of Distance Education, 19(2). http://www.asianjde.com/ojs/index.php/AsianJDE/article/view/819 [3] Luna, Javier Canales. (2023). ChatGPT vs Google Bard: A comparative guide to AI chatbots. DataCamp Blog. https://www.datacamp.com/blog/bard-vs-chat-gpt [4] Borji, Ali. (2023, April 3). A categorical archive of ChatGPT failures (arXiv:2302.03494v8) [Preprint]. arXiv. https://arxiv.org/abs/2302.03494v8 [5] Guo, Biyang., Zhang, Xin., Wang, Ziyuan., Jiang, Minqi., Nie, Jinran., Ding, Yuxuan., Yue, J., Wu, Yupeng. (2023). How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection (arXiv:2301.07597v1). https://arxiv.org/abs/2301.07597v1 [6] Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., Hovy, E., Ji, H., Menczer, F., Miguez, R., Nakov, P., Scheufele, D., Sharma, S., Zagni, G. (2024). Factuality challenges in the era of large language models and opportunities for fact checking. Nature Machine Intelligence, 6(8), 852–863. [7] Zhang, Peng., Boulos, M. (2023). Generative AI in medicine and healthcare: Promises, opportunities and challenges. Future Internet, 2023. [8] Li, Alice., Sinnamon, L. (2024). Generative AI search engines as arbiters of public knowledge: An audit of bias and authorit. https://arxiv.org/abs/2405.14034 [9] Pichappan, P., Krishnamurthy, M., Vijayakumar, P. (2023). Analysis of ChatGPT as a question-answering tool. Journal of Digital Information Management, 21(2), 50–60. [10] Ou, M., Zheng, H., Zeng, Y., Hansen, P. (2024). Trust it or not: understanding users’ motivations and strategies for assessing the credibility of AI generated information. New Media & Society, November. [11] Zhang, He., Xie, Jingyi., Wu, Chuhao., Cai, Jie., Kim, Chanmin., Carroll, John. M. (2024). The future of learning: Large language models through the lens of students, Association for Computing Machinery, New York, NY, USA. [12] Huschens, Martin., Briesch, Martin., Sobania, Dominik., Rothlauf, Franz. (2023, September 5). Do you trust ChatGPT? Perceived credibility of human and AI generated content (arXiv:2309.02524v1)[cs.HC]. [13] Knight, Samsun., Bart, Yakov. (2023). Generative AI and user-generated content: Evidence from online reviews. Social Science Research Network. [14] Zhou,Tao., Lu, Hailin. (2025). The effect of trust on user adoption of AI generated content. The Electronic Library, 43(1), 61–76. [15] Cheng, S., Tsai, S., Bai, Y., Ko, C., Hsu, C., Yang, F., et al. (2023, December 25). Comparisons of quality,correctness, and similarity between ChatGPT generated and human written abstracts for basic research: Cross sectional study. Journal of Med Internet Res, 25, e51229. [16] Kim, H. J., Yang, J. H., Chang, D., Lenke, L. G., Pizones, J., Castelein, R., et. al. (2024, June 26). Assessing the reproducibility of structured abstracts generated by ChatGPT and Bard compared to human written abstracts in spine surgery: Comparative analysis. Journal of Med Internet Res, 26, e52001. [17] Jang., Lukasiewicz. (2023). Consistency analysis of ChatGPT (arXiv:2303.06273) [Preprint]. arXiv. 2303.06273 [18] Shen, X., Chen, Z., Backes, M., Zhang, Y. (2023). In ChatGPT we trust? Measuring and characterising the reliability of ChatGPT, arXiv [Preprint]. arXiv: 2304.08979 [19] Xu, Danni., Fan, Shaojing., Kankanhalli, Shaojing. (2023, October 29–November 3). Combating misinformation in the era of generative AI models. In: Proceedings of the 31st ACM International Conference on Multimedia (MM ’23) (pp. 1–8). Ottawa, ON, Canada: ACM. https://doi.org/10.1145/3581783.3612704 [20] Wahid, Risqo M., Mero, J., Ritala, P. (2023). Editorial: Written by ChatGPT, illustrated by Midjourney: Generative AI for content marketing. Asia Pacific Journal of Marketing and Logistics. [21] Tai, Yuehong Cassandra., Patni, Khushi., Hemauer, N. D., Desmarais, B. A., Lin, Y.-R. (2025). GenAI vs. human fact checkers: Accurate ratings, flawed rationales. In Proceedings of the Web Science Conference (pp. 1–12). [22] Hanimann, Anina., Heimann, Andri., Hellmueller, Lea., Trilling, D. (2023). Believing in credibility measures: Reviewing credibility measures in media research from 1951 to 2018. International Journal of Communication, 17(2023), 214–235. [23] Guo, Biyang., Zhang, Xin., Wang, Ziyuan., Jiang, M., Nie, J., Ding, Y., Yue, J., Wu, Y. (2023). How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection (arXiv:2301.07597v1). [24] Travieso, Gonzalo., Benatti, Alexandre., Costa, Luciano da F. (2024). An analytical approach to the Jaccard similarity index (arXiv:2410.16436v1) [25] Talaver, Oleh V., Vakaliuk, Tetiana A. (2025, May 14). A model for improving the accuracy of educational content created by generative AI. In: 7th International Workshop on Augmented Reality in Education (AREdu 2024) (pp. 149–158). Kryvyi Rih, Ukraine. [26] Finstad, Kraig. (2010). Response interpolation and scale sensitivity: Evidence against 5 point scales. Journal of User Experience, 5(3), 104–110. [27] Farquhar, S., Kossen, J., Kuhn, L., et. al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630, 625–630.[28] Xiao, Y., Wang, W. Y. (2021). On hallucination and predictive uncertainty in conditional language generation. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (pp. 2734–2744). PA: Association for Computational Linguistics. [29] Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., Saenko, K. (2018). Object hallucination in image captioning. In Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (Eds.), In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (pp. 4035–4045). Association for Computational Linguistics. [30] Blau, Wolfgang., Cerf, Vinton. G., Enriquez, Juan., Francisco, Joseph. S., Gasser, Urs., Gray, Mary. L., Greaves, Mark., Grosz, Barbara J., Jamieson, Kathleen Hall., Haugk, Gerald H., Hennessy, John L., Horvitz, Eric., Kaiser, David I., Londono, A. J., Lovell-Badge, R., McNutt, M. K., Minow, M., Mitchell, T. M., Ness, Susan ., Parthasarathy, Shobita., Perlmutter, Saul., Press, William H., Wing, Jeannette M., Witherell, Michael. (2024). Protecting scientific integrity in an age of generative AI. Proceedings of the National Academy of Sciences of the United States of America, 121(22), e2407886121. https://doi.org/10.1073/pnas.2407886121, pp 1-3
Published
2025-09-11
How to Cite
PICHAPPAN, Preethi; PICHAPPAN, Pit.
Measuring Similarity, Credibility and Value of Information Content of Google and Generative AI Platforms.
Journal of Digital Information Management(JDIM), [S.l.], v. 23, n. 3, p. 149-170, sep. 2025.
ISSN 0972-7272.
Available at: <https://www.dline.info/ojs/index.php/jdim/article/view/550>. Date accessed: 21 apr. 2026.
Section
Research
