ABSTRACT

BACKGROUND/AIMS:

To examine the informative features and qualities of YouTube videos on high tibial osteotomy (HTO) in terms of the informative reliability of the videos.

MATERIALS AND METHODS:

The included videos were evaluated individually using two orthopedic surgeons using modified DISCERN score and JAMA Benchmark score. Videos were categorized according to publishing year, video type, video length, number of views, and view rates. Any correlation between these categories and JAMA and modified DISCERN scores were analyzed.

RESULTS:

Twenty-six videos were available for evaluation. Mean-modified DISCERN and JAMA scores were 2.6 and 2.0 for observer 1 and 2.5 and 1.9 for observer 2, respectively. Median view rates of the videos published between 2008-2011, 2012-2015, and 2015-2020 were 2559.00, 3314.0, and 7458.00, respectively (p=0.003). Other variables showed no difference compared to the publishing year groups (p>0.05). JAMA scores by both observers were positively correlated with video length. A weak positive correlation between mean JAMA score and view rate and a positive correlation between modified DISCERN scores by both observers and video length were found. Mean-modified DISCERN scores and video length values had a strong correlation (p=0.001; r=0.609). A significant correlation was found between JAMA and modified DISCERN scores. Meanmodified DISCERN scores showed a moderate increase correlated with the mean JAMA score increase (p=0.017; r=0.463).

CONCLUSION:

Although the view rate has increased over time, the informative quality and reliability of YouTube videos concerning HTO are low and did not improve over the last decade.

Keywords:

High tibial osteotomy, YouTube, JAMA Benchmark score, modified DISCERN score, osteoarthritis

INTRODUCTION

Medial compartment osteoarthritis (MCOA) of the knee is a common and challenging pathology in young and active patients. High tibial osteotomy (HTO) was developed to restore the mechanical axis of the knee, decrease the load on the medial joint compartment, and delay the progression of MCOA.^1-4 Due to favorable outcomes^5-7 in appropriately indicated young and active patients and technical developments, HTO has recently become more popular as a corrective and joint-preserving surgery. HTO being a very technically detailed operation,⁸ any patient offered HTO should be informed in detail before surgery.

YouTube is a global social network and an open source of information with over two billion monthly logged-in users in over 100 countries around the world, across 80 languages, and is a rapidly growing visual database with over 500 h of content uploaded every minute.⁹

Although started as a recreational and entertainment platform; YouTube, due to patient interest, has become a platform for medical information for academicians and colleagues as well as patients. Many studies have revealed that medical information on orthopaedic as well as other medical fields found on YouTube is of low quality and liability, misleading and poor.^10-16

The aim of this study was to examine the informative features and qualities of the videos with a high number of views that were shared on YouTube about HTO surgery as a treatment option in young patients with medial knee osteoarthritis in terms of the informative efficiency of the videos on disease and treatment options.

While YouTube is a very popular video platform with no supervision by healthcare professionals, this study hypothesized that the videos on YouTube were providing poor medical information on HTO.

MATERIALS AND METHODS

A search with the keywords “high tibial osteotomy”, “HTO”, “proximal tibial osteotomy”, “tibia osteotomy” and “knee osteotomy” was performed on YouTube on October 8, 2020. Non-English videos, repeated videos, inhumane videos, videos without any audio or text narration, and videos that were viewed less than 10,000 times were excluded. As the primary results, the included videos were evaluated individually by two orthopaedic surgeons using two different scoring systems: modified DISCERN score and JAMA Benchmark score. For secondary results, the videos were categorized according to publishing year, video type (animation, live surgery, patient diary, etc.), video length, number of views, and view rates. Any correlation between these categories and JAMA and modified DISCERN scores were analyzed.

DISCERN is a scoring system developed at Oxford and used to evaluate the quality of health care information, originally made up of 16 questions.¹⁷ Establishing a scoring system for clarity, reliability, bias/balance, providing additional information, and uncertainty criteria, Singh et al.¹⁵ modified DISCERN for the evaluation of YouTube. The reliability of information was scored from 0 to 5 (reliability score) based on 5 questions for the reliability and completeness of information. A higher modified DISCERN score means higher reliability.

The JAMA benchmarks were published as a suggestion for basic quality standards for internet information on health care.¹⁸ It evaluates four features that must be clearly visible on a website: authorship (writers and contributors), attribution of references (references for all content and copyright information), disclosure (potential conflict of interest of the website), and currency (the dates on which the content was uploaded and updated). Scores are between 0 and 4, in which a higher score indicates higher video reliability.

The study was conducted in accordance with the Declaration of Helsinki Principles.

Statistical Analysis

Data were analyzed with IBM^® SPSS^® Statistics 23 (IBM^®, Armonk, NY, United States). The conformity to normal distribution was examined using the Shapiro-Wilk test. The Kruskal-Wallis test was used to compare quantitative data that were not normally distributed to groups of three or more. Spearman’s rho correlation coefficient was used to examine the relationship between non-normally distributed quantitative variables. The in-class correlation coefficient was used to examine the agreement between the first and second experts. Analysis results are mean ± standard deviation for quantitative data presented as deviation and median (minimum-maximum). The significance level was taken as p<0.05.

RESULTS

When the exclusion criteria were applied, 26 videos were available for evaluation. 19.2% of the videos were uploaded in 2011 and 42.3% of the videos were live surgery recordings. Descriptive statistics for categorical variables are given in Table 1. Mean-modified DISCERN and JAMA Benchmark scores were 2.6 and 2.0 for observer 1 and 2.5 and 1.9 for observer 2, respectively. The mean length of the videos was 538.2 sec., whereas the shortest was 16.0 sec. and the longest was 3131.0 sec. The maximum view rate of the videos was 18979.0 views per year (v/yr), minimum was 928.0 v/yr, and the mean view rate was 6198.2 v/yr. (Table 2). Video types and median value of video lengths had significant difference (p=0.013). The median values of animated videos, live surgery recordings, and other types were 187.00 sec., 606.00 sec., and 301.50 s, respectively. Video type and other variablesmedian values showed no difference (p>0.050) (Table 3).

Table 1

Table 2

Table 3

Video publishing year groups and view rate variables showed a statistically significant difference (p=0.003). The median view rates of the videos published between 2008-2011, 2012-2015 and 2015-2020 was 2559.00, 3314.0, and 7458.00, respectively. Other variables showed no significant difference compared to the publishing year groups (p>0.050) (Table 4).

Table 4

Total JAMA Benchmark scores by observer 1(p=0.015; r=0.473) and observer 2 (p=0.041; r=0.403) were significantly positively correlated with video length values. A weak positive correlation between mean JAMA benchmark and view rate values was found (p=0.047; r=0.393). A positive correlation between modified DISCERN scores by observer 1 (p<0.001; r=0.639) and observer 2 (p=0.003; r=0.563) and video length values was found. Mean-modified DISCERN scores and video length values had a statistically significant strong correlation (p=0.001; r=0.609) (Table 5).

Table 5

The statistical analysis of JAMA (ICC=0,860) and DISCERN (ICC=0.867) scores showed a strong correlation between both observers (Table 6). A significant correlation was also found between the mean JAMA Benchmark and mean-modified DISCERN scores. Mean-modified DISCERN scores showed a moderate increase correlated with the mean JAMA Benchmark score increase (r=0.463; p=0.017).

Table 6

DISCUSSION

The most important finding of this study is that the informative quality and reliability of YouTube videos concerning HTO are low and that they did not improve over the last decade, which proves our hypothesis true. In 2017, a study analyzing the quality of YouTube videos on anterior cruciate ligament injury and reconstruction¹⁰showed that the videos on YouTube were of low quality with a mean modified DISCERN score between 2.2 and 2.3 and a mean JAMA score between 2.3 and 2.5. Another study in 2016 analyzed the informative quality of YouTube videos on the diagnosis and treatment of hip arthritis¹³ and showed that 86% of the videos provided poor information capacity, while only 3% provided excellent quality. The study concluded that YouTube is a poor source of accurate information about the diagnosis and treatment of hip arthritis. Several recent studies have researched the informative quality of YouTube videos on several common orthopedic problems and procedures, such as rehabilitation and return to sports after anterior cruciate ligament reconstruction, developmental dysplasia of the hip, total hip/knee arthroplasty, posterior cruciate ligament injuries, rotator cuff injuries, and treatment of Bankart lesions. All referred studies concluded that the informative/educational quality and reliability of YouTube videos were poor.^19-24 Among a large number of studies, we did not find any study that analyzed YouTube content concerning HTO. The results of our study were no exception to other studies mentioned above. Similar to the results of other studies analyzing videos on other orthopedic issues, our results showed that YouTube videos concerning HTO were poor with a mean JAMA score of 2 and a mean modified DISCERN score of 2.6.

Cassidy et al.¹⁰ showed that neither the video quality or source, nor the number of views had no correlation with the recorded scores. Other studies^22-24 stated that video content uploaded by medical professionals/academicians had higher informative quality and reliability. A study that analyzed video content quality on YouTube concerning Bankart lesion and its treatment showed that there was no difference in DISCERN or JAMA scores according to video type.¹⁹ Similarly, we found no significant difference after the comparison of the scores of three different groups of video types. The type of the video did not have any effect on the scores, which means that animations, live surgery videos, and other types (webinar, seminar, patient diary etc.) had similar low informative quality and reliability. Moreover, animations and live surgery videos had similar view rates. In contrast to live surgery videos, animations were significantly shorter in time. A recent study analyzing YouTube videos on arteriovenous malformations showed that videos featuring animations had higher popularity compared to other types of videos.²⁵ The practical translation of this may be that short animations may be enough to inform patients instead of recording live surgery videos and not jeopardizing sterility and concentration during surgery.

Another result of this study was that video length and view rate showed positive correlations with both JAMA and modified DISCERN scores. A study analyzing the quality of testicular cancer videos on YouTube revealed a positive correlation between video length and both JAMA and DISCERN scores.²⁶ Similarly, another study on the informative quality of YouTube videos on hallux valgus found a positive correlation between video length and DISCERN score.²⁷ This may be explained either by the fact that answers to questions that need answering for higher quality and reliability require more time; or that any information that comes up incidentally as the video gets longer addresses the required answer to a question.

Videos from three different time periods (2008-11, 2012-15 and 2015-19) had similar poor JAMA and modified DISCERN scores. This may be interpreted as that orthopaedic society failed to improve itself in terms of self-expression and education on digital media in the last decade. View rates, however, increased in time from 2559 views per year between 2008 and 2011 to 9666 between 2015 and 2019. This may be explained by society’s increasing habit of using digital streaming platforms and/or increasing use of the Internet by patients to research health-related issues.²⁸

Moreover, the number of YouTube videos uploaded between 2015 and 2019 comprises 38% of all included videos, with this period having the highest number of videos. Similarly, the view rate increased from 3314 per year between 2011 and 2015 to 7458 between 2015 and 2019. A search on PubMed with the keywords “HTO AND proximal tibial osteotomy” between 2008 and 2019 resulted in 281 studies, where 182 (68%) of them were published between 2015 and 2019. The increase in the number of videos, view rate, and number of studies published in PubMed may be regarded as a confirmation of the “re-interest” in HTO recently.

In addition, the consistency in the comparison of JAMA and DISCERN scores indicates the reliability, precision, and value of these scales in their evaluative qualities. Similarly, a recent study that evaluated the quality of YouTube videos on spondylolisthesis showed that DISCERN, JAMA, and Global Quality Score scores correlated among themselves.²⁹

Study Limitations

There are several limitations to our study. First of all, this study evaluates videos only on a single platform, namely YouTube. Videos on other popular digital platforms might have provided different results. On the other hand, because of the lack of validated tools specifically designed to evaluate online video content, we used the JAMA Benchmark scoring system that has been developed for written content, not for video content and the modified DISCERN scoring system that was developed for YouTube video content.¹⁵ Another limitation of the scoring systems used in this study is that both systems are the observer dependent. Moreover, by selecting videos with a view number over 10.000, we may have overlooked videos that may be probably of higher quality but with a smaller number of views.

CONCLUSION

Although the view rate has increased over time, the informative quality and reliability of YouTube videos concerning HTO are low and did not improve over the last decade.

MAIN POINTS

• Although the view rate has increased over time, the informative quality and reliability of YouTube videos concerning HTO are low and did not improve over the last decade.

• Over time, the change in the social media usage habits of the population is also seen in the field of health. It seems that the increase in viewer demand for informative videos on this subject has also increased the number of shared videos over time.

• JAMA or modified DISCERN sores can be used for the evaluation or standardization of informative videos.

ETHICS

Ethics Committee Approval: Ethics committee form was not required.

Informed Consent: Informed consent was not required.

Peer-review: Externally peer-reviewed.

Authorship Contributions

Concept: K.Y., M.Y., Design: K.Y., M.Y., Materials: K.Y., Data Collection and/or Processing: K.Y., Analysis and/or Interpretation: K.Y., M.Y., Literature Search: K.Y., Writing: K.Y., M.Y., Critical Review: M.Y.

DISCLOSURES

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study had received no financial support.

References

Jackson JP, Waugh W. Tibial osteotomy for osteoarthritis of the knee. J Bone Joint Surg Br. 1961;43-B:746-51.

Jung WH, Takeuchi R, Chun CW, Lee JS, Ha JH, Kim JH, et al. Second-look arthroscopic assessment of cartilage regeneration after medial opening-wedge high tibial osteotomy. Arthroscopy. 2014; 30(1): 72-9.

Koshino T, Wada S, Ara Y, Saito T. Regeneration of degenerated articular cartilage after high tibial valgus osteotomy for medial compartmental osteoarthritis of the knee. Knee. 2003; 10(3): 229-36.

Thambiah MD, Tan MKL, Hui JHP. Role of High Tibial Osteotomy in Cartilage Regeneration - Is Correction of Malalignment Mandatory for Success? Indian J Orthop. 2017; 51(5): 588-99.

Bonasia DE, Dettoni F, Sito G, Blonna D, Marmotti A, Bruzzone M, et al. Medial opening wedge high tibial osteotomy for medial compartment overload/arthritis in the varus knee: prognostic factors. Am J Sports Med. 2014; 42(3): 690-8.

Day M, Wolf BR. Medial Opening-Wedge High Tibial Osteotomy for Medial Compartment Arthrosis/Overload. Clin Sports Med. 2019; 38(3): 331-49.

Noyes FR, Mayfield W, Barber-Westin SD, Albright JC, Heckmann TP. Opening wedge high tibial osteotomy: an operative technique and rehabilitation program to decrease complications and promote early union and function. Am J Sports Med. 2006; 34(8): 1262-73.

Pehlivanoglu T, Yildirim K, Beyzadeoglu T. High Tibial Osteotomy. In: Nikolopoulos DD, Safos GK, Michos J, editors. Tibia Pathology and Fractures, IntechOpen; 2020.p.109-31.

YouTube. (2020). https://blog.youtube/press/

Cassidy JT, Fitzgerald E, Cassidy ES, Cleary M, Byrne DP, Devitt BM, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018; 26(3): 840-5.

Gokcen HB, Gumussuyu G. A Quality Analysis of Disc Herniation Videos on YouTube. World Neurosurg. 2019; 124: e799-e804.

Kaicker J, Dang W, Mondal T. Assessing the quality and reliability of health information on ERCP using the DISCERN instrument. Health Care Curr Rev. 2013; 1(1): 1-4.

Koller U, Waldstein W, Schatz KD, Windhager R. YouTube provides irrelevant information for the diagnosis and treatment of hip arthritis. Int Orthop. 2016; 40(10): 1995-2002.

O’Neill SC, Baker JF, Fitzgerald C, Fleming C, Rowan F, Byrne D, et al. Cauda equina syndrome: assessing the readability and quality of patient information on the Internet. Spine (Phila Pa 1976). 2014; 39(10): E645-9.

Singh AG, Singh S, Singh PP. YouTube for information on rheumatoid arthritis--a wakeup call? J Rheumatol. 2012; 39(5): 899-903.

Smith PE, McGuire J, Falci M, Poudel DR, Kaufman R, Patterson MA, et al. Analysis of YouTube as a Source of Information for Diabetic Foot Care. J Am Podiatr Med Assoc. 2019; 109(2): 122-6.

Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999 ; 53(2): 105-11.

Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA. 1997; 277(15): 1244-5.

Akpolat AO, Kurdal DP. Is quality of YouTube content on Bankart lesion and its surgical treatment adequate? J Orthop Surg Res. 2020; 15(1): 78.

Kunze KN, Cohn MR, Wakefield C, Hamati F, LaPrade RF, Forsythe B, et al. YouTube as a Source of Information About the Posterior Cruciate Ligament: A Content-Quality and Reliability Analysis. Arthrosc Sports Med Rehabil. 2019; 1(2): e109-14.

Kuru T, Erken HY. Evaluation of the Quality and Reliability of YouTube Videos on Rotator Cuff Tears. Cureus. 2020; 12(2): e6852.

Ng MK, Emara AK, Molloy RM, Krebs VE, Mont M, Piuzzi NS. YouTube as a Source of Patient Information for Total Knee/Hip Arthroplasty: Quantitative Analysis of Video Reliability, Quality, and Content. J Am Acad Orthop Surg. 2021; 29(20): e1034-44.

Oztermeli A, Karahan N. Evaluation of YouTube Video Content About Developmental Dysplasia of the Hip. Cureus. 2020; 12(8): e9557.

Springer B, Bechler U, Koller U, Windhager R, Waldstein W. Online Videos Provide Poor Information Quality, Reliability, and Accuracy Regarding Rehabilitation and Return to Sport After Anterior Cruciate Ligament Reconstruction. Arthroscopy. 2020; 36(12): 3037-47.

Krakowiak M, Szmuda T, Fercho J, Ali S, Maliszewska Z, Słoniewski P. YouTube as a source of information for arteriovenous malformations: A content-quality and optimization analysis. Clin Neurol Neurosurg. 2021; 207: 106723.

Duran MB, Kizilkan Y. Quality analysis of testicular cancer videos on YouTube. Andrologia. 2021; 53(8): e14118.

Sari E, Umur LF. Quality Analysis of Hallux Valgus Videos on YouTube. J Am Podiatr Med Assoc. 2021; 111(5).

Cassidy JT, Baker JF. Orthopaedic Patient Information on the World Wide Web: An Essential Review. J Bone Joint Surg Am. 2016; 98(4): 325-38.

Yaradilmis YU, Evren AT, Okkaoglu, MC, Ozturk O, Haberal B, Ozdemir M. Evaluation of quality and reliability of YouTube videos on spondylolisthesis. Interdisciplinary Neurosurgery. 2020; 22: 100827.

YouTube Videos Provide Poor Information on High Tibial Osteotomy