![]() The body of research investigating technologies designed to assist with data extraction, one of the most time- and resource-intensive steps of completing a systematic review, is comparatively immature. ![]() Most research investigating the use of machine learning tools in systematic reviews has focused on creating efficiencies during the study selection step. To date, nearly 200 software tools aimed at facilitating systematic review processes have been developed, with machine learning and text mining being the driver behind the proposed efficiencies of many tools. Since living systematic reviews are held to the same methodological standards as traditional systematic reviews, the efficiency of their production will be critical to their feasibility and sustainability Because living systematic reviews are updated in real time, the total workload for keeping them up to date is broken down into more manageable tasks. Living systematic reviews, which are continually updated as new evidence becomes available, represent a relatively new form of evidence synthesis aimed at addressing the heavy workload and fleeting currency associated with most traditional systematic reviews. Especially in rapidly evolving fields, it is no longer feasible for traditional systematic review production to keep pace with the publication of new trial data, seriously undermining the currency, validity, and utility of even the most recently published reviews.Īs the number of newly registered randomized trials continues to grow, the need to create efficiencies in the production of systematic reviews is increasingly pressing. A typical systematic review will take a highly skilled team of clinician-experts, methodologists, and statisticians many months or even years to complete. To be useful, systematic reviews must be conducted with a high degree of methodological rigor, and are therefore time and resource intensive. Timely systematic reviews provide an indispensable resource for decision makers, many of whom lack the time and expertise to independently identify and evaluate new evidence. The tool’s ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required. The tool was reliable for identifying the reporting of most data elements. Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. 21.6 h total extraction time across 75 randomized trials). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. A median 48% of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Among a median (IQR) 90% (86% to 97%) of relevant sentences, pertinent fragments had been highlighted by the tool exact matches were unreliable (median (IQR) 52% ). Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88% (83% to 99%) of cases. The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91% (75% to 99%) accuracy. We calculated the median (interquartile range ) time for manual and semi-automated data extraction, and overall time savings. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by the tool. We uploaded the randomized trials to an online machine learning and text mining tool, and quantified performance by evaluating its ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. Methodsįor 75 randomized trials, we manually extracted and verified data for 21 data elements. We evaluated a machine learning and text mining tool’s ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification. Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production.
0 Comments
Leave a Reply. |