The article written above was describing the recent success of an artificial intelligence model “Aristo”, developed by the Allen Institute for Artificial Intelligence, in correctly answering 90% of questions correctly on an 8th grade science exam. This news article does well in describing the history of the competition surrounding this challenge, and the historical failures of previous models. Oversimplification of some findings in the research article occurs and the author does not substantiate some claims made with evidence. The title is not clickbait, and overall is representative of the article. The author does a good job of avoiding the injection of personal opinion in the article, and provides perspectives from members of the team at the Allen Institute. Additionally, opinions from other Microsoft researchers and a Fast.AI employee that were somewhat skeptical of the results were included, providing a wide range of insight into the topic. Intent of the article is clearly mentioned in the title, and the news story correctly describes the intent in the main body of the text as well. Results from the original research article are listed in the form of test scores of the system and are correctly described. However, gaps in reasoning do exist when mentioning future applications of the models used, with no mention on why internet search engines or record-keeping systems may be affected by Aristo. No sources were provided to back up claims on the range of areas that this program will apply to and affect, including newly developed systems. Some claims were oversimplified, such as how the program “correctly answered 90% of the questions”, without specifying that these questions were only non-diagram, multiple choice questions. Overall, there was an appropriate context for article claims and statements, with no cherry picking of results and no logical fallacies committed. No fear mongering or sensationalist language was used, and there was no exaggeration or minimization of the aspects of the article, which is a pleasant surprise given the nature of the portrayal of A.I. in the media. The vocabulary is suited for the target audience, with nothing overtly technical. Furthermore, jargon used—such as neural networks—is explained in layman’s terms and for those with general knowledge. For further research, although there is no link to the Aristo research article itself, the link provided above does lead to the open access paper, such that anyone can read for themselves from the source.
The news article covered a recent publication by the Allen Institute of results on their machine learning software Aristo. The content of the article was informative, and the language was well suited to the audience. The article accurately reported some of the results from the paper and emphasized the use of Bidirectional Encoder Representations from Transformers (BERT) software as a large part of the success for Aristo. While all of the information presented in the article is factually correct, much of the information lacks appropriate context and could be misleading. The article did not justify its emphasis on BERT software, as BERT was only of 8 solvers used by Aristo. As well, the author included some unsourced claims and irrelevant information that directed reader’s attention to other unrelated articles that the author written. While multiple perspectives and sources were provided, they were short and unhelpful in understanding the topic of AI. The new perspectives were used for a single quote about AI in different contexts, although these perspectives were from credible sources. The story would benefit greatly from the availability of a transcript for these conversations and more context for the quotes provided. As stated in the official research article published by the Allen Institute, the intent of the research was to find and test appropriate benchmarks for machine intelligence using Aristo. The news story mentions the intent of the research but misleadingly identifies intent with the use of BERT technology and some claims are overstated from their description in the original research paper. The story does contain a casual unsourced claim: “The new research could lead to systems that can carry on a decent conversation.” Additionally, there are gaps in reasoning that are problematic. The quote from Oren Etzioni about the business potential of Aristo has absolutely no context or justification and is misleading and confusing. Another quote—“The system, called Aristo, is an indication that in just the past several months researchers have made significant progress in developing A.I.”—is unjustified because it provides no explanation. In reality, much of the research for Aristo has been ongoing for the past 10 years. While the author includes links to other articles that he has written by add ambiguous claims and statements containing the link. These added statements are unnecessary and only vaguely related to the article. The views expressed by the reviewers for this article are not endorsed or shared by SciFeye. The interpretation of the review of the news story using the SciFeye Index was done independently by two SciFeye reviewers. We encourage you to conduct your own evaluation of the accuracy and quality of the news story using the Index.
0 Comments
Leave a Reply. |
