探花直播 of Cambridge - 探花直播Alan Turing Institute /taxonomy/external-affiliations/the-alan-turing-institute en Fully AI driven weather prediction system could start revolution in forecasting /research/news/fully-ai-driven-weather-prediction-system-could-start-revolution-in-forecasting <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/rt-aardvark-results-1-dp.jpg?itok=JXS2j86k" alt="Scientist looking at a computer screen with two weather forecasts" title="Professor Richard Turner using Aardvark Weather, Credit: 探花直播Alan Turing Institute" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p> 探花直播system, Aardvark Weather, has been supported by the Alan Turing Institute, Microsoft Research and the European Centre for Medium Range Weather Forecasts. It provides a blueprint for a new approach to weather forecasting with the potential to transform current practices. The<a href="https://www.nature.com/articles/s41586-025-08897-0"> results</a> are reported in the journal <em>Nature</em>.</p> <p>鈥淎ardvark reimagines current weather prediction methods offering the potential to make weather forecasts faster, cheaper, more flexible and more accurate than ever before, helping to transform weather prediction in both developed and developing countries,鈥 said Professor Richard Turner from Cambridge鈥檚 Department of Engineering, who led the research. 鈥淎ardvark is thousands of times faster than all previous weather forecasting methods.鈥</p> <p>Current weather forecasts are generated through a complex set of stages, each taking several hours to run on powerful supercomputers. Aside from daily usage, the development, maintenance and use of these systems require significant time and large teams of experts.</p> <p>More recently, research by Huawei, Google, and Microsoft has shown that one component of the weather forecasting pipeline, the numerical solver (which calculates how weather evolves over time), can be replaced with AI, resulting in faster and more accurate predictions. This combination of AI and traditional approaches is now being used by the European Centre for Medium Range Weather Forecasts (ECMWF).</p> <p>But with Aardvark, researchers have replaced the entire weather prediction pipeline with a single, simple machine learning model. 探花直播new model takes in observations from satellites, weather stations and other sensors and outputs both global and local forecasts.</p> <p>This fully AI driven approach means predictions that were once produced using many models 鈥 each requiring a supercomputer and a large support team to run 鈥 can now be produced in minutes on a desktop computer.</p> <p>When using just 10% of the input data of existing systems, Aardvark already outperforms the United States national GFS forecasting system on many variables. It is also competitive with United States Weather Service forecasts that use input from dozens of weather models and analysis by expert human forecasters.</p> <p>鈥淭hese results are just the beginning of what Aardvark can achieve,鈥 said first author Anna Allen, from Cambridge鈥檚 Department of Computer Science and Technology. 鈥淭his end-to-end learning approach can be easily applied to other weather forecasting problems, for example hurricanes, wildfires, and tornadoes. Beyond weather, its applications extend to broader Earth system forecasting, including air quality, ocean dynamics, and sea ice prediction.鈥</p> <p> 探花直播researchers say that one of the most exciting aspects of Aardvark is its flexibility and simple design. Because it learns directly from data it can be quickly adapted to produce bespoke forecasts for specific industries or locations, whether that's predicting temperatures for African agriculture or wind speeds for a renewable energy company in Europe.</p> <p>This contrasts to traditional weather prediction systems where creating a customised system takes years of work by large teams of researchers.</p> <p>鈥 探花直播weather forecasting systems we all rely on have been developed over decades, but in just 18 months, we鈥檝e been able to build something that鈥檚 competitive with the best of these systems, using just a tenth of the data on a desktop computer,鈥 said Turner, who is also Lead Researcher for Weather Prediction at the Alan Turing Institute.</p> <p>This capability has the potential to transform weather prediction in developing countries where access to the expertise and computational resources required to develop conventional systems is not typically available.</p> <p>鈥淯nleashing AI鈥檚 potential will transform decision-making for everyone from policymakers and emergency planners to industries that rely on accurate weather forecasts,鈥 said Dr Scott Hosking from 探花直播Alan Turing Institute. 鈥淎ardvark鈥檚 breakthrough is not just about speed, it鈥檚 about access. By shifting weather prediction from supercomputers to desktop computers, we can democratise forecasting, making these powerful technologies available to developing nations and data-sparse regions around the world.鈥</p> <p>鈥淎ardvark would not have been possible without decades of physical-model development by the community, and we are particularly indebted to ECMWF for their ERA5 dataset which is essential for training Aardvark,鈥 said Turner.</p> <p>鈥淚t is essential that academia and industry work together to address technological challenges and leverage new opportunities that AI offers,鈥 said Matthew Chantry from ECMWF. 鈥淎ardvark鈥檚 approach combines both modularity with end-to-end forecasting optimisation, ensuring effective use of the available datasets."</p> <p>鈥淎ardvark represents not only an important achievement in AI weather prediction but it also reflects the power of collaboration and bringing the research community together to improve and apply AI technology in meaningful ways,鈥 said Dr Chris Bishop, from Microsoft Research.</p> <p> 探花直播next steps for Aardvark include developing a new team within the Alan Turing Institute led by Turner, who will explore the potential to deploy Aardvark in the global south and integrate the technology into the Institute鈥檚 wider work to develop high-precision environmental forecasting for weather, oceans and sea ice.</p> <p><em><strong>Reference:</strong><br /> Anna Allen, Stratis Markou et al. 鈥<a href="https://www.nature.com/articles/s41586-025-08897-0">End-to-end data-driven weather prediction</a>.鈥 Nature (2025). DOI: 10.1038/s41586-025-08897-0</em></p> <p><em>Adapted from a media release by 探花直播Alan Turing Institute</em></p> </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>A new AI weather prediction system, developed by researchers from the 探花直播 of Cambridge, can deliver accurate forecasts tens of times faster and using thousands of times less computing power than current AI and physics-based forecasting systems.</p> </p></div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="/www.turing.ac.k" target="_blank"> 探花直播Alan Turing Institute</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Professor Richard Turner using Aardvark Weather</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license"><img alt="Creative Commons License." src="/sites/www.cam.ac.uk/files/inner-images/cc-by-nc-sa-4-license.png" style="border-width: 0px; width: 88px; height: 31px;" /></a><br /> 探花直播text in this work is licensed under a <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways 鈥 on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p> </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Thu, 20 Mar 2025 15:56:54 +0000 sc604 248791 at Under the bonnet at Dawn, the UK's fastest AI supercomputer /stories/under-the-bonnet-at-AI-supercomputer-Dawn <div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>How AI supercomputer, Dawn, is being used to tackle some of the most pressing issues facing humanity.</p> </p></div></div></div> Mon, 17 Mar 2025 15:58:08 +0000 hcf38 248780 at Artificial intelligence outperforms clinical tests at predicting progress of Alzheimer鈥檚 disease /research/news/artificial-intelligence-outperforms-clinical-tests-at-predicting-progress-of-alzheimers-disease <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/gettyimages-1357965100-web.jpg?itok=GwKB7a8J" alt="Brain on molecular structure, circuitry, and programming code background" title="Brain on molecular structure, circuitry, and programming code background, Credit: Yuichiro Chino (Getty Images)" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p> 探花直播team say this new approach could reduce the need for invasive and costly diagnostic tests while improving treatment outcomes early when interventions such as lifestyle changes or new medicines may have a chance to work best.</p>&#13; &#13; <p>Dementia poses a significant global healthcare challenge, affecting over 55 million people worldwide at an estimated annual cost of $820 billion. 探花直播number of cases is expected to almost treble over the next 50 years.</p>&#13; &#13; <p> 探花直播main cause of dementia is Alzheimer鈥檚 disease, which accounts for 60-80% of cases. Early detection is crucial as this is when treatments are likely to be most effective, yet early dementia diagnosis and prognosis may not be accurate without the use of invasive or expensive tests such as positron emission tomography (PET) scans or lumbar puncture, which are not available in all memory clinics. As a result, up to a third of patients may be misdiagnosed and others diagnosed too late for treatment to be effective.</p>&#13; &#13; <p>A team led by scientists from the Department of Psychology at the 探花直播 of Cambridge has developed a machine learning model able to predict whether and how fast an individual with mild memory and thinking problems will progress to developing Alzheimer鈥檚 disease. In research published today in <em>eClinical Medicine</em>, they show that it is more accurate than current clinical diagnostic tools.</p>&#13; &#13; <p>To build their model, the researchers used routinely-collected, non-invasive, and low-cost patient data 鈥 cognitive tests and structural MRI scans showing grey matter atrophy 鈥 from over 400 individuals who were part of a research cohort in the USA.</p>&#13; &#13; <p>They then tested the model using real-world patient data from a further 600 participants from the US cohort and 鈥 importantly 鈥 longitudinal data from 900 people from memory clinics in the UK and Singapore.</p>&#13; &#13; <p> 探花直播algorithm was able to distinguish between people with stable mild cognitive impairment and those who progressed to Alzheimer鈥檚 disease within a three-year period. It was able to correctly identify individuals who went on to develop Alzheimer鈥檚 in 82% of cases and correctly identify those who didn鈥檛 in 81% of cases from cognitive tests and an MRI scan alone.</p>&#13; &#13; <p> 探花直播algorithm was around three times more accurate at predicting the progression to Alzheimer鈥檚 than the current standard of care; that is, standard clinical markers (such as grey matter atrophy or cognitive scores) or clinical diagnosis. This shows that the model could significantly reduce misdiagnosis.</p>&#13; &#13; <p> 探花直播model also allowed the researchers to stratify people with Alzheimer鈥檚 disease using data from each person鈥檚 first visit at the memory clinic into three groups: those whose symptoms would remain stable (around 50% of participants), those who would progress to Alzheimer鈥檚 slowly (around 35%) and those who would progress more rapidly (the remaining 15%). These predictions were validated when looking at follow-up data over 6 years. This is important as it could help identify those people at an early enough stage that they may benefit from new treatments, while also identifying those people who need close monitoring as their condition is likely to deteriorate rapidly.</p>&#13; &#13; <p>Importantly, those 50% of people who have symptoms such as memory loss but remain stable, would be better directed to a different clinical pathway as their symptoms may be due to other causes rather than dementia, such as anxiety or depression.</p>&#13; &#13; <p>Senior author Professor Zoe Kourtzi from the Department of Psychology at the 探花直播 of Cambridge said: 鈥淲e鈥檝e created a tool which, despite using only data from cognitive tests and MRI scans, is much more sensitive than current approaches at predicting whether someone will progress from mild symptoms to Alzheimer鈥檚 鈥 and if so, whether this progress will be fast or slow.</p>&#13; &#13; <p>鈥淭his has the potential to significantly improve patient wellbeing, showing us which people need closest care, while removing the anxiety for those patients we predict will remain stable. At a time of intense pressure on healthcare resources, this will also help remove the need for unnecessary invasive and costly diagnostic tests.鈥</p>&#13; &#13; <p>While the researchers tested the algorithm on data from a research cohort, it was validated using independent data that included almost 900 individuals who attended memory clinics in the UK and Singapore. In the UK, patients were recruited through the Quantiative MRI in NHS Memory Clinics Study (QMIN-MC) led by study co-author Dr Timothy Rittman at Cambridge 探花直播 Hospitals NHS Trust and Cambridgeshire and Peterborough NHS Foundation Trusts (CPFT).</p>&#13; &#13; <p> 探花直播researchers say this shows it should be applicable in a real-world patient, clinical setting.</p>&#13; &#13; <p>Dr Ben Underwood, Honorary Consultant Psychiatrist at CPFT and assistant professor at the Department of Psychiatry, 探花直播 of Cambridge, said: 鈥淢emory problems are common as we get older. In clinic I see how uncertainty about whether these might be the first signs of dementia can cause a lot of worry for people and their families, as well as being frustrating for doctors who would much prefer to give definitive answers. 探花直播fact that we might be able to reduce this uncertainty with information we already have is exciting and is likely to become even more important as new treatments emerge.鈥</p>&#13; &#13; <p>Professor Kourtzi said: 鈥淎I models are only as good as the data they are trained on. To make sure ours has the potential to be adopted in a healthcare setting, we trained and tested it on routinely-collected data not just from research cohorts, but from patients in actual memory clinics. This shows it will be generalisable to a real-world setting.鈥</p>&#13; &#13; <p> 探花直播team now hope to extend their model to other forms of dementia, such as vascular dementia and frontotemporal dementia, and using different types of data, such as markers from blood tests.</p>&#13; &#13; <p>Professor Kourtzi added: 鈥淚f we鈥檙e going to tackle the growing health challenge presented by dementia, we will need better tools for identifying and intervening at the earliest possible stage. Our vision is to scale up our AI tool to help clinicians assign the right person at the right time to the right diagnostic and treatment pathway. Our tool can help match the right patients to clinical trials, accelerating new drug discovery for disease modifying treatments.鈥</p>&#13; &#13; <p>This work was in collaboration with a cross-disciplinary team including Professor Peter Tino at the 探花直播 of Birmingham and Professor Christopher Chen at the National 探花直播 of Singapore. It聽was funded by Wellcome, the Royal Society, Alzheimer鈥檚 Research UK, the Alzheimer鈥檚 Drug Discovery Foundation Diagnostics Accelerator, the Alan Turing Institute, and the National Institute for Health and Care Research Cambridge Biomedical Research Centre.</p>&#13; &#13; <p><em><strong>Reference</strong><br />&#13; Lee, LY &amp; Vaghari, D et al. <a href="https://doi.org/10.1016/j.eclinm.2024.102725">Robust and interpretable AI-guided marker for early dementia prediction in real-world clinical settings.</a> eClinMed; 12 July 2024; DOI: 10.1016/j.eclinm.2024.102725</em></p>&#13; </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Cambridge scientists have developed an artificially-intelligent tool capable of predicting in four cases out of five whether people with early signs of dementia will remain stable or develop Alzheimer鈥檚 disease.</p>&#13; </p></div></div></div><div class="field field-name-field-content-quote field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even">We鈥檝e created a tool which is much more sensitive than current approaches at predicting whether someone will progress from mild symptoms to Alzheimer鈥檚</div></div></div><div class="field field-name-field-content-quote-name field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Zoe Kourtzi</div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="https://www.gettyimages.co.uk/detail/photo/brain-of-neuro-technology-royalty-free-image/1357965100?phrase=artificial intelligence mental health" target="_blank">Yuichiro Chino (Getty Images)</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Brain on molecular structure, circuitry, and programming code background</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license"><img alt="Creative Commons License." src="/sites/www.cam.ac.uk/files/inner-images/cc-by-nc-sa-4-license.png" style="border-width: 0px; width: 88px; height: 31px;" /></a><br />&#13; 探花直播text in this work is licensed under a <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways 鈥 on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p>&#13; </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Fri, 12 Jul 2024 22:30:52 +0000 cjb250 246841 at New open-source platform allows users to evaluate performance of AI-powered chatbots /research/news/new-open-source-platform-allows-users-to-evaluate-performance-of-ai-powered-chatbots <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/gettyimages-1485822619-dp_0.jpg?itok=YW1eav0N" alt="Chatbot" title="Chatbot, Credit: da-kuk via Getty Images" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p>A team of computer scientists, engineers, mathematicians and cognitive scientists, led by the 探花直播 of Cambridge, developed an open-source evaluation platform called CheckMate, which allows human users to interact with and evaluate the performance of large language models (LLMs).</p> <p> 探花直播researchers tested CheckMate in an experiment where human participants used three LLMs 鈥 InstructGPT, ChatGPT and GPT-4 鈥 as assistants for solving undergraduate-level mathematics problems.</p> <p> 探花直播team studied how well LLMs can assist participants in solving problems. Despite a generally positive correlation between a chatbot鈥檚 correctness and perceived helpfulness, the researchers also found instances where the LLMs were incorrect, but still useful for the participants. However, certain incorrect LLM outputs were thought to be correct by participants. This was most notable in LLMs optimised for chat.</p> <p> 探花直播researchers suggest models that communicate uncertainty, respond well to user corrections, and can provide a concise rationale for their recommendations, make better assistants. Human users of LLMs should verify their outputs carefully, given their current shortcomings.</p> <p> 探花直播<a href="https://www.pnas.org/doi/10.1073/pnas.2318124121">results</a>, reported in the <em>Proceedings of the National Academy of Sciences (PNAS)</em>, could be useful in both informing AI literacy training, and help developers improve LLMs for a wider range of uses.</p> <p>While LLMs are becoming increasingly powerful, they can also make mistakes and provide incorrect information, which could have negative consequences as these systems become more integrated into our everyday lives.</p> <p>鈥淟LMs have become wildly popular, and evaluating their performance in a quantitative way is important, but we also need to evaluate how well these systems work with and can support people,鈥 said co-first author Albert Jiang, from Cambridge鈥檚 Department of Computer Science and Technology. 鈥淲e don鈥檛 yet have comprehensive ways of evaluating an LLM鈥檚 performance when interacting with humans.鈥</p> <p> 探花直播standard way to evaluate LLMs relies on static pairs of inputs and outputs, which disregards the interactive nature of chatbots, and how that changes their usefulness in different scenarios. 探花直播researchers developed CheckMate to help answer these questions, designed for but not limited to applications in mathematics.</p> <p>鈥淲hen talking to mathematicians about LLMs, many of them fall into one of two main camps: either they think that LLMs can produce complex mathematical proofs on their own, or that LLMs are incapable of simple arithmetic,鈥 said co-first author Katie Collins from the Department of Engineering. 鈥淥f course, the truth is probably somewhere in between, but we wanted to find a way of evaluating which tasks LLMs are suitable for and which they aren鈥檛.鈥</p> <p> 探花直播researchers recruited 25 mathematicians, from undergraduate students to senior professors, to interact with three different LLMs (InstructGPT, ChatGPT, and GPT-4) and evaluate their performance using CheckMate. Participants worked through undergraduate-level mathematical theorems with the assistance of an LLM and were asked to rate each individual LLM response for correctness and helpfulness. Participants did not know which LLM they were interacting with.</p> <p> 探花直播researchers recorded the sorts of questions asked by participants, how participants reacted when they were presented with a fully or partially incorrect answer, whether and how they attempted to correct the LLM, or if they asked for clarification. Participants had varying levels of experience with writing effective prompts for LLMs, and this often affected the quality of responses that the LLMs provided.</p> <p>An example of an effective prompt is 鈥渨hat is the definition of X鈥 (X being a concept in the problem) as chatbots can be very good at retrieving concepts they know of and explaining it to the user.</p> <p>鈥淥ne of the things we found is the surprising fallibility of these models,鈥 said Collins. 鈥淪ometimes, these LLMs will be really good at higher-level mathematics, and then they鈥檒l fail at something far simpler. It shows that it鈥檚 vital to think carefully about how to use LLMs effectively and appropriately.鈥</p> <p>However, like the LLMs, the human participants also made mistakes. 探花直播researchers asked participants to rate how confident they were in their own ability to solve the problem they were using the LLM for. In cases where the participant was less confident in their own abilities, they were more likely to rate incorrect generations by LLM as correct.</p> <p>鈥淭his kind of gets to a big challenge of evaluating LLMs, because they鈥檙e getting so good at generating nice, seemingly correct natural language, that it鈥檚 easy to be fooled by their responses,鈥 said Jiang. 鈥淚t also shows that while human evaluation is useful and important, it鈥檚 nuanced, and sometimes it鈥檚 wrong. Anyone using an LLM, for any application, should always pay attention to the output and verify it themselves.鈥</p> <p>Based on the results from CheckMate, the researchers say that newer generations of LLMs are increasingly able to collaborate helpfully and correctly with human users on undergraduate-level maths problems, as long as the user can assess the correctness of LLM-generated responses. Even if the answers may be memorised and can be found somewhere on the internet, LLMs have the advantage of being flexible in their inputs and outputs over traditional search engines (though should not replace search engines in their current form).</p> <p>While CheckMate was tested on mathematical problems, the researchers say their platform could be adapted to a wide range of fields. In the future, this type of feedback could be incorporated into the LLMs themselves, although none of the CheckMate feedback from the current study has been fed back into the models.</p> <p>鈥淭hese kinds of tools can help the research community to have a better understanding of the strengths and weaknesses of these models,鈥 said Collins. 鈥淲e wouldn鈥檛 use them as tools to solve complex mathematical problems on their own, but they can be useful assistants if the users know how to take advantage of them.鈥</p> <p> 探花直播research was supported in part by the Marshall Commission, the Cambridge Trust, Peterhouse, Cambridge, 探花直播Alan Turing Institute, the European Research Council, and the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).</p> <p>聽</p> <p><em><strong>Reference:</strong><br /> Katherine M聽Collins, Albert Q聽Jiang, et al. 鈥<a href="https://www.pnas.org/doi/10.1073/pnas.2318124121">Evaluating Language Models for Mathematics through Interactions</a>.鈥 Proceedings of the National Academy of Sciences (2024). DOI: 10.1073/pnas.2318124121</em></p> <p>聽</p> </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Researchers have developed a platform for the interactive evaluation of AI-powered chatbots such as ChatGPT.聽</p> </p></div></div></div><div class="field field-name-field-content-quote field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even">Anyone using an LLM, for any application, should always pay attention to the output and verify it themselves</div></div></div><div class="field field-name-field-content-quote-name field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Albert Jiang</div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="/" target="_blank">da-kuk via Getty Images</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Chatbot</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license"><img alt="Creative Commons License." src="/sites/www.cam.ac.uk/files/inner-images/cc-by-nc-sa-4-license.png" style="border-width: 0px; width: 88px; height: 31px;" /></a><br /> 探花直播text in this work is licensed under a <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways 鈥 on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p> </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Tue, 04 Jun 2024 10:34:36 +0000 sc604 246271 at How sure is sure? Incorporating human error into machine learning /research/news/how-sure-is-sure-incorporating-human-error-into-machine-learning <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/gettyimages-1477483014-dp.jpg?itok=9-VpM8kc" alt="Futuristic image of a doctor looking at brain scans" title="Futuristic image of a doctor looking at brain scans, Credit: PeopleImages via Getty Images" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p>Human error and uncertainty are concepts that many artificial intelligence systems fail to grasp, particularly in systems where a human provides feedback to a machine learning model. Many of these systems are programmed to assume that humans are always certain and correct, but real-world decision-making includes occasional mistakes and uncertainty.</p>&#13; &#13; <p>Researchers from the 探花直播 of Cambridge, along with 探花直播Alan Turing Institute, Princeton, and Google DeepMind, have been attempting to bridge the gap between human behaviour and machine learning, so that uncertainty can be more fully accounted for in AI applications where humans and machines are working together. This could help reduce risk and improve trust and reliability of these applications, especially where safety is critical, such as medical diagnosis.</p>&#13; &#13; <p> 探花直播team adapted a well-known image classification dataset so that humans could provide feedback and indicate their level of uncertainty when labelling a particular image. 探花直播researchers found that training with uncertain labels can improve these systems鈥 performance in handling uncertain feedback, although humans also cause the overall performance of these hybrid systems to drop. Their results will be reported at the <a href="https://www.aies-conference.com/2023/"><em>AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES 2023)</em></a> in Montr茅al.</p>&#13; &#13; <p>鈥楬uman-in-the-loop鈥 machine learning systems 鈥 a type of AI system that enables human feedback 鈥 are often framed as a promising way to reduce risks in settings where automated models cannot be relied upon to make decisions alone. But what if the humans are unsure?</p>&#13; &#13; <p>鈥淯ncertainty is central in how humans reason about the world but many AI models fail to take this into account,鈥 said first author Katherine Collins from Cambridge鈥檚 Department of Engineering. 鈥淎 lot of developers are working to address model uncertainty, but less work has been done on addressing uncertainty from the person鈥檚 point of view.鈥</p>&#13; &#13; <p>We are constantly making decisions based on the balance of probabilities, often without really thinking about it. Most of the time 鈥 for example, if we wave at someone who looks just like a friend but turns out to be a total stranger 鈥 there鈥檚 no harm if we get things wrong. However, in certain applications, uncertainty comes with real safety risks.</p>&#13; &#13; <p>鈥淢any human-AI systems assume that humans are always certain of their decisions, which isn鈥檛 how humans work 鈥 we all make mistakes,鈥 said Collins. 鈥淲e wanted to look at what happens when people express uncertainty, which is especially important in safety-critical settings, like a clinician working with a medical AI system.鈥</p>&#13; &#13; <p>鈥淲e need better tools to recalibrate these models, so that the people working with them are empowered to say when they鈥檙e uncertain,鈥 said co-author Matthew Barker, who recently completed his MEng degree at Gonville聽&amp; Caius College, Cambridge. 鈥淎lthough machines can be trained with complete confidence, humans often can鈥檛 provide this, and machine learning models struggle with that uncertainty.鈥</p>&#13; &#13; <p>For their study, the researchers used some of the benchmark machine learning datasets: one was for digit classification, another for classifying chest X-rays, and one for classifying images of birds. For the first two datasets, the researchers simulated uncertainty, but for the bird dataset, they had human participants indicate how certain they were of the images they were looking at: whether a bird was red or orange, for example. These annotated 鈥榮oft labels鈥 provided by the human participants allowed the researchers to determine how the final output was changed. However, they found that performance degraded rapidly when machines were replaced with humans.</p>&#13; &#13; <p>鈥淲e know from decades of behavioural research that humans are almost never 100% certain, but it鈥檚 a challenge to incorporate this into machine learning,鈥 said Barker. 鈥淲e鈥檙e trying to bridge the two fields so that machine learning can start to deal with human uncertainty where humans are part of the system.鈥</p>&#13; &#13; <p> 探花直播researchers say their results have identified several open challenges when incorporating humans into machine learning models. They are releasing their datasets so that further research can be carried out and uncertainty might be built into machine learning systems. 聽</p>&#13; &#13; <p>鈥淎s some of our colleagues so brilliantly put it, uncertainty is a form of transparency, and that鈥檚 hugely important,鈥 said Collins. 鈥淲e need to figure out when we can trust a model and when to trust a human and why. In certain applications, we鈥檙e looking at probability over possibilities. Especially with the rise of chatbots, for example, we need models that better incorporate the language of possibility, which may lead to a more natural, safe experience.鈥</p>&#13; &#13; <p>鈥淚n some ways, this work raised more questions than it answered,鈥 said Barker. 鈥淏ut even though humans may be miscalibrated in their uncertainty, we can improve the trustworthiness and reliability of these human-in-the-loop systems by accounting for human behaviour.鈥</p>&#13; &#13; <p> 探花直播research was supported in part by the Cambridge Trust, the Marshall Commission, the Leverhulme Trust, the Gates Cambridge Trust and the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).</p>&#13; &#13; <p>聽</p>&#13; &#13; <p><em><strong>Reference:</strong><br />&#13; Katherine M Collins et al. 鈥楬uman Uncertainty in Concept-Based AI Systems.鈥 Paper presented at the <a href="https://www.aies-conference.com/2023/">Sixth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES 2023)</a>, August 8-10, 2023. Montr茅al, QC, Canada.</em></p>&#13; </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Researchers are developing a way to incorporate one of the most human of characteristics 鈥 uncertainty 鈥 into machine learning systems.</p>&#13; </p></div></div></div><div class="field field-name-field-content-quote field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even">Uncertainty is central in how humans reason about the world but many AI models fail to take this into account</div></div></div><div class="field field-name-field-content-quote-name field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Katherine Collins</div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="https://www.gettyimages.co.uk/detail/photo/doctor-hospital-and-futuristic-brain-mri-in-cancer-royalty-free-image/1477483014?phrase=doctor working with ai&amp;amp;adppopup=true" target="_blank">PeopleImages via Getty Images</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Futuristic image of a doctor looking at brain scans</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" rel="license"><img alt="Creative Commons License." src="/sites/www.cam.ac.uk/files/inner-images/cc-by-nc-sa-4-license.png" style="border-width: 0px; width: 88px; height: 31px;" /></a><br />&#13; 探花直播text in this work is licensed under a <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified.聽 All rights reserved. We make our image and video content available in a number of ways 鈥 as here, on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p>&#13; </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Wed, 09 Aug 2023 23:40:58 +0000 sc604 241171 at Machine learning algorithm predicts how to get the most out of electric vehicle batteries /research/news/machine-learning-algorithm-predicts-how-to-get-the-most-out-of-electric-vehicle-batteries <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/car-charging.jpg?itok=BFjKv9sq" alt="People charging their electric cars at charging station" title="People charging their electric cars at charging station in York, Credit: Monty Rakusen via Getty Images" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p> 探花直播researchers, from the 探花直播 of Cambridge, say their algorithm could help drivers, manufacturers and businesses get the most out of the batteries that power electric vehicles by suggesting routes and driving patterns that minimise battery degradation and charging times.</p> <p> 探花直播team developed a non-invasive way to probe batteries and get a holistic view of battery health. These results were then fed into a machine learning algorithm that can predict how different driving patterns will affect the future health of the battery.</p> <p>If developed commercially, the algorithm could be used to recommend routes that get drivers from point to point in the shortest time without degrading the battery, for example, or recommend the fastest way to charge the battery without causing it to degrade. 探花直播<a href="https://www.nature.com/articles/s41467-022-32422-w">results</a> are reported in the journal <em>Nature Communications</em>.</p> <p> 探花直播health of a battery, whether it鈥檚 in a smartphone or a car, is far more complex than a single number on a screen. 鈥淏attery health, like human health, is a multi-dimensional thing, and it can degrade in lots of different ways,鈥 said first author Penelope Jones, from Cambridge鈥檚 Cavendish Laboratory. 鈥淢ost methods of monitoring battery health assume that a battery is always used in the same way. But that鈥檚 not how we use batteries in real life. If I鈥檓 streaming a TV show on my phone, it鈥檚 going to run down the battery a whole lot faster than if I鈥檓 using it for messaging. It鈥檚 the same with electric cars 鈥 how you drive will affect how the battery degrades.鈥</p> <p>鈥淢ost of us will replace our phones well before the battery degrades to the point that it鈥檚 unusable, but for cars, the batteries need to last for five, ten years or more,鈥 said <a href="https://www.alpha-lee.com/">Dr Alpha Lee</a>, who led the research. 鈥淏attery capacity can change drastically over that time, so we wanted to come up with a better way of checking battery health.鈥</p> <p> 探花直播researchers developed a non-invasive probe that sends high-dimensional electrical pulses into a battery and measures the response, providing a series of 鈥榖iomarkers鈥 of battery health. This method is gentle on the battery and doesn鈥檛 cause it to degrade any further.</p> <p> 探花直播electrical signals from the battery were converted into a description of the battery鈥檚 state, which was fed into a machine learning algorithm. 探花直播algorithm was able to predict how the battery would respond in the next charge-discharge cycle, depending on how quickly the battery was charged and how fast the car would be going the next time it was on the road. Tests with 88 commercial batteries showed that the algorithm did not require any information about previous usage of the battery to make an accurate prediction.</p> <p> 探花直播experiment focused on lithium cobalt oxide (LCO) cells, which are widely used in rechargeable batteries, but the method is generalisable across the different types of battery chemistries used in electric vehicles today.</p> <p>鈥淭his method could unlock value in so many parts of the supply chain, whether you鈥檙e a manufacturer, an end user, or a recycler, because it allows us to capture the health of the battery beyond a single number, and because it鈥檚 predictive,鈥 said Lee. 鈥淚t could reduce the time it takes to develop new types of batteries, because we鈥檒l be able to predict how they will degrade under different operating conditions.鈥</p> <p> 探花直播researchers say that in addition to manufacturers and drivers, their method could be useful for businesses that operate large fleets of electric vehicles, such as logistics companies. 鈥 探花直播framework we鈥檝e developed could help companies optimise how they use their vehicles to improve the overall battery life of the fleet,鈥 said Lee. 鈥淭here鈥檚 so much potential with a framework like this.鈥</p> <p>鈥淚t鈥檚 been such an exciting framework to build because it could solve so many of the challenges in the battery field today,鈥 said Jones. 鈥淚t鈥檚 a great time to be involved in the field of battery research, which is so important in helping address climate change by transitioning away from fossil fuels.鈥</p> <p> 探花直播researchers are now working with battery manufacturers to accelerate the development of safer, longer-lasting next-generation batteries. They are also exploring how their framework could be used to develop optimal fast charging protocols to reduce electric vehicle charging times without causing degradation.</p> <p> 探花直播research was supported by the Winton Programme for the Physics of Sustainability, the Ernest Oppenheimer Fund, 探花直播Alan Turing Institute and the Royal Society.</p> <p><br /> <em><strong>Reference:</strong><br /> Penelope K Jones, Ulrich Stimming &amp; Alpha A Lee. 鈥<a href="https://www.nature.com/articles/s41467-022-32422-w">Impedance-based forecasting of lithium-ion battery performance amid uneven usage</a>.鈥 Nature Communications (2022). DOI: 10.1038/s41467-022-32422-w</em></p> <p><em><strong>For more information on聽energy-related research in Cambridge, please visit聽<a href="https://www.energy.cam.ac.uk/">Energy聽IRC</a>, which brings together Cambridge鈥檚 research knowledge and expertise, in collaboration with global partners, to create solutions for a sustainable and resilient energy landscape for generations to come.聽</strong></em></p> </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Researchers have developed a machine learning algorithm that could help reduce charging times and prolong battery life in electric vehicles by predicting how different driving patterns affect battery performance, improving safety and reliability.</p> </p></div></div></div><div class="field field-name-field-content-quote field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even">This method could unlock value in so many parts of the supply chain, whether you鈥檙e a manufacturer, an end user, or a recycler, because it allows us to capture the health of the battery beyond a single number</div></div></div><div class="field field-name-field-content-quote-name field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Alpha Lee</div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="https://www.gettyimages.co.uk/detail/photo/york-people-charging-their-electric-cars-at-royalty-free-image/1351964126?adppopup=true" target="_blank">Monty Rakusen via Getty Images</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">People charging their electric cars at charging station in York</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="http://creativecommons.org/licenses/by/4.0/" rel="license"><img alt="Creative Commons License" src="https://i.creativecommons.org/l/by/4.0/88x31.png" style="border-width:0" /></a><br /> 探花直播text in this work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified.聽 All rights reserved. We make our image and video content available in a number of ways 鈥 as here, on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p> </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Tue, 23 Aug 2022 09:01:34 +0000 sc604 233851 at 鈥楾ransformational鈥 approach to machine learning could accelerate search for new disease treatments /research/news/transformational-approach-to-machine-learning-could-accelerate-search-for-new-disease-treatments <div class="field field-name-field-news-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even"><img class="cam-scale-with-grid" src="/sites/default/files/styles/content-580x288/public/news/research/news/abstract.jpg?itok=k804QNlD" alt="Woman in grey shirt" title="Woman in grey shirt, Credit: mahdis mousavi via Unsplash" /></div></div></div><div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p> 探花直播method, called transformational machine learning (TML), was developed by a team from the UK, Sweden, India and Netherlands. It learns from multiple problems and improves performance while it learns.</p>&#13; &#13; <p>TML could accelerate the identification and production of new drugs by improving the machine learning systems which are used to identify them. 探花直播<a href="https://www.pnas.org/doi/10.1073/pnas.2108013118">results</a> are reported in the <em>Proceedings of the National Academy of Sciences</em>.</p>&#13; &#13; <p>Most types of machine learning (ML) use labelled examples, and these examples are almost always represented in the computer using intrinsic features, such as the colour or shape of an object. 探花直播computer then forms general rules that relate the features to the labels.</p>&#13; &#13; <p>鈥淚t鈥檚 sort of like teaching a child to identify different animals: this is a rabbit, this is a donkey and so on,鈥 said Professor Ross King from Cambridge鈥檚 Department of Chemical Engineering and Biotechnology, who led the research. 鈥淚f you teach a machine learning algorithm what a rabbit looks like, it will be able to tell whether an animal is or isn鈥檛 a rabbit. This is the way that most machine learning works 鈥 it deals with problems one at a time.鈥</p>&#13; &#13; <p>However, this is not the way that human learning works: instead of dealing with a single issue at a time, we get better at learning because we have learned things in the past.</p>&#13; &#13; <p>鈥淭o develop TML, we applied this approach to machine learning, and developed a system that learns information from previous problems it has encountered in order to better learn new problems,鈥 said King, who is also a Fellow at 探花直播Alan Turing Institute. 鈥淲here a typical ML system has to start from scratch when learning to identify a new type of animal - say a kitten - TML can use the similarity to existing animals: kittens are cute like rabbits, but don鈥檛 have long ears like rabbits and donkeys. This makes TML a much more powerful approach to machine learning.鈥</p>&#13; &#13; <p> 探花直播researchers demonstrated the effectiveness of their idea on thousands of problems from across science and engineering. They say it shows particular promise in the area of drug discovery, where this approach speeds up the process by checking what other ML models say about a particular molecule. A typical ML approach will search for drug molecules of a particular shape, for example. TML instead uses the connection of the drugs to other drug discovery problems.</p>&#13; &#13; <p>鈥淚 was surprised how well it works 鈥 better than anything else we know for drug design,鈥 said King. 鈥淚t鈥檚 better at choosing drugs than humans are 鈥 and without the best science, we won鈥檛 get the best results.鈥</p>&#13; &#13; <p><em><strong>Reference:</strong><br />&#13; Ivan Olier et al. 鈥<a href="https://www.pnas.org/doi/10.1073/pnas.2108013118">Transformational Machine Learning: Learning How to Learn from Many Related Scientific Problems</a>.鈥 Proceedings of the National Academy of Sciences (2021). DOI: 10.1073/pnas.2108013118</em></p>&#13; </div></div></div><div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Researchers have developed a new approach to machine learning that 鈥榣earns how to learn鈥 and out-performs current machine learning methods for drug design, which in turn could accelerate the search for new disease treatments.</p>&#13; </p></div></div></div><div class="field field-name-field-content-quote field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even">I was surprised how well it works 鈥 better than anything else we know for drug design</div></div></div><div class="field field-name-field-content-quote-name field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Ross King</div></div></div><div class="field field-name-field-image-credit field-type-link-field field-label-hidden"><div class="field-items"><div class="field-item even"><a href="https://unsplash.com/photos/woman-wearing-grey-shirt-hJ5uMIRNg5k" target="_blank">mahdis mousavi via Unsplash</a></div></div></div><div class="field field-name-field-image-desctiprion field-type-text field-label-hidden"><div class="field-items"><div class="field-item even">Woman in grey shirt</div></div></div><div class="field field-name-field-cc-attribute-text field-type-text-long field-label-hidden"><div class="field-items"><div class="field-item even"><p><a href="http://creativecommons.org/licenses/by/4.0/" rel="license"><img alt="Creative Commons License" src="https://i.creativecommons.org/l/by/4.0/88x31.png" style="border-width:0" /></a><br />&#13; 探花直播text in this work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>. Images, including our videos, are Copyright 漏 探花直播 of Cambridge and licensors/contributors as identified.聽 All rights reserved. We make our image and video content available in a number of ways 鈥 as here, on our <a href="/">main website</a> under its <a href="/about-this-site/terms-and-conditions">Terms and conditions</a>, and on a <a href="/about-this-site/connect-with-us">range of channels including social media</a> that permit your use and sharing of our content under their respective Terms.</p>&#13; </div></div></div><div class="field field-name-field-show-cc-text field-type-list-boolean field-label-hidden"><div class="field-items"><div class="field-item even">Yes</div></div></div> Mon, 29 Nov 2021 20:00:00 +0000 sc604 228391 at AI could detect dementia years before symptoms appear /stories/AIdementia <div class="field field-name-field-content-summary field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even"><p><p>Artificial intelligence could spot the early signs of dementia from a simple brain scan long before major symptoms appear 鈥 and in some cases before any symptoms appear 鈥 say Cambridge researchers.</p> </p></div></div></div> Thu, 12 Aug 2021 08:44:50 +0000 cjb250 225921 at