Jovana Jankovic

Topic

Schwartz Reisman Institute for Technology and Society

The Institute for the History and Philosophy of Science and Technology

Department of Computer Science

Centre for Ethics

Department of Philosophy

Vector Institute

Victoria College

Subheadline

"We should be building AI systems that promote human flourishing – that allow human beings to live with dignity and purpose, and to be valued contributors to society”

What does safe artificial intelligence look like? Could AI go rogue and pose an existential threat to humanity? How have media portrayals of AI influenced people’s perceptions of the technology’s benefits and risks?

These were among the pressing questions tackled by four experts at the University of Toronto and its partner institutions – in disciplines ranging from computer science to philosophy – during a recent panel discussion on AI safety.

Sheila McIlraith, professor in ��˾��ֱ��’s department of computer science at the Faculty of Arts & Science and Canada CIFAR AI Chair at the Vector Institute, said the notion of AI safety evokes different things to different people.

“Computer scientists often think about safety critical systems – the types of systems that we’ve built to send astronauts to the moon or control our nuclear power plants – but AI safety is actually quite different,” said McIlraith, an associate director at the ��˾��ֱ��’s Schwartz Reisman Institute for Technology and Society (SRI).

“For me personally, I have a higher bar, and I really think we should be building AI systems that promote human flourishing – that allow human beings to live with dignity and purpose, and to be valued contributors to society.”

The event, hosted by SRI in partnership with the Vector Institute, the Institute for the History & Philosophy of Science & Technology, the Centre for Ethics and Victoria College, invited McIlraith and her fellow panelists to discuss how AI technologies can be aligned with human values in an increasingly automated world.

They also discussed how risks surrounding the technology can be mitigated in different sectors.

Karina Vold, the event’s moderator, underscored the challenge of building safe AI systems in an uncertain world (photo by Duane Cole)

Moderator, Karina Vold, assistant professor in the Institute for the History & Philosophy of Science & Technology in the Faculty of Arts & Science, noted that because AI systems operate “in a world filled with uncertainty and volatility, the challenge of building safe and reliable AI is not easy and mitigation strategies vary widely.”

She proceeded to ask the panel to share their thoughts on the portrayal of AI in popular culture.

“The media devotes more attention to different aspects of AI – the social, philosophical, maybe even psychological,” said Sedef Kocak, director of AI professional development at the Vector Institute.

“These narratives are important to help show the potential fears, as well as the positive potential of the technology.”

The discussion touched on several topics related to AI safety (photo by Duane Cole)

Roger Grosse, associate professor in ��˾��ֱ��’s department of computer science in the Faculty of Arts & Science and a founding member of the Vector Institute, said that safety concerns around AI are not merely rooted in science and pop culture, but also in philosophy.

“Many people think that the public’s concerns regarding AI risks come from sci-fi, but I think the early reasoning regarding AI risks actually has its roots in philosophy,” said Grosse, who also holds Schwartz Reisman Chair in Technology and Society.

“If we’re trying to reason about AI systems that don’t yet exist, we don’t have the empirical information, and don’t yet know what their design would be, what we can do is come up with various thought experiments. For example, what if we designed an AI that has some specific role, and all of the actions that it takes are in service of the role?

“For the last decade, a lot of the reasons for being concerned about the long-term existential risks really came from this careful philosophical reasoning.”

The discussion also touched on the dangers of AI models misaligning themselves, how to guard against bias in the training of large language models, and how to ensure that AI models with potentially catastrophic capabilities are safeguarded.

“This [safeguarding] is an area where new research ideas and principles will be required to make the case,” said Grosse. “Developers saying, ‘Trust us’ is not sufficient. It’s not a good foundation for policy.”

Despite addressing topics surrounding potential harms and risks of AI, the panelists also shared their optimism about how AI can be wielded for the greater good – with Grosse noting AI offers the promise of making knowledge more widely accessible, and Kocak focusing on the myriad benefits for industries.

Watch the Sept. 10 conversation below:

Four AI trends to watch in 2024

Christopher.Sorensen — Fri, 19 Jan 2024 17:02:40 +0000

Four AI trends to watch in 2024

Christopher.Sorensen Fri, 01/19/2024 - 12:02

Cutline

AI was a hot topic at this week’s annual meeting of the World Economic Forum in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket via Getty Images)

Daniel Browne

Topic

Schwartz Reisman Institute for Technology and Society

Munk School of Global Affairs & Public Policy

Faculty of Applied Science & Engineering

Faculty of Information

Rotman School of Management

Global

Graduate Students

Research & Innovation

��˾��ֱ�� Mississauga

Subheadline

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions”

As artificial intelligence continues to develop rapidly, the world is watching with excitement and apprehension – as evidenced by the AI buzz in Davos this week at the World Economic Forum’s annual meeting.

University of Toronto researchers are using AI to advance scientific discovery and improve health-care delivery, exploring how to mitigate potential harms and finding new ways to ensure the technology aligns with human values.

“The advancement of AI is moving quickly, and the year ahead holds a lot of promise but also a lot of unanswered questions,” says Monique Crichlow, executive director of the Schwartz Reisman Institute for Technology and Society (SRI). “Researchers at SRI and across the university are tackling how to build and regulate AI systems for safer outcomes, as well as the social impacts of these powerful technologies.”

“From health-care delivery to accessible financial and legal services, AI has the potential to benefit society in many ways and tackle inequality around the world. But we have real work to do in 2024 to ensure that happens safely.”

As AI continues to reshape industries and challenge many aspects of society, here are four emerging themes ��˾��ֱ�� researchers are keeping their eyes on in 2024:

1. AI regulation is on its way

U.S. Vice President Kamala Harris applauds as U.S. President Joe Biden signs an executive order on the safe, secure, and trustworthy development and use of artificial intelligence on Oct. 30, 2023 (photo by Brendan Simialowski/AFP/Getty Images)

As a technology with a wide range of potential applications, AI has the potential to impact all aspects of society – and regulators around the world are scrambling to catch up.

Set to pass later this year, the Artificial Intelligence and Data Act (AIDA) is the Canadian government’s first attempt to comprehensively regulate AI. Similar attempts by other governments include the European Union’s AI Act and the Algorithmic Accountability Act in the United States.

But there is still much to be done.

In the coming year, legislators and policymakers in Canada will tackle many questions, including what counts as fair use when it comes to training data and what privacy means in the 21st century. Is it illegal for companies to train AI systems on copyrighted data, as a recent lawsuit from the New York Times alleges? Who owns the rights to AI-generated artworks? Will Canada’s new privacy bill sufficiently protect citizens’ biometric data?

On top of this, AI’s entry into other sectors and industries will increasingly affect and transform how we regulate other products and services. As Gillian Hadfield, a professor in the Faculty of Law and the Schwartz Reisman Chair in Technology and Society, Policy Researcher Jamie Sandhu and Faculty of Law doctorial candidate Noam Kolt explore in a recent policy brief for CIFAR (formerly the Canadian Institute for Advanced Research), a focus on regulating AI through its harms and risks alone “obscures the bigger picture” of how these systems will transform other industries and society as a whole. For example: are current car safety regulations adequate to account for self-driving vehicles powered by AI?

2. The use of generative AI will continue to stir up controversy

Microsoft Bing Image Creator is displayed on a smartphone (photo by Jonathan Raa/NurPhoto/Getty Images)

From AI-generated text and pictures to videos and music, use of generative AI has exploded over the past year – and so have questions surrounding issues such as academic integrity, misinformation and the displacement of creative workers.

In the classroom, teachers are seeking to understand how education is evolving in the age of machine learning. Instructors will need to find new ways to embrace these tools – or perhaps opt to reject them altogether – and students will continue to discover new ways to learn alongside these systems.

At the same time, AI systems created more than 15 billion images last year by some counts – more than the entire 150-year history of photography. Online content will increasingly lack human authorship, and some researchers have proposed that by 2026 as much as 90 per cent of internet text could be generated by AI. Risks around disinformation will increase, and new methods to label content as trustworthy will be essential.

Many workers – including writers, translators, illustrators and designers – are worried about job losses. But a tidal wave of machine-generated text could also have negative impacts on AI development. In a recent study, Nicolas Papernot, an assistant professor in the Edward S. Rogers Sr. department of electrical and computer engineering in Faculty of Applied Science & Engineering and an SRI faculty affiliate, and his co-authors found training AI on machine-generated text led to the system becoming less reliable and subject to “model collapse.”

3. Public perception and trust of AI is shifting

A person walks past a temporary AI stall in Davos, Switzerland (photo by Andy Barton/SOPA Images/LightRocket/Getty Images)

Can we trust AI? Is our data secure?

Emerging research on public trust of AI is shedding light on changing preferences, desires and viewpoints. Peter Loewen – the director of the Munk School of Global Affairs & Public Policy, SRI’s associate director and the director of the Munk School’s Policy, Elections & Representation Lab (PEARL) – is developing an index measuring public perceptions of and attitudes towards AI technologies.

Loewen’s team conducted a representative survey of more than 23,000 people across 21 countries, examining attitudes towards regulation, AI development, perceived personal and societal economic impacts, specific emerging technologies such as ChatGPT and the use of AI by government. They plan to release their results soon.

Meanwhile, 2024 is being called “the biggest election year in history,” with more than 50 countries headed to the polls, and experts expect interference and misinformation to hit an all-time high thanks to AI. How will citizens know which information, candidates, and policies to trust?

In response, some researchers are investigating the foundations of trust itself. Beth Coleman, an associate professor at ��˾��ֱ�� Mississauga’s Institute of Communication, Culture, Information and Technology and the Faculty of Information who is an SRI research lead, is leading an interdisciplinary working group on the role of trust in interactions between humans and AI systems, examining how trust is conceptualized, earned and maintained in our interactions with the pivotal technology of our time.

4. AI will increasingly transform labour, markets and industries

A protester in London holds a placard during a rally in Leicester Square (photo by Vuk Valcic/SOPA Images/LightRocket via Getty Images)

Kristina McElheran, an assistant professor in the Rotman School of Management and an SRI researcher, and her collaborators may have recently found a gap between AI buzz in the workplace and businesses who are actually using it – but there remains a real possibility that labour, markets and industries will undergo massive transformation.

��˾��ֱ�� researchers who have published books on how AI will transform industry include: Rotman faculty members Ajay Agrawal, Joshua Gans and Avi Goldfarb, whose Power and Prediction: The Disruptive Economics of Artificial Intelligence argues that “old ways of doing things will be upended” as AI predictions improve; and the Faculty of Law’s Benjamin Alarie and Abdi Aidid, who propose in The Legal Singularity: How Artificial Intelligence Can Make Law Radically Better that AI will improve legal services by increasing ease of access and fairness for individuals.

In 2024, institutions – public and private – will be creating more guidelines and rules around how AI systems can or cannot be used in their operations. Disruptors will be challenging the hierarchy of the current marketplace.

The coming year promises to be transformative for AI as it continues to find new applications across society. Experts and citizens must stay alert to the changes AI will bring and continue to advocate that ethical and responsible practices guide the development of this powerful technology.

Research shows decision-making AI could be made more accurate when judging humans

siddiq22 — Tue, 23 May 2023 22:08:43 +0000

Research shows decision-making AI could be made more accurate when judging humans

siddiq22 Tue, 05/23/2023 - 18:08

Cutline

(photo by wildpixel/iStock)

Schwartz Reisman Institute for Technology and Society

Alexander Bernier

Topic

Breaking Research

Equity

machine learning

Research & Innovation

Subheadline

New study by researchers from ��˾��ֱ�� and MIT suggests that clearly labelling data might help reduce bias

A new study from researchers at the University of Toronto and the Massachusetts Institute of Technology (MIT) is challenging conventional wisdom on human-computer interaction and reducing bias in AI.

The paper, which was published this month in the journal Science Advances, demonstrates empirical evidence on the relationship between the methods used to label the data that trains machine learning (ML) models and the performance of those models when applying norms.

MIT PhD student Aparna Balagopalan, a graduate of ��˾��ֱ��'s masters program in applied computing, is lead author, with co-authors Gillian Hadfield, director of ��˾��ֱ��’s Schwartz Reisman Institute for Technology (SRI), Schwartz Reisman Chair in Technology and Society, CIFAR AI Chair, and a professor of law and strategic management in the Faculty of Law; David Madras, a PhD student in the Machine Learning Group at the department of computer science in the Faculty of Arts & Science and the Vector Institute; research assistant David H. Yang, a graduate student in the applied computing program in the Faculty of Arts & Science; Marzyeh Ghassemi, a faculty affiliate at SRI and an assistant professor at MIT; and Dylan Hadfield-Menell, an assistant professor at MIT.

Much of the scholarship in this area presumes that calibrating AI behaviour to human conventions requires value-neutral, observational data from which AI can best reason toward sound normative conclusions. But the new research suggests that labels explicitly reflecting value judgments, rather than the facts used to reach those judgments, might yield ML models that assess rule adherence and rule violation in a manner that humans would deem acceptable.

To reach this conclusion, the authors conducted experiments to see how individuals behaved when asked to provide factual assessments as opposed to when asked to judge whether a rule had been followed.

From left to right: MIT PhD student Aparna Balagopalan, SRI Director Gillian Hadfield and SRI Faculty Affiliate Marzyeh Ghassemi (supplied photos)

For example, one group of participants was asked to label dogs that exhibited certain characteristics – namely, those that were large, not well groomed, or aggressive. Meanwhile, another group of participants was instead asked whether or not the dogs shown to them violate a building pet code predicated on the same characteristics, rather than assessing the presence or absence of specific features.

The first group was asked to make a factual assessment – and the second, a normative one.

Hadfield says the researchers were surprised by the findings.

“When you ask people a normative question, they answer it differently than when you ask them a factual question,” she says.

Human participants in the experiments were more likely to recognize (and label) a factual feature than the violation of an explicit rule predicated on the factual feature.

Current thinking on this topic presumes that calibrating AI behaviour to human conventions requires value-neutral, observational data from which AI can best reason toward sound normative conclusions.

But this new research suggests that labelling data with labels that explicitly reflect value judgments, rather than the facts used to reach those judgments, might yield ML models that assess rule-adherence and rule-violation in a manner that we humans would deem acceptable.

The results of these experiments showed that ML models trained on normative labels achieve higher accuracy in predicting human normative judgments. Essentially, they are better at predicting. Therefore, if we train automated judgment systems on factual labels – which is how several existing systems are being built – they are likely overpredicting rule violations.

The implications of the research are significant. Not only does it show that reasoning about norms is qualitatively different from reasoning about facts, but it also has important real-world ramifications.

“People could say, ‘I don’t want to be judged by a machine – I want to be judged by a human,’ given that we’ve got evidence to show that the machine will not judge them properly,” Hadfield says.

“Our research shows that this factor has a bigger effect [on an ML model’s performance] than things like model architecture, label noise and subsampling – factors that are often looked to for errors in prediction.”

Ensuring that the data used to train decision-making ML algorithms mirrors the results of human judgment – rather than simple factual observation – is no small feat. Proposed approaches include ensuring that the training data used to reproduce human assessments is collected in an appropriate context.

To this end, the authors of the paper recommend that the creators of trained models and of datasets supplement those products with clear descriptions of the approaches used to tag the data – taking special care to establish whether the tags relate to facts perceived or judgments applied.

“We need to train on and evaluate normative labels. We have to pay the money for normative labels, and probably for specific applications. We should be a lot better at documenting that labelling practice. Otherwise, it’s not a fair judgment system,” Hadfield says.

“There’s a ton more research we need to be doing on this.”

The study was funded by the Schwartz Reisman Institute for Technology and Society and the Vector Institute, among others.

Pilot program embeds ethics into ��˾��ֱ�� undergraduate technology courses

Christopher.Sorensen — Mon, 05 Jul 2021 17:18:27 +0000

Pilot program embeds ethics into ��˾��ֱ�� undergraduate technology courses

Christopher.Sorensen Mon, 07/05/2021 - 13:18

Cutline

Diane Horton and Sheila McIlraith are co-leading a pilot program to embed ethics into a cross section of undergraduate computer science courses at the university (photos by Ken Jones and Johnny Guatto)

Topic

Schwartz Reisman Institute for Technology and Society

Computer Science

Ethics

Philosophy

A new pilot program at the University of Toronto will embed ethics modules into existing undergraduate computer science courses in a bid to ensure future technologies designed and deployed in ways that consider their broader societal impact.

From learning about the complex trade-off between data privacy and public benefit to making design decisions that impact marginalized communities, the pilot program – led by the department of computer science, in the Faculty of Arts & Science, and the Schwartz Reisman Institute for Technology and Society (SRI) – will teach computer science students skills to identify potential ethical risks in the technologies they are learning to build.

The initiative aims to equip ��˾��ֱ�� graduates, who may go on to become global tech leaders, to make informed decisions about technology and its wide-ranging effects on justice, health care, education, economies, human rights and beyond.

“We want to teach students how to think, not what to think,” says Sheila McIlraith, a professor of computer science and a research lead at SRI who is co-leading the initiative, which includes scholars who specialize in ethics from ��˾��ֱ��’s department of philosophy.

“We’re not proselytizing about ‘right’ or ‘wrong,’ But we want students to identify ethical questions because, when they enter the workforce, they will be on the front lines. They’ll be the ones writing the code, developing the systems, using the data. It’s imperative that ethical considerations are part of fundamental design principles.”

McIlraith points to the rapidly changing role technology plays in society as evidence of the urgent need for such a program.

“It used to be that technologists would build systems for a particular purpose or industry,” she says. “But now technology is no longer just for individual tasks like completing tax returns or keeping track of company inventory. Technology impacts the way all of us live, work and interact with each other. A lot of the money and investment that fuels our economy is related to technology. And emerging tech companies are often led by young people who have just come out of computer science degrees.”

When SRI was founded in 2019, McIlraith was appointed as one of its inaugural research leads. She quickly approached SRI Director Gillian K. Hadfield about the need for an embedded ethics initiative in computer science, citing a similar pioneering program already underway at Harvard University. Hadfield immediately saw the alignment with SRI’s mission to explore the dynamics between technology and the human agenda – and to solve problems at the intersection of technology and public good.

McIlraith and Horton are joined on the team by Benjamin Wald, most recently a post-doctoral researcher at SRI and an alumnus of ��˾��ֱ��’s department of philosophy; Maryam Majedi, a post-doctoral researcher in the department of computer science; and Emma McClure, a PhD candidate in the department of philosophy.

“Embedding ethical considerations into existing courses helps students see their relevance at the very moment they’re learning the computer science,” says Diane Horton, a professor, teaching stream, in the department who is co-leading the pilot program with McIlraith. “The ethics modules are associated very closely with the technical content, so when students are eventually in the workplace, we hope the two will remain very connected in their minds.”

Horton, who has been teaching in the department for 25 years, has seen first-hand how eager students are to talk about ethics. She also noted that they bring different perspectives to the conversation.

“One student had a very intense appreciation for the vulnerability of the homeless population,” says Horton, “and she brought that from her personal experience. Another student talked about the hospital where he works, and how private medical data is so carefully protected.”

“There has been so much curiosity from the students,” adds Majedi of the initiative so far. “They ask a lot of questions and offer interesting and creative ideas. Some get so excited, and they stay long after class to talk with us.”

Majedi says her own research into data privacy has highlighted a gap in curricula where ethical training for students is badly needed.

“It's critical to teach ethics in computer science,” she says, “because these students will be responsible for many important tasks in the future.”

Both Wald and McClure say they are excited to see the enthusiasm among computer science students when it comes to addressing ethical questions.

“I think the students really want to have these critical thinking tools, because it’s clear they’ve been considering these issues already,” says McClure.

“Sometimes, a computer science student might recognize a potential ethical issue,” says Wald, “but might not know how it’s been discussed by other people, or where to find the right resources to address it. They might think, ‘How do I put the concern I have into words?’ Hopefully we can give them the tools to do that.”

The embedded ethics initiative will produce a longitudinal study to inform its future directions. The goal is for every computer science student to encounter ethics modules at several points in their ��˾��ֱ�� computer science program – and bring those insights to their future careers.

“Big tech companies like Apple often employ people in specialized ethics roles, but our program aims to equip people who are actually building the technologies at a company like that,” says McClure. “That way, the ethical behaviour comes from within the design of technologies. It comes from the bottom instead of being imposed from the outside by an ‘ethics specialist.’”

McIlraith and Horton both credit Harvard’s Barbara Grosz and Jeff Behrends for supporting the ��˾��ֱ�� team at the early stages of the pilot program’s conception and development. Grosz is a founder of Harvard’s Embedded EthiCS program, while Behrends is a faculty team leader.

The ��˾��ֱ�� team aims to engage other faculty, instructors and researchers as it grows – in particular, computer science faculty who have already been teaching undergraduate courses in the core curriculum for years.

“Longer-term, we aspire to have ethical considerations as a cornerstone of many of our tech-oriented disciplines within the university,” says McIlraith. “One of our goals is to create a winning strategy so that this pilot can transform into something broader.”

Algorithms and art: Researchers explore impact of AI on music and culture

Christopher.Sorensen — Fri, 11 Jun 2021 17:05:38 +0000

Algorithms and art: Researchers explore impact of AI on music and culture

Christopher.Sorensen Fri, 06/11/2021 - 13:05

Cutline

(Photo by Krisanapong Detraphiphat via Getty Images)

Topic

Schwartz Reisman Institute for Technology and Society

Computer Science

Music

Global access to art, culture, and entertainment products – music, movies, books, and more – has undergone fundamental changes over the past 20 years in light of groundbreaking developments in artificial intelligence.

For example, users of streaming services like Netflix and Spotify have data collected and analyzed by algorithms to determine their streaming habits – resulting in recommendations that cater to their tastes. But this is only one of the many ways in which AI tools are transforming the arts and culture industries. AI is also being used in the production of music and other art, with algorithms generating photos or writing songs on their own.

Warner Music even “signed” an algorithm to a record deal in 2019.

Yet, while AI is drastically reshaping cultural industries around the world, we have yet to fully understand the consequences.

Ashton Anderson

“The societal impacts these algorithmic developments are having on the production, circulation, and consumption of culture remain largely unknown,” says Ashton Anderson, an assistant professor in the department computer science in the University of Toronto’s Faculty of Arts & Science and a faculty affiliate at the Schwartz Reisman Institute for Technology and Society.

Anderson’s research aims to bridge the divide between computer science and the social sciences. He uses computation to study online well-being – for example, studying the impact of “echo chambers” on social media. Such interdisciplinary work is a key component of the Schwartz Reisman Institute’s mission to re-conceptualize common notions of the ways technology, systems and society interact.

In October of 2019, Anderson and his collaborators – Georgina Born, a professor of music and anthropology at Oxford University; Jeremy Morris, an associate professor of media and cultural studies at the University of Wisconsin-Madison; and Fernando Diaz, a research scientist at Google in Montreal and a Canada CIFAR AI chair – convened a CIFAR AI & Society workshop to explore the effects of AI on the curation of culture, with a particular focus on the music industry.

“We deliberately brought together communications and computer science scholars, musicians, industry members and users to map out the major issues we could foresee now that cultural products are largely being distributed via algorithms on global platforms,” says Anderson.

The researchers’ findings and recommendations were published in a report titled “Artificial Intelligence, Music Recommendation and the Curation of Culture.”

“This report is the first major document to recognize and describe the societal effects of the algorithmic revolution in cultural industries,” says Anderson. “Existing journals are virtually entirely aligned with only one of the many stakeholder groups that took part in this interdisciplinary effort so, unfortunately, publishing this report in one of these journals would be next to impossible.

“We’re very happy to have Schwartz Reisman publish this report, as our methodology and the cross-disciplinary expertise we convened is well-aligned with SRI’s mission to straddle traditional academic boundaries in the pursuit of understanding how powerful new technologies shape the world around us.”

The report contains three overarching themes. First, participants agreed there will be major long-term impacts of the use of AI-driven technologies on cultural consumption and creation. For example, if algorithms decide what to distribute and recommend – and to whom and when – then arts and culture creators, and the organizations who fund them, may be incentivized to produce content that is more likely to get listener exposure or reach fans based on how algorithms work to connect audiences with content.

Participants also saw “a clear need to enrich existing AI-driven technologies so they can better serve diverse communities and genres of culture, art, and music,” says Anderson. If algorithms overly generalize information about certain groups, subcultures or communities – whether by race, gender, or other identity markers – Anderson notes that “we risk reinforcing rigid and potentially harmful social boundaries.”

A third theme emerging from the workshop is that the curation of culture always has involved, and always will involve, balancing competing objectives.

“The extraction of personal data has been privatized and corporatized by curation platforms, but as yet without any public debate or intervention for accountability and transparency,” says Anderson.

In other words, how should we measure the convenience and increased accessibility that streaming platforms provide against the fact that they threaten to harm important ideals of public safety, such as the right to privacy and cultural sovereignty?

The workshop’s report considers almost every step of the music industry’s processes, from the ways in which algorithms manipulate the existing variety of content itself, to the ways in which content is produced, distributed, valued, understood and consumed around the world – including the ways in which artists and creators are remunerated.

Important questions explored in the report include:

What assumptions are built into media recommendation systems?
What happens to the crucial role of social and community relationships at the heart of the experience of music?
How can we ensure appropriate cultural expertise is represented in algorithmic and technological design?
AI-based classifications of music may be highly efficient, but they do not necessarily reflect a truly intelligent analysis of music. Can they ever have any real understanding of what music is?
Will music and musical tastes become increasingly homogenized due to AI-based systems of production, promotion and distribution? Do we risk misrepresenting or underrepresenting marginalized communities or their agency to represent themselves?
Is the personalization of algorithms too seductive? Do we risk no longer “thinking for ourselves”?

“I’m delighted to see us producing this report at SRI,” says Professor Gillian K. Hadfield, the director of the Schwartz Reisman Institute for Technology and Society. “This project is aligned with the work that SRI Engineering Lead Ron Bodkin is leading to improve AI systems’ objectives and recommender systems, surveying research in this area and building new techniques. Recommendation is a huge part of the daily role AI plays in our lives, and ensuring it’s aligned with human values is a key part of SRI’s mission.”

“I'm glad to see SRI contributing a thoughtful interdisciplinary perspective to the considerations of how AI is affecting media and culture,” adds Bodkin. “This report raises important topics for how various stakeholders should be able to participate and how to allow for more autonomy and diversity.

“I believe that incorporating a wider range of values and increasing agency is a critical direction for recommendation systems and algorithmically curated media,” says Bodkin. “The report's call for giving stakeholders meaningful controls over recommendation systems is important and it's an area where we're exploring how AI research can contribute.”

��˾��ֱ��'s Schwartz Reisman Institute and AI Global to develop global certification mark for trustworthy AI

Christopher.Sorensen — Tue, 01 Dec 2020 22:14:13 +0000

��˾��ֱ��'s Schwartz Reisman Institute and AI Global to develop global certification mark for trustworthy AI

Christopher.Sorensen Tue, 12/01/2020 - 17:14

Cutline

The partnership between ��˾��ֱ��'s Schwartz Reisman Institute for Technology and Society and the non-profit organization AI Global aims to build trust in responsible, ethical and fair AI systems (photo by Andriy Onufriyenko/Getty Images)

Schwartz Reisman Institute for Technology and Society

Topic

Global Lens

Rotman School of Management

Global

The products and services we use in our daily lives have to abide by safety and security standards, from car airbags to construction materials. But no such broad, internationally agreed-upon standards exist for artificial intelligence.

And yet, AI tools and technologies are steadily being integrated into all aspects of our lives. AI’s potential benefits to humanity, such as improving health-care delivery or tackling climate change, are immense. But potential harms caused by AI tools –from algorithmic bias and labour displacement to risks associated with autonomous vehicles and weapons – risk leading to a lack of trust in AI technologies.

To tackle these problems, a new partnership between AI Global, a nonprofit organization focused on advancing responsible and ethical adoption of artificial intelligence, and the Schwartz Reisman Institute for Technology and Society (SRI) at the University of Toronto will create a globally recognized certification mark for the responsible and trusted use of AI systems.

In collaboration with the World Economic Forum’s Shaping the Future of Technology Governance: Artificial Intelligence and Machine Learning platform, the partnership will convene industry actors, policy-makers, civil society representatives and academics to build a universally recognized framework that validates AI tools and technologies as responsible, trustworthy, ethical and fair.

“In addition to our fundamental multidisciplinary research, SRI also aims to craft practical, implementable and globally appealing solutions to the challenge of building responsible and inclusive AI,” says Gillian Hadfield, the director of the Schwartz Reisman Institute for Technology and Society and a professor at ��˾��ֱ��’s Faculty of Law and Rotman School of Management.

Hadfield’s current research is focused on innovative design for legal and regulatory systems for AI and other complex global technologies. She also works on “the alignment problem”: a term that refers to the ideal that an AI’s actions should align with what humans would want.

“One of the reasons why we’re excited to partner with AI Global is that they’re focused on building tangible, usable tools to support the responsible development of AI,” says Hadfield. “And we firmly believe that’s what the world currently needs. The need for clear, objective regulations has never been more urgent.”

A wide variety of initiatives have already sought to drive AI development and deployment in the right directions: governments around the world have established advisory councils or created rules for singular AI tools in certain contexts; NGOs and think tanks have published sets of principles and best practices; and private companies such as Google have released official statements about the ways in which their AI practices pledge to be “responsible.”

But none of these initiatives amounts to enforceable and measurable regulations. Furthermore, there isn’t always agreement between regions, sectors and stakeholders about what, exactly, is “responsible” and why.

“We’ve heard a growing group of voices in recent years sharing insights on how AI systems should be built and managed,” says Ashley Casovan, executive director of AI Global. “But the kinds of high-level, non-binding principles we’ve seen proliferating are simply not enough given the scope, scale and complexity of these tools. It’s imperative that we take the next step now, pulling these concepts out of theory and into action.”

A global certification mark like the one being built by SRI and AI Global is the next step.

“Recognizing the importance of an independent and authoritative certification program working across sectors and across regions, this initiative aims to be the first third-party accredited certification for AI systems,” says Hadfield.

How will it work? First, experts will examine the wealth of existing research and calls for global reform in order to define the key requirements for a global AI certification program. Next, they’ll design a framework to support the validation of the program by a respected accreditation body or bodies. They’ll also design a framework for independent auditors to assess AI systems against the requirements for global certification. Finally, the framework will be applied to various use cases across sectors and regions.

“AI should empower people and businesses, impacting customers and society fairly, while allowing companies to engender trust and scale AI with confidence,” says Kay Firth-Butterfield, head of AI and machine learning at the World Economic Forum. “Industry actors that receive certification would be able to show that they have implemented credible, independently-validated and tested processes for the responsible use of AI systems.”

The project will unfold over a 12- to 18-month timeline, with two global workshops scheduled for May and November of 2021. A virtual kick-off event will be held on Dec. 9, 2020.

'Making uncertainty visible': ��˾��ֱ�� researcher says AI could help avoid improper denial of refugee claims

Christopher.Sorensen — Tue, 20 Oct 2020 12:56:34 +0000

'Making uncertainty visible': ��˾��ֱ�� researcher says AI could help avoid improper denial of refugee claims

Christopher.Sorensen Tue, 10/20/2020 - 08:56

Cutline

Avi Goldfarb is a professor at the University of Toronto's Rotman School of Management and a faculty affiliate at the Schwartz Reisman Institute for Technology and Society (photo courtesy of the Rotman School of Management)

Topic

Schwartz Reisman Institute for Technology and Society

��˾��ֱ��

Rotman School of Management

Refugees

Avi Goldfarb is an economist and data scientist specializing in marketing. So how is it that he came to publish a paper on reducing false denials of refugee claims through artificial intelligence?

Goldfarb, a professor at the Rotman School of Management at the University of Toronto and a faculty affiliate at the Schwartz Reisman Institute for Technology and Society, read Refugee Law's Fact-Finding Crisis: Truth, Risk, and the Wrong Mistake, a 2019 book by Hilary Evans Cameron, a ��˾��ֱ�� alumna and assistant professor at the Ryerson University Faculty of Law.

He found some remarkable overlaps with his own work, particularly the methodology he employs in his 2018 book, Prediction Machines: The Simple Economics of Artificial Intelligence.

It just so happened that Evans Cameron had read Goldfarb’s book, too.

“It turned out we effectively had the same classic decision theoretic framework,” says Goldfarb, “although hers applied to refugee law and problems with fact-finding in the Canadian refugee system, and mine applied to implementing AI in business.”

Decision theory is a methodology often used in economics and some corners of philosophy – in particular, the branch of philosophy known as formal epistemology. Its concern is figuring out how and why an “agent” (usually a person) evaluates and makes certain choices.

The main idea around which Evans Cameron’s and Goldfarb’s thoughts coalesced was this: Human decision-makers who approve or deny refugee claims are, as Goldfarb noted in his research presentation at the Schwartz Reisman weekly seminar on Oct. 7, “often unjustifiably certain in their beliefs.”

In other words: people who make decisions about claimants seeking refugee status are more confident about the accuracy of their decisions than they should be.

Why? Because “refugee claims are inherently uncertain,” says Goldfarb. “If you’re a decision-maker in a refugee case, you have no real way of knowing whether your decision was the right one.”

If a refugee claim is denied and the refugee is sent back to their home country where they may face persecution, there is often no monitoring or recording of that information.

Goldfarb was particularly struck by the opening lines of Evans Cameron’s book: “Which mistake is worse?” That is, denying a legitimate refugee claim or approving an unjustified one?

In Goldfarb’s view, the answer is clear: sending a legitimate refugee back to their home country is a much greater harm than granting refugee status to someone who may not be eligible for it. This is what Goldfarb refers to as “the wrong mistake.”

So, from Goldfarb’s perspective as an economist and data scientist with specialization in machine learning (ML), a type of artificial intelligence, he started to wonder: Could ML’s well-known ability to reduce uncertainty help reduce incidences of “the wrong mistake”?

Goldfarb’s collaboration with and Evans Cameron reflects the intersections between the four “conversations” that guide the Schwartz Reisman Institute’s mission and vision. Their work asks not only how information is generated, but also who it benefits, and to what extent it aligns – or fails to align – with human norms and values.

“ML has the ability to make uncertainty visible,” says Goldfarb. “Human refugee claim adjudicators may think they know the right answer, but if you can communicate the level of uncertainty [to them], it might reduce their overconfidence.”

Refugee law expert Hilary Evans Cameron is a ��˾��ֱ�� alumna and an assistant professor at Ryerson University’s Faculty of Law (photo courtesy of Ryerson University)

Goldfarb is careful to note that shedding light on “the wrong mistake” is only part of the battle. “Using AI to reduce confidence would only work in the way described if accompanied by the changes to the law and legal reasoning that Evans Cameron recommends,” he says.

“When uncertainty is large, that does not excuse you from being callous or making a decision at all. Uncertainty should help you make a better-informed decision by helping you recognize that all sorts of things could happen as a result.”

So, what can AI do to help people realize the vast and varied consequences of their decisions, reducing their overconfidence and helping them make better decisions?

“AI prediction technology already provides decision support in all sorts of applications, from health to entertainment,” says Goldfarb. But he’s careful to outline AI’s limitations: It lacks transparency and can introduce and perpetuate bias, among other things.

Goldfarb and Evans Cameron advocate for AI to play an assistive role – one in which the necessary statistical predictions of evaluating refugee claim decisions could be improved.

“Fundamentally, this is a point about data science and stats. Yes, we’re talking about AI, but really the point is that statistical prediction tools can give us the ability to recognize uncertainty, reduce human overconfidence and increase protection of vulnerable populations.”

So, how would AI work in this context? Goldfarb is careful to specify that this doesn’t mean an individual decision-maker would immediately be informed whether they made a poor decision, and given the chance to reverse it. That level of precision and individual-level insight is not possible, he says. So, while we may not solve “the wrong mistake” overnight, he says AI could at least help us understand what shortfalls and data gaps we’re working with.

There are many challenges to implementing the researchers’ ideas. It would involve designing an effective user interface, changing legal infrastructure to conform with the information these new tools produce, ensuring accurate data-gathering and processing and firing up the political mechanisms necessary for incorporating these processes into existing refugee claim assessment frameworks.

While we may be far from implementing AI to reduce incidences of “the wrong mistake” in refugee claim decisions, Goldfarb highlights the interdisciplinary collaboration with Evans Cameron as a promising start to exploring what the future could bring.

“It was really a fun process to work with someone in another field,” he says. “That’s something the Schwartz Reisman Institute is really working hard to facilitate between academic disciplines, and which will be crucial for solving the kinds of complex and tough problems we face in today’s world.”

��˾��ֱ��'s Centre for Ethics explores ethical questions surrounding the COVID-19 pandemic

Christopher.Sorensen — Mon, 25 May 2020 15:58:02 +0000

��˾��ֱ��'s Centre for Ethics explores ethical questions surrounding the COVID-19 pandemic

Christopher.Sorensen Mon, 05/25/2020 - 11:58

Cutline

Markus Dubber, director of ��˾��ֱ��'s Centre for Ethics, says ��˾��ֱ�� is uniquely situated to tackle the ethical dimensions of the COVID-19 crisis because it's a "global research university with unusual excellence across the board" (photo by Chris Sorensen)

Topic

Is it acceptable to financially profit from the COVID-19 crisis? Should researchers publish preliminary COVID-19 drug research in real time? Should governments be using criminal sanctions to enforce public health guidelines? The continuing spread of COVID-19 around the world has raised a wide variety of ethical questions in many areas of our lives.

At the University of Toronto, the Centre for Ethics in the Faculty of Arts & Science is exploring these and other pressing issues in their new series of remotely broadcast talks, the Ethics of COVID.

“One of our regular attendees recently asked us to host online events about the ethical issues raised by COVID-19, and we thought it was a terrific idea,” says Markus Dubber, who is director of the centre and a professor in the Faculty of Law. “So, we’re building a virtual resource of interdisciplinary takes on the ethical dimension of the current crisis.

“This series highlights ��˾��ֱ��’s unique interdisciplinary strength as a global research university with unusual excellence across the board. Lots of universities are covering the science of COVID-19; there’s much less on the normative dimensions of the crisis.”

As a centre that draws researchers of all levels from across the university – from undergraduate to faculty members – the Centre for Ethics has a unique perspective on the ethical dimensions of crucial issues from a variety of academic streams, including law, medicine, public policy, philosophy and beyond.

All of the Ethics of COVID videos that have been produced so far are available on a YouTube playlist, with future sessions being added on an ongoing basis.

Watch the Ethics of COVID series at the Centre for Ethics

��˾��ֱ�� cognitive scientist livestreams daily meditation lessons during COVID-19

Christopher.Sorensen — Thu, 30 Apr 2020 19:38:29 +0000

��˾��ֱ�� cognitive scientist livestreams daily meditation lessons during COVID-19

Christopher.Sorensen Thu, 04/30/2020 - 15:38

Cutline

(photo by visualspace via Getty Images)

Topic

Cognitive Science Program

��˾��ֱ��

Psychology

University College

How do our minds deal with the increasing complexity of the modern world? How can we train ourselves to face life’s challenges? How do we stay connected in a world full of distractions and alienation?

Such questions are at the centre of the University of Toronto’s John Vervaeke’s academic work – not to mention his successful YouTube channel – and have never been more relevant than during COVID-19.

An assistant professor, teaching stream, in the department of psychology in the Faculty of Arts & Science and the cognitive science program at University College, Vervaeke (left) recently launched a new series of videos in response to the pandemic. Every weekday morning, he livestreams a short lesson about meditation followed by a brief silent meditation period.

Arts & Science writer Jovana Jankovic recently spoke to Vervaeke about mindfulness and meditation, particularly during times of stress and anxiety.

What are the biggest misconceptions about meditation?

One is that meditation is about achieving a kind of relaxation akin to sleepiness – that it should make your body and mind cloudy and dull, and your consciousness fade away. That’s not the kind of relaxation you want in meditation. You want a type of relaxation that enhances your sense of stability and your sensitivity. Meditation is not a vacation, it’s an education.

The other misconception is that you’re not meditating unless your mind goes wide open and blank. That’s exactly the wrong attitude. Every time you catch yourself in distraction and come back to your meditative focus, you’re actually building the mindfulness muscle. It’s like doing reps in weight training.

Why did you decide to do this series of morning sessions during the pandemic?

I think our culture in general is going through a meaning crisis in which we lack a sense of how we are connected to ourselves, to each other, to the world; how much we matter, how much we’re in touch with reality, how much we’re overcoming self-deception, how much we’re affording wisdom.

If the meaning crisis is a fundamental sense of disconnection, the COVID crisis certainly exacerbates that. People feel very disconnected from their life, disconnected from the world, disconnected from each other. So, while it’s good to develop a mindfulness practice in general, I think it’s especially pertinent right now.

You say that frequency of meditation is more important than length of sessions. Could you tell us more about that?

Continuity of practice is more important than quantity, but that doesn’t mean the quantity is irrelevant. If you’re trying to learn something new and you just stay inside your comfort zone, you’re not challenging yourself, which is how learning happens.

So, when you’re sitting for meditation, if you only sit as long as it’s comfortable, you don’t get into what psychologists call the “zone of proximal development.” That’s where you learn new things. You have to keep sitting when it’s challenging and you have to use the principles and practices you’re taught to keep going.

But if you say, "Well, I can’t sit for a full 15 minutes, so I won’t sit at all," that will erode your practice. If you can honestly say to yourself that you can only sit for five minutes, then sit for five minutes. It’s not always sufficient, but it’s certainly better than nothing.

What are some quick and easy tips for beginners who are just trying meditation?

People try to get into a posture that is free from unpleasant sensations or discomfort. But you’ll never get there. I’ve been meditating for 20 years and I’ve never found such a thing. That’s really important to remember.

And centring your mind doesn’t mean just focusing your attention. It means stepping back and looking at your mind rather than looking through it. The metaphor I use is my glasses: I look through my glasses all day long, but if they are full of gunk, what I have to do is actually step back and look at them.

You need to try to do the same thing with your mind: Step back and look at its patterns and processes. And don’t frame meditation as instantly getting your mind to go blank. You have to learn to sit until your mind settles.

Finally, you need both a meditative practice and a contemplative practice. We use those terms as if they’re synonyms, but they’re not. To go back to my analogy, if meditation is like stepping back and looking at your glasses, how do you know if you’ve spotted a distortion or a defect in your glasses? You have to put them back on, right?

If you put them back on and you see better, that’s a contemplative aspect of the practice. Can you see through what was previously illusion into reality? This relates to work I did with my former student Leo Ferraro, who was a TA and an alumnus in the cognitive science program as well as my co-author for a book chapter called “Reformulating the Mindfulness Construct.”

Cognitive science is perhaps not as well-known as psychology or neuroscience. What is it?

In many ways, cognitive science is close to philosophy and particularly ancient philosophy like that of Socrates and Plato. It’s about studying the mind and its relationship to reality in a very comprehensive manner.

In the modern world, we have a discipline that studies the brain and that’s neuroscience. We have computer science, which studies artificial intelligence – that’s about systems that can solve problems and do information processing. Or we have psychologists, who study human behaviour. We might go to a linguist who studies how we communicate through grammar and syntax with each other. Or we might be concerned with culture, so we might go to an anthropologist for that.

Those things are not isolated. They interact with and affect each other. And cognitive science gets all of those other disciplines to insightfully talk to each other as they study the mind in their different ways. It’s sort of a very powerful bridging discourse between them.

��˾��ֱ��'s School of the Environment launches first stand-alone graduate degree

Christopher.Sorensen — Wed, 11 Mar 2020 13:40:14 +0000

��˾��ֱ��'s School of the Environment launches first stand-alone graduate degree

Christopher.Sorensen Wed, 03/11/2020 - 09:40

Cutline

Nicolas Côté, Rashad Brugmann and Nathan Postma in Trinity College's rooftop garden on the St. George campus (photo by Geoffrey Vendeville)

Topic

Computer Science