# Book Notes: Human Compatible: AI and the Problem of Control #Booknote #AI [https://people.eecs.berkeley.edu/~russell/](https://people.eecs.berkeley.edu/~russell/) [https://www.goodreads.com/book/show/44767248-human-compatible?ac=1&from_search=true&qid=SjFG8jUCJS&rank=1](https://www.goodreads.com/book/show/44767248-human-compatible?ac=1&from_search=true&qid=SjFG8jUCJS&rank=1) "The most important book on AI this year." --The Guardian"Mr. Russell's exciting book goes deep, while sparkling with dry witticisms." --The Wall Street Journal"The most important book I have read in quite some time" (Daniel Kahneman); "A must-read" (Max Tegmark); "The book we've all been waiting for" (Sam Harris) A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly intelligent machinesIn the popular imagination, superhuman artificial intelligence is an approaching tidal wave that threatens not just jobs and human relationships, but civilisation itself. Conflict between humans and machines is seen as inevitable and its outcome all too [predictable.In](http://predictable.In) this groundbreaking book, distinguished AI researcher Stuart Russell argues that this scenario can be avoided, but only if we rethink AI from the ground up. Russell begins by exploring the idea of intelligence in humans and in machines. He describes the near-term benefits we can expect, from intelligent personal assistants to vastly accelerated scientific research, and outlines the AI breakthroughs that still have to happen before we reach superhuman AI. He also spells out the ways humans are already finding to misuse AI, from lethal autonomous weapons to viral sabotage.If the predicted breakthroughs occur and superhuman AI emerges, we will have created entities far more powerful than ourselves. How can we ensure they never, ever, have power over us? Russell suggests that we can rebuild AI on a new foundation, according to which machines are designed to be inherently uncertain about the human preferences they are required to satisfy. Such machines would be humble, altruistic, and committed to pursue our objectives, not theirs. This new foundation would allow us to create machines that are provably deferential and provably beneficial. ## Chapter 1.  IF WE SUCCEED - Instead of designing AI to set actions to meet their objectives, AI should meet human objectives. - History of AI research. The current social media, social platform example of the potential risk of AI on human civilisation. Code change human behaviour. - Though we don't really know what want, that's a feature perhaps? Because AI will keep checking and deferring to human. - Don't underestimate human ingenuity. We will have ultra intelligent AI **Computer do what we tell them to do, what if the objective we ask them to do is bad? and the AI came out with their own methods, which are bad.** **We design conditions to create the outcome we want, in every where. #systemic** - _"Because machines, unlike humans, have no objectives of their own, we give them objectives to achieve. In other words, we build optimizing machines, we feed objectives into them, and off they go._ - _This general approach is not unique to AI. It recurs throughout the technological and mathematical underpinnings of our society. In the field of control theory, which designs control systems for everything from jumbo jets to insulin pumps, the job of the system is to minimize a cost function that typically measures some deviation from a desired behavior. In the field of economics, mechanisms and policies are designed to maximize the utility of individuals, the welfare of groups, and the profit of corporations.9 In operations research, which solves complex logistical and manufacturing problems, a solution maximizes an expected sum of rewards over time. Finally, in statistics, learning algorithms are designed to minimize an expected loss function that defines the cost of making prediction errors._ - _Evidently, this general scheme—which I will call the standard model—is widespread and extremely powerful. Unfortunately, we don’t want machines that are intelligent in this sense._ - _The drawback of the standard model was pointed out in 1960 by Norbert Wiener, a legendary professor at MIT and one of the leading mathematicians of the mid-twentieth century. Wiener had just seen Arthur Samuel’s checker-playing program learn to play checkers far better than its creator. That experience led him to write a prescient but little-known paper, “Some Moral and Technical Consequences of Automation.”10 Here’s how he states the main point:_ - _If we use, to achieve our purposes, a mechanical agency with whose operation we cannot interfere effectively . . . we had better be quite sure that the purpose put into the machine is the purpose which we really desire._ - _“__The purpose put into the machine” is exactly the objective that machines are optimizing in the standard model. If we put the wrong objective into a machine that is more intelligent than us, it will achieve the objective, and we lose__. The social-media meltdown I described earlier is just a foretaste of this, resulting from optimizing the wrong objective on a global scale with fairly unintelligent algorithms._ ## Chapter 2.  INTELLIGENCE IN HUMANS AND MACHINES - The more we understand how our brain/mind works, the closer we are at creating  Artificial Intelligence. - What is Intelligence? The ability to perceive environment, make decision, to move closer to it's objective. - Simple organisms may be able to react to environment, make decision and move closer to objective but do not learn and remember. - We do not know how our mind works. We understand reward system, much like simple organism. Nature way to help organism to adapt and move closer to goal of propagating gene. But can at times be maladaptive, like dopamine addiction to games, drugs, etc. - Baldwin Effect, coined by James Baldwin 1986 , and independently by Conwy Lloyd Morgan proposed that Learning is connected with Evolution. The ability for an organism to learn sped up evolution. Organised civilisation protect the young, while they learn and passes on the information to the individual. - What we learn may not be related to end goal of propagating genes. But somehow Learning Mechanisms must have been passed down through evolution, because it aid in survival. - From evolution point of view, nature treat us as only an _agent_ to pass down gene. Doesn't really care about whether we are "unique individuals" ## Chapter 3.  HOW MIGHT AI PROGRESS IN THE FUTURE? - Can we use lookahead (based on current bio-digital marker, and past observations (from other patients data), to predict trajectory? and advice how to adjust that path to prevent relapse? #hope-s - Using Lookahead search, based on probable future trajectories of relevant objects, based on both current and past observations, then use lookahead search to find a trajectory that optimisers safety and progress. - Constructing different models of all kinds of events and transactions that make up our daily lives. I wonder, if i need to construct different models for relapsing, distresses, loneliness etc. - Loc 1076 - What is common operational picture? loc 1084 - [physiome.org](http://physiome.org) current physiological modeling. - Can i keep track of what patients know or don't know in term of psychoeducation? - Probabilistic reasoning technology can now keep track of what students know or don't know (loc 1101) - The chapter also cover the future of AI imagined, with abilities, connected one superinteligent AI, but not God-like, will have limitation. - AI don't understand human, and language as human do.. so still not that smart. ## Chapter 4.  MISUSES OF AI - Read this on 24 Jan 2022 - Weapon - Surveillance - Losing job - Universal Basic Income - Argument always about helping profession will not be replaced by AI, and it make people look weak and needy to be "helped" but the author said that it's not about dependency, but about growth, as human. - Loc 2021 - "...Consider this observation, again from Keynes: '_It will be those peoples, who can keep alive, and cultivate into a fuller perfection, the art of life itself and do not sell themselves for the means of life, who will be able to enjoy the abundance when it comes.'_ All of us need help in learning "the art of life itself." This is not a matter of dependency but of growth. The capacity to inspire others and to confer the ability to appreciate and to create - be it in art, music, literature, conversation, gardening, architecture, food, wine, or video games - is like to be more needed then ever." - Usurping human function - not dignifying, no reason to make it human like. Or higher rank then human - ## Chapter 5.  OVERLY INTELLIGENT AI Different analogies and scenario of how AI can be dangerous. Like human evolved to be more intelligent then animals and decided how animal lives. And King Mindra (who turn anything he touches into Gold...) ## Chapter 6.  THE NOT-SO-GREAT AI DEBATE Current denial stance even within AI research community about the risk of AI. From deflecting, "what about..", too early to worry about it etc. ## Chapter 7.  AI: A DIFFERENT APPROACH ## Chapter 8.  PROVABLY BENEFICIAL AI ## Chapter 9.  COMPLICATIONS: US ## Chapter 10.  PROBLEM SOLVED? ## Appendix A. SEARCHING FOR SOLUTIONS ## Appendix B. KNOWLEDGE AND LOGIC ## Appendix C. UNCERTAINTY AND PROBABILITY ## Appendix D. LEARNING FROM EXPERIENCE