Working
draft
More Notes on Self Awareness
Abstract
This paper is under development and will
change over time. These notes make explicit some
general issues relating to the development of a computational agent with
self awareness – representation of self, architecture, single or multi-agent
computations, user interface. We do this by considering
the construction of a particular self aware agent, a self debugging agent.
A strawman proposal for a Self Awareness Engine is also included. The have a short appendix on self awareness as it appears
in eastern philosophies of mind.
I
Think Therefore I Am
Philosophers through the ages have wrestled with issues of self in humans, and today, bookstores are loaded with self help books based on self awareness. Humans struggle with their self identity. They have identity crises, mid-life crises, and even throw themselves off bridges. Do we really want to give a computer agent self awareness? Not in the same way perhaps. Drawing on our own experiences as humans, drawing on our own introspective processes is, as John pointed out, a valuable source of information. But we may need a Wizard of Oz test.
The promise of self awareness includes 1) agents that can self debugging 2) agents that take action to avoid maintenance and operational problems before they get to the breaking point 3) agents that help us to debug them 4) new methods for machine learning, and 5) fraud detection - if an agent can distinguish between self and others, an intruder would be observable.
These notes consider the idea of self awareness with a particular application in mind – a self debugging computational agent. The intention in part is to make some of the implicit assumptions of our discussions more explicit but also to drive up ideas about what kinds of things we need to consider in order to really build such an agent. Many issues will pertain to all sorts of self aware agents, but picking a particular application can shed light on issues like architecture, representation of self, issues in reasoning, meta control, etc. We also consider a Self Awareness Engine.
It
All Begins With Awareness
Self Awareness is both a deliberative process and a representation of state. In determining issues of SELF AWARENESS in a computational agent, it may be of use to set out what each of us means by the notion of “SELF AWARENESS” for a computational agent. Is SELF simply mental state and the objects within that mental state, or does it encompass the entire machine, including the state of the operating system, devices, files, and so on? Are these notions implicit or explicit? Can a toaster have SELF AWARENESS? A car? A keychain that knows where it is, or a car that knows where it is would be extremely useful if it can communicate to us its whereabouts.
But we really don’t need the car or keychain to have a Self Awareness Engine when all we need is to press a button and it honks or flashes its light at us. If I lost my toaster, well, that’s another matter.
A
Useful Sense of Self
Perhaps the best way to define SELF for a computational agent is to leave it to what appears “useful”. Task context can determine what self awareness should look like. For instance, debugging could be driven by asking the system what its state is, and comparing it to what you think it should be. An agent that debugs itself must examine its own state of self, asking which parts of self are all right, which ones are problematic. Without explicit representation of self, any kind of reasoning mechanism – whether about diagnosis, repair , learning patterns of problems, could not function. Representation of state would have to be derived to kick start the self evaluation engine. A Self Awareness Engine pictured below, could deal with this.
It may be useful to create explicit representations of
the state of devices in mental state, or it may be sufficient to simply poll
the operating system. It may be useful in creating
In many ways our approach, as humans, in debugging our emotions, can be useful in extracting how a machine might debug itself. Problem solving in emotions requires us to examine our state of self, asking which parts of self are all right, which ones are problematic. Without awareness of state there is nothing that can necessarily help us to change and we are doomed to do the wrong things over and over. Self awareness helps keep us on track. How long have I been doing this? Is this good, not good? Marvin Minsky talks about such issues in his book, The Emotion Machine, available on his website at MIT.
Parameters
Parameters of agent circumstance are one way to line up the states an agent may be concerned with. Whether an agent exists in a multi-agent environment or alone, and whether such multi-agent environment is a hybrid one, including both humans and computational agents. We might also parameterize along agent architecture lines – reactive, hybrid, symbolic. Yet another parameter is to characterize the MENTAL STATE of the agent – emotive, logical, or hybrid.
The
HUMAN Condition (Chocolate, Beer, and Cheese Please)
While it is useful to bring our own introspection of self to bear on the problem of computational self awareness, doing so will in some sense force us to find the boundary between man and machine. For instance, many humans are lazy and will not stop for a check-up on or change ourselves unless or until something stops working. In humans, preventative maintenance, preventative debugging, is based on increased self awareness. Few humans engage in this activity, although it is possible. If introspection of ourselves is enough, why are there so many self help books out there? Apparently there are deliberative routines we humans must pay attention to in order to change our awareness so that we need help us change, making improvements or repairs. But even that is not always enough. I know that chocolate is making me fat. And I have read a book about it. And everyday I wake up to post-its on my door reminding me about my weight. But I eat it anyway. Is there some other aspect of self awareness I am missing? Perhaps. If my best friend quit eating chocolate too, it would be easier for me as a human, to quit.
Can a Robot die of a broken heart? Can a computational agent be ill-willed or good-natured? Does The Power of Positive Thinking pay off in the same way as with humans? Is there a circumstance in which “Going Against The Odds” makes perfect sense for a computational agent? Is it useful to define the computational states of self awareness so that we may encompass the breadth and depth of human self experience? I think yes.
A “depressed” robot, such as Marvin, could be programmed to find the worst thing that might ever happen in a given situation. Pairing Marvin with an overly optimistic human would be useful. Alternatively, an “optimistic” Robot, would do well to continue pointing out how things might go positively and encourage humans not to give up. Would it be useful for Marvin to be aware that he is depressed? In AI, the robot David, is equipped with self awareness. His architects gave him not only traditional AI logic, but social and emotional components. He reaches an emotional crisis when his parent abandons him and “commits suicide”.
Although we can argue that Spielburg only did this because he could find no other way to end the movie, the situation is relevant. Had the robot been equipped with some type of self awareness software, or had a robot psychologist like Susan Calvin (I Robot), perhaps, awareness of the threatening emotional state within could have directed David to change focus of attention, find Susan Calvin, etc., to change the state leading to such a dire outcome.
In humans, our ability to recognize things in ourselves is what gives us the capability to recognize them in others. There may be applications and circumstances in which representation of self in a computational agent can be useful in this way, a robot that acts as a psychologist for humans….
Recognition that states of self are out of order does not just pertain to emotional state. Awareness of physical state before physical breakdown or expensive repairs must occur would be useful. The process by which awareness of self is derived and the processing of information may or may not be the same for physical, emotional, or logical aspects of a computational representation of self.
CHALLENGES
The vision of a self aware robot will require technical solutions to many different problems, including
· Determining, based on the situation, objectives or intent – what information about the self needs to be gathered, its priority, and value (these can be context dependent)
· Representation of self that may include both explicit and implicit information sources.
· Updating model(s) of self awareness based both on operator supplied information and agent supplied information - self monitoring and discovery processes, deliberation processes, sensing registers, a mirror, etc.
· How to revise and resolve mismatches between machine supplied information of machine state and external human appraisal or any other information gathered from external sources.
· Human/device communication - Similarly, human observation of machine states of self awareness may be both explicit and implicit (asking versus watching)
· Maintaining a model of self awareness that explicitly incorporates several levels of self awareness, possibly several representations, etc.
· Trading off deliberation of self and action –similar to tradeoffs we must face in planning and execution, etc.
· Controlling and regulating deliberation processes executing concurrently and possibly in parallel with other cognitive processes, including focus of attention, selecting and processing perceptual information, domain problem solving, context switching and maintenance, etc
· Managing and filtering awareness of self for multiple views of self, selecting the mode of presentation of the information for sharing or for self debugging, for the task at hand, and so forth
· Relating to this is the presentation of information for human/human interface from both implicit and explicit data structures. Typically this is achieved upward through layers of abstractions. One hypothesis of this approach is that minimal path transitions through the abstraction structure will also represent minimal cost switches of context. This may or may not be true with information models requiring persistent updates (compare transaction databases with analytics databases).)
· Others?
APPROACH
Echoing John’s observation, introspection of self may well be our closest ally in sorting out some of the issues of building a computational agent possessing self awareness.
Each of us will no doubt have varying levels of self awareness, so perhaps the more opinions and ideas about this the better. It is also useful to consider where the line is between what is useful for a computation agent and what is not. However, it may also be useful to consider philosophies of mind where considerable instruction is directed at self awareness – developing it and using it for self debugging and regulation. This is discussed briefly in an appendix to these notes.
It is an intentional act to develop more self awareness. Resources are necessary for creating, deriving, and maintaining self awareness. It may be necessary to transform implicit awareness of state (such as from the OS) into explicit forms of state for further reasoning. The architecture of a computational agent would include deliberative processes and components to support such processes, and perhaps meta control to the problem of endless deliberation.
How do common CS concepts or AI concepts apply in an agent with self awareness? If
self awareness begins with a “representation of self”, what data structures make sense for self awareness? If self awareness is partially or wholly implicit, such as in the OS registers, files, logs, it will look different than if it is represented in a belief based agent – where concepts of mental state are explicitly represented in languages like LISP (BBRL[Mason, 1994], AOP[Shoam, 1994], EOP [Mason and Sloman 2001], etc.)
If self awareness is considered to be an action (as well as a state), what kinds of processes will be needed to generate, maintain and act based on states and patterns of self? Are the learning processes of an agent inherently different? Data warehouses of states over time would be valuable in debugging and detecting trends, much like any other sort of data warehouse. Analytic processes of self awareness may access different data warehouses than highly volatile “transaction” data bases, where state is frequently updated.
In [Mason, 1994], belief based agents were attributed as having “introspection” and representation of self (of others) since their mental state included explicit representation of whether or not they believed a particular proposition. A “regular” sort of agent would not represent explicit belief statuses in its language or state, but rather rely on the failure of the rule to fire. In this way an agent achieves control of its own behavior determining a course of action based on explicit representation and reasoning about what it presently believes, disbelieves (knows). Here, UNKNOWN refers to an assertion that is neither Believed nor Disbelieved.
(AGENT-MAINTENANCE
(EVALUATE-AND-RESPOND-INCOMING-BATTERY-STATUS-REQUEST
((BELIEVES(REQUEST instance $R1)
($R1 type $rt1)
($rt1 type ‘BATTERY-STATUS-UPDATE)
($R1 requesting-agent $id )
($R1 authorization (eval-authority $id LIST-OF-AUTHORIZED-AGENTS))
($R1 request-reason $r)
($R1 request-status $s)
((UNKNOWN
(
($b1 battery-status $s)
(PRIORITY-LEVEL SURVIVAL)
(let ((EVALR1 (create-snode)))
(format t “Evaluating current battery status”)
(assert ((EVALR1 request-evaluation (eval-request($r1 $rt1 $s)))))
(sndmsg Monitor-Agent ((EVALR1 request-evaluation $e) $r1)) )))))
Agent
Architectures and Representations That Support Reflective Processes
To increase understanding of the usefulness of self awareness in agent behavior we should not restrict ourselves to a particular architecture but encourage the exploration of various compositions of architectural components and interactions. Construction of an architecture with several concurrently processing layers (in terms of an information processing model) may be supported would be important, where each layer relates to different aspects of problem solving, including self awareness, as well as an architecture that may call out all self awareness functions into a separate component.
Representation of state has implications in almost all of the computations we are concerned with here. In the Self Awareness Engine pictured earlier, a granularity machine can help us to work with multiple abstractions, and the Conflict Resolver seems to attack the problem of multiple observers, but there is no mention of whether separation of Self Awareness into mental, physical, emotional can be useful.
In both [Sloman and Logan 2000] and [Mason and Sloman 2001], only mental and emotional states are explicitly represented in the agent, since physical was not of interest, and they were maintained in separate databases. The architecture for an emotionally self aware agent in [Mason and Sloman 2001] features 3 TMSes, all context based. One TMS worked to keep consistency among logical assertions and assumptions, another for emotional states, and a third to orchestrate consistency between emotional and physical. Explicit representation regarding agent “personality” provided the third TMS with guidelines for conflict detection.
References
[Sloman and Logan 2000] A. Sloman and B.S. Logan, Evolvable architectures for Human-Like Minds, in Affective Minds, G. Hatano, N. Okada and H. Tanabe, Eds., Elsevier, Amsterdam, 2000.
[Mason and Sloman 2001] A. Sloman and C. Mason, Emotionally Intelligent Agents, Unpublished White Paper, 2001.
[Mason 1994] C. Mason,
Introspection as Control in Multi-Agent Systems, AAAI Workshop on Distributed
Artificial Intelligence,
[Mason 2003a] C. Mason,
Reducing Number of Hospital Days and Side Effects Using Physiophilosophy,
Psychoneuroimmunology Research Society Meeting,
[Mason 2003b] C. Mason, Global Medical Technology, Proc. Symposium on Future Healthcare Technology, MIT, 2003.
[Minsky 2003] M. Minsky, The Emotional Machine, www.media.mit.edu/~minsky.
[Zadeh 2002] L. Zadeh, Granularity Computing, BISC Seminar, U.C. Berkeley, 2002.
APPENDIX
Notes from eastern philosophy
The west has had good fortune in developing science and technologies, such as antibiotics and artificial intelligence, while the east has profited from soft technologies based on mind[1] training that have evolved over thousands of years.
Eastern philosophies of
mind are a kind of correctional facility based on self awareness training.
Much of the training aims to develop, monitor, and
influence self awareness so that cognitive capabilities are either corrected
or trained to operate at their peak, regardless of circumstances we may encounter
in life. The mental processes
and deliberations that are part of mind training have been documented in
having effects on mental, physical and emotional functioning. Established studies in the east
and west have documented changes in brain chemistry, blood pressure, immune
function, pain, etc. as a result of persistent mind training (the interested
reader may wish to look at
As human beings, we face many difficulties existing – we overeat, we have anxiety, we need positive encouragement, we fear being alone, we fear being around others, we have addictions, hatreds, worries, we are deeply sad, we have existential crises, we have mid life crises, we have fear of abandonment, and so on. These keep us from performing optimally or from staying healthy and happy. Many of our human behaviors can stem from areas of the mind we are not yet consciously (explicitly) aware of and which control what we do although we are not “aware” of these beliefs.
Mindfulness training or “insight meditation” is based on the idea of creating an observer. Noticing the train of thoughts. In a thirty minute period how many times did you think about cheese? Or how angry you are about something. Or about the pain in your back. Or about your desire for a cigarette. Naming the train of thoughts, conflicts, tensions, without judgment. A nonjudgmental line of inquiry by the observer is the first step to develop “insight” into where these thoughts come from and why. Measurably, concentration improves when we are able to clear away these intrusive thoughts. In humans, gentle, accepting inquiry into where the thoughts come from generates more self awareness, which leads to understanding and chance to correct things. Noticing that you thought about chocolate 25 minutes out of the last 30, you make a goal of only thinking about chocolate for 15 minutes out of the next 30 minutes.
Other concepts include something called Beginners Mind, which describes the state of our mind when self awareness deliberations have “completed” and – like a beginner, everything is in working order and we see things fresh, without coloring what we see in order to avoid pain, or other difficulties stemming from prior experience. The idea of seeing the self with a mirror is another useful concept. Most of us have mirrors that favor our reflections and result in a gap between what we perceive as self and what is the self. A clear mirror of self reflection enables us to see our imperfections and register a sense of self awareness of the imperfection without judging. The awareness leads to gentle inquiry. The observer plays neither judge nor critic, but simply keeps asking until the answers arise, a positive approach which promotes the further development of non judgmental inquiry and further development of self awareness. To realize after some time that you really crave cheese because as a child your mother used to bring it home as a special treat and you felt loved. Now that you are alone, you need to feel loved, so you eat chocolate. But since the doctors say you have to have surgery to save your life, perhaps its time to find another way of feeling loved. This makes you feel sad. Observing the sadness, this leads to inquiry about the loves you have had, where they went, and how your awareness of your behavior or can create more love, using awareness of self. Etc. Etc., Etc.
How do some of these ideas map onto a computational agent? Which ones don’t? Do you see any parallels for self awareness and debugging in computational agents? Is the observer conducting a depth first search in a way?
It may be of interest to
some AI researchers that a kind of physiophilosophy from