Logic, language and the brain Michiel van Lambalgen

Logic, language and the brain Michiel van Lambalgen Cognitive Science Center Amsterdam https://staff.science.uva.nl/˜michiell Aim and program • aim: explain the use of computational logic in cognitive science – the domain is language comprehension and production – show how logical modelling leads to testable predictions, both for behaviour and brain imaging – show how logical modelling connects to biological issues, e.g. neural substrate of linguistic processing, and evolutionary considerations • lecture 1: the big picture – time, tense and cognition • lecture 2: the event calculus • lecture 3: verb tense and closed world reasoning • lecture 4: predictions for EEG • lecture 5: executive function and behavioural predictions for autism and ADHD; neural network architecture Literature 1. series of articles by Baggio, vL & Hagoort (most recently in Journal of Memory and Language 2008) 2. (with Fritz Hamm) The proper treatment of events, Blackwell 2004 (linguistic details); cited as PToE 3. (with Keith Stenning), Human reasoning and cognitive science, MIT Press 2008 (evolutionary and methodological considerations); cited as HRCS The big picture – time, tense and cognition • methodology: a role for logic in cognitive science • an ancestral precursor to language: planning • planning, causality, time and tense Why logic is unpopular in cognitive science: 5 misconceptions • experiments with reasoning tasks, such as the famous Wason selection task show that logical form is not a determinant of reasoning • logic cannot deal with vagueness and graded concepts • logic cannot deal with uncertainty and must be replaced by probability theory, which is after all the calculus of uncertainty • what we know about the neocortex suggests that the computations executed by the brain are very different from logical computations • the computational complexity of logic is too high for logic to be a realistic model of cognition A novel variation: the two-rule task (S & vL) Below is depicted a set of four cards, of which you can see only the exposed face but not the hidden back. On each card, there is a 4 or 7 on one of its sides and a A or K on the other. Also below there are two rule which apply only to the four cards. It is given that exactly one rule is true. Your task is to decide which if any of these four cards you must turn in order to decide which rule is true. Don’t turn unnecessary cards. Tick the cards you want to turn. 1. if there is a A on one side, then there is an 4 on the other side 2. if there is an K on one side, then there is an 4 on the other side Cards: A K 4 7 A role for logic in cognitive science • David Marr’s hierarchy of explanation in cognitive science: a cognitive capacity can be studied at (at least) the following levels 1. abstract specification of the capacity, including input–output behaviour, in a formal language [informational level] 2. algorithm computing the input–output behaviour [algorithmic level] 3. neural implementation of the algorithm [neural level] • example: informational level description of language comprehension could run as follows • input: discourse is transformed into output: discourse model (called situation model in pyscholinguistics) • DRT is an example, but we will not adopt it because (i) it is monotonic (this conflicts with a processing principle called immediacy – see lecture 4), and (ii) the algorithm is not easily related to neural implementation, or to what is known about cognition Logic as high-level description of cognitive functions • another example: ’executive function’ • ‘executive function’ is an umbrella term for processes responsible for higher-level action control that are necessary for maintaining a goal and achieving it in possibly adverse circumstances • we may take executive function to be composed of planning, initiation, inhibition, coordination and control of action sequences, leading toward a goal held in working memory • several of these components have logical specifications • this will be illustrated by showing how executive function is involved in discourse production and comprehension Discourse • discourse is not just bunch of sentences: what is special about discourse is that it is always interpreted relative to the verbal and nonverbal context – e.g. world knowledge • experimental evidence for the reality of temporally organised discourse models – memory for narrated events not in order of narration, but in order of occurrence (‘Max fell. John pushed him.’) – events occurring at temporally contiguous times are related more closely in memory • experimental clues to the constitution of discourse models – causal information about the events of interest is represented, including default expectations: sentences of the type ‘John was beaming. A moment later . . . ’ have shorter reading times then sentences of type ‘John was beaming. An hour later . . . ’ – this is interpreted as indicating that the default continuation of the discourse model that supports the first sentence doesn’t support the second Discourse comprehension in a wider context • the construction of discourse models is not specific to language! • a discourse model consists of entities (e.g. introduced by indefinite NPs), relations between these entities, events (e.g. introduced by VPs), temporal relations between events • structures consisting of events and relations between them occur elsewhere in cognition as well • a plan is a sequence of actions (together with the times at which these actions have to be executed) which achieves a given goal • (the assumption of sequentiality can be relaxed: often actions have to be executed in parallel) • a discourse model is analogous to a plan and it may be the case that the same construction algorithm is applied Why is planning possibly important? • amplified definition: planning consists in setting a goal and proposing a sequence of actions which provably leads to the achievement of that goal, taking into account properties of, and events in, the world • (hierarchical) planning occurs in humans and in nonhuman primates • evolution of language: planning has been hypothesized to be co-opted for syntax because of ‘recursive’ structure (Lashley 1951; Greenfield 1991; Steedman 2002) • Broca’s area (involved in language) immediately adjacent to areas for motor planning (Greenfield et al. 1972) • possibility for discourse may distinguish ape language/human language • planning can organize (e.g. temporal features of) discourse so as to determine the event structure described by the discourse analogous to a structured set of actions leading to a goal • here, events are the denotata of untensed sentences and verb phrases Intermezzo: planning and time ‘Using goal-plan knowledge to merge the past with the present and the future in narrating events on line’ (Trabasso and Stein 1993) • uses Berman and Slobin’s ‘Frog where are you?’ paradigm • plans act like glue: ‘the plan unites the past (a desired state) with the present (an attempt) and the future (the attainment of that state) . . . ‘causality and planning provide the medium through which the past is glued to the present and the future’ • basic narrative unit: GAO – ‘goal-action-outcome sequence’ • the capacity to structure a narrative in GAOs develops gradually: – absent at 3 (almost no GAOs; story is basically a sequence of stills, none of the actions described relevant to the boy’s goal) – sharp increase GAOs starting age 4 until age 9 – maximum reached at age 21 The frog story: opening scenes #1 #2 #3 The frog story: a typical mishap Planning: what is the informational level description? • amplified definition: planning consists in setting a goal and proposing an ordering of actions which provably leads to the achievement of that goal, taking into account properties of, and events in, the world • one needs – facts and causal laws holding in the world, including laws about consequences and preconditions of actions – a goal – a reasoning mechanism that outputs a plan – the reasoning mechanism should not suffer from the ‘frame problem’, i.e. should not take into account all logical possibilities – this makes it non-monotonic: goal G can be expected to be attainable in circumstances C, but not anymore in C ∪ D Logical characterisation of planning • computing a plan is closed world reasoning (CWR) in a formal system for representing and reasoning with causes, actions and consequences • CWR: ’if you have no reason to expect event e, assume e will not occur’ • this principle can be viewed as the construction of a very special kind of model • toy example – a:= turn light on (‘action’ or ‘event’), r:= light is on (‘state’) – predicates: holds(x , n), do(e, n), ab(n, k ) – causal axiom: ∀n∀k(¬holds(r , n)∧do(a, n)∧k > n∧¬ab(n, k ) → holds(r , k )) – goal: holds(r , 10 ) – boundary condition: ¬holds(r , 1 ); time is interval [1, 10] The output of planning is a model with events and states • the obvious plan is do(a, 9 ) • (this works only if ¬holds(r , 9 ) and ¬ab(9 , 10 )) • the obvious plan together with the conditions for its execution determine a model defined by holds(r, n) ↔ n = 10, do(a, n) ↔ n = 9, ab(n, k) ↔ n 6= n. • this model is minimal in the sense that the extensions of the predicates are as small as possible (‘closed world model’) • there are other models, e.g. holds(r, n) ↔ n = 2, 4, . . . , 10, do(a, n) ↔ n = 1, 3, . . . , 9, ab(n, k) ↔ (n, k) = (1, 3), (3, 5), (5, 7), (7, 9). • here extensions of predicates are larger – and ‘unexplained’ Computational features of the ‘closed world model’ • CWR comes in 3 forms – CWR for rules: if one knows that q and the only rule with q as consequent is p → q, then assume q; in effect one thus replaces p → q by p ↔ q – CWR for facts: if there is no rule with q as consequent (in particular, if q is not a fact), one may assume ¬q – CWR for goals: if the given goal is to make A(c) true, then one may assume A(x) can be made false for x 6= c • CWRr : ∃n(¬holds(r , n) ∧ do(a, n) ∧ k > n ∧ ¬ab(n, k )) ↔ holds(r , k ) • CWRf : ∀k∀n¬ab(n, k) • CWRg : try to ensure that the goal is satisfied minimally, i.e. holds(r, n) ↔ n = 10 The computation • CWRr : ∃n(¬holds(r , n) ∧ do(a, n) ∧ k > n ∧ ¬ab(n, k )) ↔ holds(r , k ) • CWRf : ∀k∀n¬ab(n, k) • CWRg : try to ensure that holds(r, n) ↔ n = 10 CWRg gives ∃n(¬holds(r , n) ∧ do(a, n) ∧ k > n ∧ ¬ab(n, k )) ↔ k = 10 CWRf gets rid of ab: ∃n(¬holds(r , n) ∧ do(a, n) ∧ k > n) ↔ k = 10 and we have ∃n < 10(¬holds(r , n) ∧ do(a, n)) ∀n < 9(¬holds(r , n) → ¬do(a, n)) Again CWRg gives ∀n < 9¬do(a, n), whence do(a, 9) What we can learn from this • we specified a planning algorithm through closed world reasoning • the output is a time-based model containing actions/events and states (together with their times of occurrence) which is minimal in two senses: – the least number of actions to be performed – the least interference from outside events • such models are quite like discourse models • there is also a connection with tense/aspect What we can learn from this • consider the sentence in present perfect ‘Bill has turned on the light’ • identify time point 10 with now (utterance time) • the pres. perf. is formulated in planning terms as the instruction to satisfy the goal ‘light on now ’ (formally holds(r, 10)) • this captures ‘present relevance’ of the present perfect: reference time (given as now ) anchors the state consequent upon the event (namely r); event time itself is inferred (as do(a, 9)) • generally, tenses will be viewed as goals to be satisfied in the current discourse model • this can also be seen as an update process: first the (empty) discourse model is updated with holds(r, 10), and then, after CWR, with do(a, 9) • the discourse model for ‘Bill has turned on the light’ has elements {bill, a, r} and relations holds(r, 10), do(a, 9) A different computation • what we’ve just seen is, logically speaking, the computation of the minimal fixpoint of a logic program • the syntactic counterpart of this computation is backtracking from a given goal • ?holds(r, 10), ¬holds(r, 1), . . . , ¬holds(r, 9) reduces to • ?n < 10, ¬holds(r, n), do(a, n), ¬ab(n, 10), ¬holds(r, 1), . . . , ¬holds(r • the only unification that leads to success with this goal is n = 9; e.g. n = 1 leads to • ?do(a, 1), ¬ab(1, 10), ¬holds(r, 1), . . . , ¬holds(r, 9) • negation as failure reduces this goal to • ?do(a, 1), ¬holds(r, 1), . . . , ¬holds(r, 9) • but one cannot make ¬holds(r, 2) succeed, i.e. one cannot show that the derivation from the goal ?holds(r, 2) fails – indeed this derivation succeeds • the same argument works for all n ≤ 8 Language as planned action: the bigger picture • preceding considerations will lead to detailed computational model of tense/aspect processing, with predictions for brain imaging (see lecture 4) • the claim is that the computations identified are what is ‘really’ going on – at least ‘really’ in the sense of Marr’s informational and algorithmic level • there is also a suggestion for a biologically plausible neural implementation: discourse models as stable states (lectures 4, 5) • language is (externalised!) planned action in the sense that (i) discourse production results in a series of goals to be satisfied by a plan, and (ii) discourse comprehension involves the computation of a plan, which can be identified with a discourse model • importance of flexibility of planning in organising discourse – clue to origin (evodevo; HCRS)