Symposium on Artificial Intelligence and Judicial Proof
Benjamin N. Cardozo School of Law
55 Fifth Avenue, New York, New York
Sunday, April 30, 2000
Can AI Help Resolve Some Fundamental Puzzles of Judicial Proof?: Introductory Comments about the "Explosive Dynamic Complexity" of Evidentiary Processes associated with Litigation
by Peter Tillers
Two or three years ago I began searching the internet in earnest for insights into evidence in litigation. (I admit it: I am an internet addict!) I quickly discovered that there is a large number of people in AI and in related fields who are actively discussing, debating, and exploring matters such as uncertain inference; decision making under uncertainty; classical probability theory; statistical inference; value of information theory; semantic fuzziness and roughness; planning and scheduling; distributed decision making; non-deductive reasoning; induction, abductive inference, and hypothesis-formation; information retrieval & database mining; multi-stage or cascaded inference; influence diagrams; causal reasoning; common sense reasoning; defeasible reasoning; temporal logic; dynamic logic or reasoning; and a wide variety of even more esoteric matters -- matters such as artificial life and a mathematical theory of hints.
I discovered another interesting thing. I discovered that many of theories, methods, and procedures in the sorts of fields or areas I have just mentioned have been converted into software programs. I also found out that many people who think of themselves as AI people also seem to think of themselves as Bayesians, fuzzy probability theorists, and other such things; I discovered that many people who do AI have done important work with matters such as Bayes nets, common sense reasoning, defeasible reasoning, dialogic reasoning, default logic(s), fuzzy sets, and so on -- and on -- and on.
After watching Deep Thought defeat Kasparov, I decided that it was high time that legal professionals who study or manage evidentiary processes in litigation learn something about developments in AI and related fields; I concluded that it was high time that people in AI and related fields educate legal specialists in forensic investigation, evidence, and proof about relevant developments in AI and allied fields.
I believe that AI people very probably do have interesting and important things to tell people such as me. I believe this because as I scan the sorts of matters that AI people grapple with I see that many of those matters involve uncertainty, complexity, and time. I believe that some of the most intractable mysteries about the workings of evidentiary processes in litigation and the effects of societal regulation of such processes have to do with the combination of uncertainty, complexity, and time. These ingredients, when combined with the ingredient of human imagination, make it almost impossibly hard for mere mortals to understand how judicial proof works.
I have a friend who is fixated on the enormous complexity of the day-to-day inferential activities that ordinary human beings perform. If we view an analytic schema such as Schum's theory of cascaded inference as an account not only of how inferential tasks might be performed but also of how they are actually performed, there is no doubt that my friend is entirely right in believing that ordinary inference is extraordinarily complex affair. Even a very small investigation or trial generates an extraordinary large mass of evidentiary detail and, almost regardless of the logic one chooses to use to analyze it, assessment of such large masses of evidentiary details has to be a very complex enterprise, one that involves a very large number of calculations and assessments. Consider, for example, the following directed acyclic graph -- in this instance, a Bayes' net -- depicting the network of inferences generated by just a minute portion of the evidence in the Sacco & Vanzetti case:
Joseph B. Kadane & David Schum, A PROBABILISTIC ANALYSIS OF THE SACCO AND VANZETTI EVIDENCE 217 (John Wiley & Sons, Inc., 1996).
Things get worse -- i.e., matters get still more complicated -- if one thinks that the enterprise of judicial proof involves more than just a web of catenated inferences. If forensic investigation and proof also involve and are affected by constructs such as scenarios, legal rules and argument, and argument about the credibility of human sources, the enterprise of judicial proof becomes even more complex. The following diagram drawn by Dave Schum gives you a sense of the variety of marshaling operations that participants in the enterprise of judicial proof might feel that they should engage in:
As complex as the task of assessment in this sort of envisioned situation might be, the task may not yet stupefy or dumbfound most legal professionals who study or work with evidence and inference in the legal process; many legal professionals seem to have a good sense of how to do their job in a fairly orderly way when they are called upon to assess the extent to which a particular body of evidence supports a story that warrants legal relief.
Matters may stand quite differently, however, when time and one other matter are taken into consideration.
By referring here to time I am not now alluding to the quite correct premise that some of the propositions which a decision maker must assess are propositions about temporal events or sequences of events in time. I am referring to the simple fact that any legal decision maker who is called upon to assess the significance of a body of evidence is himself/herself/itself mutable and that the environment in which any such decision maker operates is also mutable.
Mutability superimposes a layer of complexity onto the layers of complexity to which I have already alluded. However, even this additional layer of complexity may not account for the sort of intellectual stupefaction that I have sometimes experienced when confronting problems of evidence of evidence and proof in the legal process.
I think it is only when one adds one more ingredient that one is likely to have the feeling that the process of judicial proof is so impenetrable that no mere mortal will ever understand it, that the workings of judicial proof are simply virtually impervious to any kind of orderly analysis. What is the additional ingredient that might have the capacity to produce such a fiendish state of affairs? It is, I suspect, human imagination!
Allow me to use some diagrams to illustrate the sort of baffling problems of evidence and proof that I have in mind. Before I begin with this, however, I want to apologize for the crudeness of the diagrams I am about to discuss.
But, no! To be completely honest, I am not really ashamed of my diagrams. If I knew how to draw elegant graphs and diagrams to explain precisely why I am puzzled by the matters that puzzle me, I wouldn't be participating in this conference. I am here because I think you may be able to tell me how to (literally) picture more clearly the sorts of situations that puzzle and trouble me.
Please consider Figure 1.
Now please consider Figure 2:
Now consider Figure 3:
Figure 4:
Figure 5:
Now consider Figure 6:
Now consider the following development:
Something very interesting and important has happened in the situation shown above. A new legal rule LR-2 -- has entered the mind of the decision maker. But it did not do this on its own, as it were. It entered the mind of the decision maker because the decision maker, seeing evidence E-2, imagined the possibility that legal rule LR-2 might come into play.
The next two figures depict some further possible workings of the decision maker's constructive imagination:
Concluding Observations
As I told you when I began this little peroration, I have a friend who is greatly impressed by the complexity of the inferential problems that ordinary people face in their day-to-day lives.
I agree that the inferential problems that ordinary people face in their daily lives are enormously complex.
But the insight that ordinary human beings face complex inferential problems tells only a half or a third of the whole story.
While I am impressed with the complexity of the inferential tasks that ordinary people face, I am also impressed by the ability that ordinary people have to deal with many of the extraordinarily complex inferential problems that they routinely face.
I am also impressed by the ability that human beings have to devise artificial procedures and tools for dealing with very complex and difficult problems and tasks.
If all I had to tell you today were that evidence in litigation generates very complex inferential problems, you might as well pack up your bags and go home.
(Shamelessly stealing from D. Schum) I have tried to suggest that the granularity of evidence is part of the reason for the complexity and difficulty of problems of forensic proof.
I have tried to suggest that the multifariousness of patterns of human thinking about evidence is another reason for the difficulty of problems of forensic proof.
I have further tried to suggest that the mutability and changeability of decision makers and their psyches and of the environments in which they operate is another factor that contributes to the complexity and difficulty of which I speak.
Finally, I have tried to illustrate why I suspect that human imagination and creativity are perhaps another very important reason for seeming mysteriousness of some of the workings of the process of judicial proof.
I suspect that you realize that some of the problems of evidence and inference that I have described for you have a semi-autobiographical tinge. Hence, I have already suggested that when I was a litigator I thought that I could manage many complexities of problems of evidence reasonably well. For example, I thought that I could do a reasonably good job of marshaling large collections of evidence around the elements of the legal rules that I knew or believed were likely to play an important role in the ultimate adjudication of the case by a judge or jury. Things got a bit more complicated and troublesome, I found, when the legal theories and the evidence in the case wouldn't stand still, when these things changed, as they almost always did. But I found that for the most part I could adequately manage both such change and the prospect of such change. But only on one condition. I could manage reasonably well only when and only if I felt I could make reasonably good predictions about the changes that the collections of evidence and the legal theories in the case would undergo. The trouble was that in many situations I felt largely helpless because I felt I could not make such predictions very well.
In any event -- regardless of what may be the precise source of the difficulty that we may have when trying to predict the consequences of discovering or generating new evidentiary matter (and, in general, of discovering or generating new information and knowledge) --, you see, don't you?, why the state of radical ignorance that I am describing might be a very important problem for an actor such as a trial lawyer. A person such as that might have a very hard time making considered decisions about future investigative and proof activity if (i) he believes that anything he might do -- even a seemingly trivial investigative or proof action, for example -- might cause profound changes in the entire structure of the case or dispute in which he is involved; and (ii) he also believes that, alas!, he has little or no ability to predict what those changes might be.
And if this is the situation that confronts trial lawyers, society may confront an even more parlous dilemma. For if the workings of the litigation & proof process are in part a result of the way that lawyers work, but if no one can predict the likely effects on lawyerly investigation and proof if lawyers work or act one way rather than another, it will be very hard for anyone to make any reasoned arguments for one type of societal regulation of the forensic proof process rather than another. If so, it may follow that the design of the entire litigation & proof process is nothing more than a result of an elaborate game of blind man's buff; it is possible that no one really has any good way of guessing what are less optimal and what are more optimal ways of regulating the process of litigation and proof.
I hope that you now see why I think lawyers, judges, and law teachers need a lot of help from the likes of people in AI and related fields.
