I’ve commented making this point before, but this seems like a particularly clear example, so I’ll give it another go. Hopefully this isn’t too repetitive:
Claude is indeed impressive, but a skeptical reading of the transcript shows a problem:
> Based on the style and content of this opinion, I believe the likely author is Justice Elena Kagan. Here’s my reasoning:
We can stop right there. An LLM picks words sequentially that continue the pattern of whatever text came before. Whatever reasoning follows, however good it sounds, it’s *not* an explanation of why Claude picked Kagan, because that already happened. The causation goes the opposite way. It’s an invented justification that sounds like how a human would write it, even though it’s not human. It invents human-sounding justifications because that’s what it’s trained to do.
But that doesn’t reveal how it thinks. The real reasons have to do with inscrutable mathematical circuits that researchers are only starting to untangle, but we do know that they only go forward. To have any hope of human-sounding reasons influencing that circuitry, you have to tell it to write out its deductions *first,* then choose an answer.
Of course, people make post-hoc rationalizations too. But a human writer will think things through as they write, and they might change their mind about something and go back and revise. That’s simply not on option for Claude. It has to continue the pattern and it will do the best it can. After coming up with the first reason, it will keep inventing one more reason to continue the list. In theory, it could admit to seeing a mistake, but that’s unlikely when it breaks the pattern, because LLM’s like to continue patterns.
(I once got GPT-4 stuck in a different pattern. I pointed out a mistake, it apologized, it tried again, it corrected it again, and then it got stuck in a loop, making suggestions and immediately apologizing. You could interpret that as some kind of mental collapse, but really, it was continuing a pattern it detected in the transcript.)
All we really know is that Claude got four out of four right. That’s pretty good! But we don’t know why. It’s common for AI’s to learn shallow signals that happen to give the right answer. (It could be like Clever Hans.) The risk is that people might be fooled by persuasive justifications into thinking they understand how an LLM “thinks” and go along with its decisions, when actually they have no idea how it thinks.
If you use an LLM as a source of clues, an idea generator, and autocomplete for your own writing, it’s less risky. Asking Claude for a list of ten good justifications is fine. But hopefully you read them over and only keep the ones you personally find persuasive. And check them, if it’s the sort of thing that can be checked. In this way, you’re making them your own and taking responsibility for the result. They’re good reasons because you think they’re good, not because the machine generated them.
I'm an Israeli lawyer, came to this substack from Youtube video it was mentioned in and just registered here in order to write this comment.
1. One of recurring questions in my legal blog is "when do we get AI judges"?
My answer is - each time an LLM generates an answer to the specific prompt you get a different one (if temperature > 0), so LLM judge will generate a well-nuanced answer, but each time different one and basically it's gonna be a lottery. Probably solvable with 100+ generations and then selecting the "majority opinion"?
2. Amazing work by Claude - probabilistic approach instead of binary one.
It's my old dream, to evaluate evidence not on binary terms but rather attaching probability to witness testimony / facts / expert opinions etc. This would enhance the precision of final result - let's say in a criminal case two lead prosecution witnesses score 95% credibility, expert opinion 87% credibility, fact A is 93% probable, fact B is 82% probable. In case of binary definitions all of them are TRUE, as a result we've got a guilty verdict, but if we multiply the probabilities we get more than 5% probability of innocence and must acquit.
P.S. I envy your comments section, almost no one tells you that you don't know law, no ad hominem attacks... You're so lucky :)
P.P.S. - a successful prompt engineering attack is still a strong possibility, thus having AI judges has inherent risk of actual jailbreaks (pun intended)
Thanks for introducing us to Claude. I’ve had lots of great conversations with him on everything from the nature of freedom to whether the future is that into which the universe expands. But before we get too excited, there’s the question of prose: who would you rather read, Claude or Adam? I don’t think Adam has to worry about his job.
Computer programming data analysis has a saying "garbage in garbage in". Compared to unfiltered internet opinions, legal opinions are decided on the well-thought-out side of the spectrum. Ironically that makes it easier for AI to learn its nuances and replace parts of it.
How long until we give the defendants the option to trial by AI, summary decision after both sides submits evidence and testimony? Add a sentence discount for court "efficiency".
I for one will continue to refer to Mr. Unikowsky as ‘Prof.’, for two reasons. One is that he informs and educates those of us who read his posts. Why he takes what must be a tremendous amount of time to do this is not clear but I am not one to look the proverbial gift horse in the mouth or any other part of his body. Two, his writing is clear and concise and informative and entertaining, such as the writings of an excellent law professor. I hope he will not mind.
So with respect to his thoughts and review of the uses of A. I. in law, his earlier post on the subject was about the ability of A. I. to predict Court outcomes, an interesting situation but one whose value I questioned. But in this post he illustrates the tremendous potential of A. I. in allowing a law practitioner to analyze common and statutory and law to assist in the preparation of legal argument and to understand the legal system and is all-too-human participants. This is where the A. I. revolution will take place.
Before getting into the A. I. discussion though, consider this statement from Prof. Unikowsky.
“Roberts was decided by a 6-3 vote, with the three most progressive Justices (Brennan, Marshall, and Stevens) dissenting. And you can see why a certain kind of judicial conservative would support Roberts’ test. Conservatives are supposed to be tough on crime. If the testimony seems reliable enough, why let the defendant, who is probably guilty, get off on a technicality?”
This certain kind of ‘judicial conservative’ is not a conservative at all. And this is the problem with today’s political environment, radicals who defy the rule of law, the rules of society and yet still call themselves conservatives. They are not. In many cases they are the very opposite of conservatism, and the damage they are doing in the name of conservatism is tremendous.
The post also does something else that is as informing as it is irritating. In his discussion of the Confrontation Clause Prof. Unikowsky provides an example in which the late Justice Scalia sounds rational and intelligent, something persons of my persuasion find disturbing and upsetting. We are made to face the fact that Justice Scalia was not all bad all of the time, which leaves us very uncomfortable.
Prof. Unikowsky illustrates how A. I., in its analysis can lead to better legal arguments and ultimately better outcomes, better outcomes meaning decisions grounded in law as opposed to the biases of the Judges and Justices. But everyone should recognize the limitations. For example, suppose one is scheduled to argue a case before the Supreme Court and asks the A. I.
“How can I tailor my arguments in a way that maximizes the likelihood that Justice Thomas would accept my position?”
If A. I. is as good as Prof. Unikowsky has demonstrated, the likely answer by A. I. is
“There is no way to do that. Based on my review of every opinion, speech and writing of Justice Thomas he decides on his position based on his personal politics and his personal preferences. There is no argument that would move him away from his pre-determined position.”
If anyone objects to this, feel free to substitute ‘Justice Alito’ for Justice Thomas in the above although it would be interesting to see if the A. I. could correctly predict Justice Alito’s failure to dissent in the Rahimi case.
Justice Breyer gives the Defendant-Centric Confrontation” (DCC) Test his full approval. Justice Gorsuch is furiously penning an emotionally persuasive dissent.
Really astonishing to this non-lawyer. Thanks so much for this work.
God I love evidence law.
I’ve commented making this point before, but this seems like a particularly clear example, so I’ll give it another go. Hopefully this isn’t too repetitive:
Claude is indeed impressive, but a skeptical reading of the transcript shows a problem:
> Based on the style and content of this opinion, I believe the likely author is Justice Elena Kagan. Here’s my reasoning:
We can stop right there. An LLM picks words sequentially that continue the pattern of whatever text came before. Whatever reasoning follows, however good it sounds, it’s *not* an explanation of why Claude picked Kagan, because that already happened. The causation goes the opposite way. It’s an invented justification that sounds like how a human would write it, even though it’s not human. It invents human-sounding justifications because that’s what it’s trained to do.
But that doesn’t reveal how it thinks. The real reasons have to do with inscrutable mathematical circuits that researchers are only starting to untangle, but we do know that they only go forward. To have any hope of human-sounding reasons influencing that circuitry, you have to tell it to write out its deductions *first,* then choose an answer.
Of course, people make post-hoc rationalizations too. But a human writer will think things through as they write, and they might change their mind about something and go back and revise. That’s simply not on option for Claude. It has to continue the pattern and it will do the best it can. After coming up with the first reason, it will keep inventing one more reason to continue the list. In theory, it could admit to seeing a mistake, but that’s unlikely when it breaks the pattern, because LLM’s like to continue patterns.
(I once got GPT-4 stuck in a different pattern. I pointed out a mistake, it apologized, it tried again, it corrected it again, and then it got stuck in a loop, making suggestions and immediately apologizing. You could interpret that as some kind of mental collapse, but really, it was continuing a pattern it detected in the transcript.)
All we really know is that Claude got four out of four right. That’s pretty good! But we don’t know why. It’s common for AI’s to learn shallow signals that happen to give the right answer. (It could be like Clever Hans.) The risk is that people might be fooled by persuasive justifications into thinking they understand how an LLM “thinks” and go along with its decisions, when actually they have no idea how it thinks.
If you use an LLM as a source of clues, an idea generator, and autocomplete for your own writing, it’s less risky. Asking Claude for a list of ten good justifications is fine. But hopefully you read them over and only keep the ones you personally find persuasive. And check them, if it’s the sort of thing that can be checked. In this way, you’re making them your own and taking responsibility for the result. They’re good reasons because you think they’re good, not because the machine generated them.
I'm an Israeli lawyer, came to this substack from Youtube video it was mentioned in and just registered here in order to write this comment.
1. One of recurring questions in my legal blog is "when do we get AI judges"?
My answer is - each time an LLM generates an answer to the specific prompt you get a different one (if temperature > 0), so LLM judge will generate a well-nuanced answer, but each time different one and basically it's gonna be a lottery. Probably solvable with 100+ generations and then selecting the "majority opinion"?
2. Amazing work by Claude - probabilistic approach instead of binary one.
It's my old dream, to evaluate evidence not on binary terms but rather attaching probability to witness testimony / facts / expert opinions etc. This would enhance the precision of final result - let's say in a criminal case two lead prosecution witnesses score 95% credibility, expert opinion 87% credibility, fact A is 93% probable, fact B is 82% probable. In case of binary definitions all of them are TRUE, as a result we've got a guilty verdict, but if we multiply the probabilities we get more than 5% probability of innocence and must acquit.
P.S. I envy your comments section, almost no one tells you that you don't know law, no ad hominem attacks... You're so lucky :)
P.P.S. - a successful prompt engineering attack is still a strong possibility, thus having AI judges has inherent risk of actual jailbreaks (pun intended)
Thanks for introducing us to Claude. I’ve had lots of great conversations with him on everything from the nature of freedom to whether the future is that into which the universe expands. But before we get too excited, there’s the question of prose: who would you rather read, Claude or Adam? I don’t think Adam has to worry about his job.
Wow. The Supreme Court can be replaced.
Computer programming data analysis has a saying "garbage in garbage in". Compared to unfiltered internet opinions, legal opinions are decided on the well-thought-out side of the spectrum. Ironically that makes it easier for AI to learn its nuances and replace parts of it.
How long until we give the defendants the option to trial by AI, summary decision after both sides submits evidence and testimony? Add a sentence discount for court "efficiency".
I for one will continue to refer to Mr. Unikowsky as ‘Prof.’, for two reasons. One is that he informs and educates those of us who read his posts. Why he takes what must be a tremendous amount of time to do this is not clear but I am not one to look the proverbial gift horse in the mouth or any other part of his body. Two, his writing is clear and concise and informative and entertaining, such as the writings of an excellent law professor. I hope he will not mind.
So with respect to his thoughts and review of the uses of A. I. in law, his earlier post on the subject was about the ability of A. I. to predict Court outcomes, an interesting situation but one whose value I questioned. But in this post he illustrates the tremendous potential of A. I. in allowing a law practitioner to analyze common and statutory and law to assist in the preparation of legal argument and to understand the legal system and is all-too-human participants. This is where the A. I. revolution will take place.
Before getting into the A. I. discussion though, consider this statement from Prof. Unikowsky.
“Roberts was decided by a 6-3 vote, with the three most progressive Justices (Brennan, Marshall, and Stevens) dissenting. And you can see why a certain kind of judicial conservative would support Roberts’ test. Conservatives are supposed to be tough on crime. If the testimony seems reliable enough, why let the defendant, who is probably guilty, get off on a technicality?”
This certain kind of ‘judicial conservative’ is not a conservative at all. And this is the problem with today’s political environment, radicals who defy the rule of law, the rules of society and yet still call themselves conservatives. They are not. In many cases they are the very opposite of conservatism, and the damage they are doing in the name of conservatism is tremendous.
The post also does something else that is as informing as it is irritating. In his discussion of the Confrontation Clause Prof. Unikowsky provides an example in which the late Justice Scalia sounds rational and intelligent, something persons of my persuasion find disturbing and upsetting. We are made to face the fact that Justice Scalia was not all bad all of the time, which leaves us very uncomfortable.
Prof. Unikowsky illustrates how A. I., in its analysis can lead to better legal arguments and ultimately better outcomes, better outcomes meaning decisions grounded in law as opposed to the biases of the Judges and Justices. But everyone should recognize the limitations. For example, suppose one is scheduled to argue a case before the Supreme Court and asks the A. I.
“How can I tailor my arguments in a way that maximizes the likelihood that Justice Thomas would accept my position?”
If A. I. is as good as Prof. Unikowsky has demonstrated, the likely answer by A. I. is
“There is no way to do that. Based on my review of every opinion, speech and writing of Justice Thomas he decides on his position based on his personal politics and his personal preferences. There is no argument that would move him away from his pre-determined position.”
If anyone objects to this, feel free to substitute ‘Justice Alito’ for Justice Thomas in the above although it would be interesting to see if the A. I. could correctly predict Justice Alito’s failure to dissent in the Rahimi case.
Justice Breyer gives the Defendant-Centric Confrontation” (DCC) Test his full approval. Justice Gorsuch is furiously penning an emotionally persuasive dissent.