Decision systems, such as Jev, as well as the recently launched OpenAI Decisions API, herald a new, but also curiously déjà vu, way of classifying legal information. The approach can work via open-source models and also deliver huge cost savings in areas such as eDiscovery, or first pass contract review.
In this in-depth AL TV Interview, Len Hickey, CEO of Litigaze, and Artificial Lawyer, first explore the potential impact of these light and agile decision systems, and then see how Litigaze is using the technology for eDiscovery needs.
There is a transcript of our discussion below. For AL, one of the key points about decision systems is that law firms, or inhouse teams, could spin up dozens of such applications across multiple practice areas in order to provide fast and efficient responses for very specific use cases, rather than relying on a single frontier LLM that is often ‘overkill’ for many of the needs that lawyers have.
Moreover, as Hickey notes, if you go the open-source route you can post-train these decision systems within your firm to drive higher accuracy, and customise them for very specific use cases. Plus, although these are still probabilistic tools, they always show % estimates for their accuracy, which helps with explainability – which you won’t get with a general LLM approach. And that matters in areas such as eDiscovery especially.
For AL, it’s a bit like a new version of some of the earlier no-code, workflow automation tools, which we saw before the genAI era arrived. I.e. you can create an application with a very narrow focus, which is light and inexpensive to run. And there’s also an echo of the older NLP / ML systems that initially were used for fairly narrow text searches as well.
Here we look at an eDiscovery use case, but it could also work for any legal need where it’s useful to segment, classify, or label text in documents, or sort entire documents. Naturally, this is not for complex language understanding, but lawyers don’t always require Astra going at full blast. Sometimes a decision system is exactly what you need.
Richard Tromans (01:42.296)
If you’ve been a longtime reader of Artificial Lawyer you’ve probably realised that Litigaze has evolved over the years. It’s now in its latest iteration, which is focused on eDiscovery. So let’s look at decision systems.
What they do is they’ll generate specific answers to questions you put to them. So a really simple case, and I think I said this in the article we did recently was you might have an email that’s come in and you you you’ve got it and you want to know if it’s spam or not. And so it might have some signs of that in the way the text is written. And so I might ask an LLM, have a look at this email content, what do you think is it spam? And the LLM is very smart. You know the modern ones will very quickly be able to tell me or relatively quickly be able to tell me what they think.
and they’ll come back and say, yes, Elaine, I think that email probably spam. The decision model is different. So what I’ll do is I’ll do the same thing. I’ll say to the decision model, here’s some text from an email. read it, and then I want you to tell me with a probability, yes or no, is it spam? And the decision model will then take that and very quickly, and this is one of the powers of it, very quickly, respond to me with yes and no, and each of those having a probability. So it never talks to me. It’s it’s a sort of in a way a lot less human than the LLMs have become.
But what it’s doing is generating these decisions is what some people are calling, but they’re really classifications or or assessments of the text that I’ve given it.
Richard Tromans (04:58.517)
Yeah. Yeah, it’s it’s interesting. I mean, the open AI example was interesting because I mean w one thing they were s they were from what I can understand, they were saying that i i it was to some degree based off one of their L LMs, but they were not using the whole of it. They were using just part of that capability. but the other thing as well, they were going on and on and on about robotics.
And you know, that’s the fun I mean I guess that’s always been one of the things with AI. In the legal world we tend to think of everything in terms of how it applies to text and meaning. but it’s so, so you know, multi-system really. you know, if you can you can classify images, you can classify music, you can classify video, all kinds of different things. but in this case, obviously what we’re interested in is documents, primarily with text.
and obviously classification, what does that apply to? Well, there’s all kinds of different jobs. As I was, you know, mentioned in that open AR OpenAI decisions article eDiscovery, which Len is working on. MA due diligence, maybe not for super details, not for what does this clause buried in page 5000 of this mega mega contract from 25 years ago mean? That might be way beyond. But if you could just say, Is there a change of control clause? Then
pull out all the change of control clauses and say which ones involve Egypt. Right, you know. like none of them. But anyway. But you know, you could do that. because you can just use simple slices. Very in a funny way, it’s almost like going back to the early days of machine learning.
Len Hickey (06:38.121)
Yeah. And so what I guess the clever bit, and this is what Jev did with their model release, which is a lot of people are just calling these JEV models now because that’s the first one that came out last month. But in a way, what they did is they took this idea of, yeah, an older concept of classifying stuff using machine learning, but they just based it on an LLM that already comes pre-built with a whole lot of smarts. So it really is just you can think of it like a generalized classifier. You don’t have to train it. You can train it. And I can talk to you about that as well, because that’s exciting too.
But at its base, it’s got like an underlying model. it might be a four billion parameter model, a 27 billion parameter model, there’s different sizes. But the the that brings it sort of pre-packaged with a whole lot of smarts about basic classification. So a spam email one, it’s gonna do very well on out of the box because you know, spam’s sort of so unfortunately ubiquitous, I guess, and it’s well trained. it might differ a bit on specialized areas, and that’s something we’re working on hard with our e-discovery sort of privilege assessments.
but yeah, by and large, they’re pretty smart out of the box because they have that same cool LLM tech that’s built in.
Richard Tromans (07:43.03)
Sure. And well, okay, two bits. First, let’s just cover cost. One reason why law firms and and in-house counsel might be interested in this if cost is an issue is because it is a far less intensive model, you can get a lot out for very little. I mean I I showed again in artificial law over the little table there that shows the comparative difference between the latest frontier models and what even OpenAI’s own decision system costs.
And then I Jev is even cheaper again, right? and I guess if you knocked out your own open source system, which you you know, which you’ve been built on top of, it could be even cheaper. You could be using Quen or something. I don’t know, you could be using something even cheaper, and on and on it goes. is that from what you’ve seen so far, do is that is that an area that interests for lawyers?
Len Hickey (08:37.809)
well, it’s huge for us, right? Because we’ve done the last of those options you said. So we’ve taken an open source model and brought that into our infrastructure. And the main reason for that is actually so we don’t have to send external API calls. It all stays within our AWS infra. the second reason though is because these models are bit smaller than like these massive frontier models. So you can host them quite effectively on modest GPUs and it doesn’t, you know, still costs a bit because NVIDIA is expensive, but it’s a lot less cheaper and manageable as a
a software company to bring this stuff in-house and and manage it. And and yeah, what that means in terms of the the user experience for our product at least, and I’m sure this is going to happen across the board soon enough, but we’re getting ahead, is what we were using LLMs to do on these volume types of cases, like even in MA or any situation we’ve got a lot of documents, you could still send an LL let an LLM loose on them. and it would do pretty well, right? They’re they’re they’re smart things, right? So they’re gonna do very well at
making calls about documents. The problem is it’s just a lot for what you’re asking of it, right? You if you’re not doing sort of deep analysis like the example you mentioned, you’re just saying, is it Egypt? Is it not? These decision models are much, much better because they do very well at those pinpoint questions and they’re cheap. and so from just again to bring it back to what I know because it’s our product is what we were doing with eDiscovery is charging people who want to do this LLM analysis on 100,000 documents, right? It’s too expensive for us to say
You know, you can do that for free. Now we can. So now we can say to them, yeah, you can do this analysis on a hundred thousand documents. And because we’re managing the infra, you know, we can obviously do clever stuff internally to make that commercially viable for us. But then clients don’t have to pay for it at all. in our case. Now I’m not sure other vendors will provide the same option, but we just think it’s better. It’s such a cool technology, such an important game changing technology that we think it should be something that should be widely adopted therefore we’re just we’re just offering it for free.
Richard Tromans (10:37.323)
And what about training then? So like if you’re using an open source, relatively small language model, if you want to call it that, although people s people seem to have stopped saying small language model. Now everything is an LLM, it’s just an LLM as open source, or an LM is not open source. But anyway, using an open source system, relatively small to medium sized one, how much realistically can you train it yourself on the So like you let’s say let’s say you’re a litigation firm and you
end up again and again and again looking for the same kind of, I don’t know, privilege type thing, right? You’re always there, you know, that’s one of the first things you do, you know, with every matter. And you just say, why can’t we just build something really cheap and fast just to get us going? Let’s say that you didn’t exist. If they wanted to build that themselves, could they do it?
Len Hickey (11:29.167)
y yeah. I think they could. I mean, they’d need to do a it’s it it’s a there’s a process to go through and do it, you know, intelligently, but I I think it’s very achievable. I think the other thing to say is that what I find really exciting, so yes, in answer your question, but this then brings in another cool idea, which is, you know, law firms at the moment are grappling with sort of knowledge leakage from the firm and you know, the fact they’re you know Yeah, it’s absolutely and this is like si a serious way to start you could train a hundred of these models to do specific
Richard Tromans (11:50.709)
AI s the whole AI sovereignty bit, yeah. Yeah.
Len Hickey (11:59.106)
you know, aspects of your depending on what sort of practice areas you’re in. So, you know, privilege is a common one and we have, but we’re we’re looking at seriously the idea of rolling these things out in a very obviously deliberate way, but you know, to firms to customize themselves. So we’re we’re not quite there with this next I’m sort of getting ahead of ourselves in the product development, but our next phase is going to be allowing firms to train models and they can keep them. Yeah.
Richard Tromans (12:03.231)
Yeah.
Richard Tromans (12:18.999)
Yeah.
Richard Tromans (12:22.423)
Well this is the thing, isn’t it? Yeah it really it it’s really funny. It reminds me it’s so funny how things go in circles. It’s like this is a lot like the original rules based decision systems to some degree. You know, people like Eubriter and Yota Logic, in their first iterations, obviously they’ve evolved since then. But you could have you know, they’re you know, they’re these companies were hoping that law firms, every different little department and sub department would use a ton of a systems, right?
decision trees and so forth to sort information. But you could effectively do the same with with with these decision models, couldn’t you? You could effectively say, right, you know, the M<unk>A team, you have 15 of these. Each one is focused on a particular area that you need to to sort. And you could have across an entire law firm, you could have dozens upon dozens of them.
Len Hickey (13:13.395)
Hm. And I think I think that’s yeah.
Richard Tromans (13:15.339)
Which which they but I was gonna just I think it’s just it it then removes the need to rely on one of these massive astralite models, right? Because for a lot of a work that a lot of law firms want to do, they they want to do the reasoning themselves anyway, right? They want to do the judgment bit themselves anyway. So if you if the purpose of using AI is just to like slice and dice and like put all your different colored l you know, red, yellow red Lego bricks in this box, green Lego bricks in that box.
If that’s all you want to do, this is good, right?
Len Hickey (13:48.342)
A hundred percent. I mean it i it y like I was saying sort of earlier, an LLM can do it, but it’s just like, you know, way, way overkill for the task and and way more expensive and and slower. So I think that’s right. Yeah. The idea is, and why they are these models are even better, I think, than those sort of older decision systems where you’d have to very very explicitly define, you know, nodes and branches. This gives you a flexibility because it’s an LLM built in. So you define a node and a branch in a way that gives the the the
decision model power to sort of make calls. So you’re not defining every option or every spin on a word, you know, what this spam email might contain by keywords. It’s more just feed it in and the thing’s smart enough with the right training and right calibration to give you very reliable outputs. So yeah, I think it is a bit of that old tech, but brought into LLM and maybe both of them coming together.
Richard Tromans (14:39.797)
Yeah, yeah, it’s really amazing actually how everything comes back.