More common sense from SLAW. They should literally legislate globally that only SLAW writers can comment on AI!
What many of us in law, legal education, and other fields still want to know at this point is: what is AI really good for? What does it do reliably well and better than we could do on our own? And when we use it for those purposes, what risks do we take on?
In the early days of ChatGPT, those risks were clear. AI hallucinated authorities and generated biased output grounded in its training data. But as models have improved and we’ve learned to guard against these problems, those concerns have become more manageable.
A different and more subtle issue has now come into view.
Having discovered some of the things AI is good at — supporting research, drafting, and editing — the main concern is not just whether its output is accurate, but when effective use of the tool crosses the line into harmful over-reliance.
When a lawyer or a self-represented litigant cites cases that don’t exist, they aren’t over-relying on AI. They’re misusing it. Over-reliance entails something else. It overlaps with automation bias — the tendency to defer uncritically to a system’s output — but is not reducible to it.
We over-rely on AI not just when we accept its output as true without question, but when we allow it to perform work we shouldn’t be delegating to it at all — even if it’s work that AI can do well.
But precisely what should we not be delegating to AI? Here, we’re in new terrain.
For certain forms of writing — a personal email, an essay, a court decision — most of us have a strong intuition that relying on AI to do the drafting is wrong, even if the result is fluent and technically sound. These forms of writing are tied to deeply seated ideas about identity and reflection. Automated prose, however polished, leaves us cold. It may be correct but it’s inhuman.
Yet in many cases there’s nothing wrong with relying on AI. Using it to transcribe an interview or summarize a case on CanLII to decide whether it’s worth reading closely can sometimes feel magical.
The trouble that many experienced users of AI are now encountering is that as these tools become more capable and we become more adept at using them, it becomes easier to slide into patterns of increasing delegation. And the more we do so, the more AI begins to encroach on doing the critical things we should be doing ourselves.
It becomes tempting, for example, in the course of a chat with AI to let it carry you from a brainstorm to an outline to a first draft, because it all happens so fast. The model can seem uncannily in sync with where you want to go. Prompts often end with suggestions for next steps, making it feel as though the system is always a step or two ahead of you. It can be hard to resist letting it take the lead.
Read More
