Qudoc the new beta Australian legal materials search engine? informed us earlier this week that Austlii would not allow them to spider some of their database. To get to the nitty gritty of the issue we contacted Graham Greenleaf at Austlii to clear up what AustlII could and could not allow.
HOB asked Greenleaf to let us know what parts of AustLII Qudoc would be allowed to spider and which information they wouldn’t be allowed access to.
Greenleaf indicated that AustLII had no objection to the spidering of “their” legislation and law journal articles for the purpose of making them searchable through a search engine and says?That appears to be what Qudoc have done, and AustLII has no problem with it. In fact, they are welcome to do so.
He also added
However, Andrew and Philip pointed out to them that AustLII does not allow web spidering of its case law, by anyone. This is primarily in order to balance privacy protection against open justice. AustLII’s robot exclusion has excluded spidering of case law since AustLII was started.
AustLII enforces this policy. If you check Australian courts or tribunals that publish their own cases, you will also find that they also block spiders. AustLII must act consistently with their policies.
As a result, you cannot find Australian case law from legal publishers or these courts via search engines, including Google. If Google does ever make cases searchable by mistake, AustLII and the courts receive high volumes of complaints until these cases are removed from Google’s search data.
Qudoc seems to have observed AustLII’s exclusion of case law from spidering, so we have no issue with them.
Google also observes this policy (except for the occasional error which it remedies), and so do other search engines.
So it looks as though Australian courts and tribunals may need pressure applied over time to make them more amenable to having their published material available to a wider selection of users via engines such as Qudoc.
Both Austlii and Qudoc operate within the parameters set by both the courts and legal publishers who are, we?d suggest, protecting revenue streams rather than thinking about allowing a wider section of the population access to their materials or at the very least a reference point to their materials.
Like everything else this will change and here at HOB we hope that it?s Austlii and Qudoc who benefit when the changes come.
?
