Our Feedback on Algorithm Governance

The Council continues to commit much of our resources toward open government, and specifically the Open Government Partnership (OGP) process. 

We continue to believe that open government is good in and of itself, in addition to being good for democracy and an essential component to defending and enlarging civil liberties.  We desire for everyone to have access to sufficient information to prove to themselves that their rights, and those of their families and communities are being respected.

The government collects and stores large amounts of data about everyone, even before accounting for the massive overreach in formal state surveillance. Yet, the Council believes that – in spite of our rights under the Privacy Act and Official Information Act – no one can prove that our rights are being respected because the data needed to prove that either does not exist or is so poorly organised as to be practically inaccessible. Lack of organisation means that no one, regardless of clearance, can assess what information even a single agency holds on an individual, never mind what data the government as a whole holds on self-identifying groups.

We continue to support the international consensus that freedom of expression depends on access to information, as set out in article 19 of both the Universal Declaration of Human Rights and the International Covenant on Civil and Political Rights.

When countries join the OGP, they agree to independent scrutiny of both the creation of their action plans (documents setting out commitments to take various actions) and how well they did in delivering the commitments. There is an OGP independent reviewer for each country. The OGP’s independent reviewer of Aotearoa New Zealand’s activities is currently gathering evidence for their assessment of delivery on our last action plan.

Commitment 8 in the action plan was headlined ‘Review of government use of algorithms’. The Council made the following comments to the independent reviewer about commitment 8 on 2 November 2021:

There has been no progress worth noting towards objective 8, Review of government algorithms. To the best of the Council’s knowledge, not a single agency has published a catalogue of their algorithms or Algorithm Impact Assessments (AIA).

Given the complete lack of openness, we are left to rely upon rumors. What seems obvious however from the lack of visible results is that algorithm governance is neither being funded nor prioritised sufficiently for success to be probable.

The Council makes no changes to our 2019 recommendations. (We also organised a public panel discussion on the Algorithm Charter, in early 2020.) There has been some education and some progress toward incorporating Te Ao Māori, but our fundamental concerns in those areas have not been meaningfully addressed. 

Since 2019, Aotearoa New Zealand has continued to fall behind. Recent work in algorithm governance, for example Articles 4, 6, and 8 of the Chinese Cyberspace Administration’s Internet Information Service Algorithmic Recommendation Management Provisions or Article 10 of the Gig Worker’s Bills of Rights, go beyond transparency by asserting rights to justice. The Council encouraged this shift in perspective in our 2019 submission, and we have relentlessly continued to advocate for this perspective in our meetings with our government.

The Council notes with some alarm that the government Data Ethics Group has not met in more than a year, during which government of use and governance of data to tackle the Covid-19 pandemic has been a concern

The Council notes the Ombudman’s Investigation into the Managed Immigration and Quarantine Booking System as an inevitable outcome of the failure to govern algorithms. Instead of seeking to proactively prevent harm, regulators have waited for the public outcry after so much harm has been done. The Council does not believe that the MIQ booking system is unusual in the amount of harm it has caused or in its failure to include human rights by design. Rather, we note that those harmed by the MIQ booking system are, on average, more privileged than the average person.

The Council is aware that our recommendations are a significant departure from current practice, and will introduce both cost and delay. However, without public oversight the Council believes the government’s use of algorithms poses a credible threat to our liberties, and that the measures we propose are reasonable and commensurate to that threat.