China’s emerging Social Credit System has attracted growing media attention as a centerpiece of the country’s increasingly advanced and pervasive surveillance regime. Since its announcement, though, much of this coverage has included either sensationalist exaggerations or smaller misunderstandings or conflations. Yale Law School’s Jeremy Daum has been a consistent critic of these errors, both on Twitter and at the China Law Translate blog he edits. The site’s extensive collection of related resources includes "Giving Credit," a series of blog posts explaining the goals and various elements of the developing social credit scheme, as well as some of the more persistent misapprehensions about it.
The series continued this week with an examination of some of the limitations and safeguards, such as consent requirements, currently in place:
Each of the four local social credit Regulations defines Social Credit Information as information useful for understanding a subject’s (ie a person or organization’s) compliance with laws, regulations, and contractual obligations.
This definition might seem like an empty bit of legalese, but it contains a critical point. It tells us that the focus of Social Credit is compliance with legal obligations, which roots the system in existing laws. The 2014-2020 social credit plan, and some of the local Regulations, also contain broad language discussing the construction of a ‘culture of creditworthiness’ and ‘moral education’, which has led many to fear that social credit will regulate and restrict behavior beyond the requirements of law. Here the rules show that when it comes to actual function, Social Credit’s core concern is strictly with legal obligations.
The understanding of ‘credit’ is thus narrower than many fear, but is still a broader concept than may be familiar elsewhere. […]
[…] The social credit apparatus is going to integrate information by regulating data exchanges and information sharing between different regions and departments, with much of it made public. That doesn’t mean that the information will somehow be crafted into a unified score; mainly because that wouldn’t be terribly useful to anyone.
[…] Information on natural persons’ religious faith, genetics, fingerprints, blood type, illnesses or medical history must not be collected as either public or market credit information. I put this first because it shows limits to the scope of Social Credit Information. The government may well have some of this information on file for its citizens, but it is not to be considered in evaluating ‘credit’ no matter how useful it is for predicting compliance with legal obligations. [Source]
Authorities, particularly in Xinjiang, are indeed widely reported to have been collecting this kind of information outside the social credit system.
Last month, the series’ second installment compared regional regulations issued by Shanghai, Hebei, Hubei, and Zhejiang, noting "the social credit systems’ primary focus on organizations rather than individuals, and its development through blacklists and credit systems targeting specific industries." It went on to discuss some of the various incentives and punishments already in operation, including widely reported professional and spending restrictions for those blacklisted for, wittingly or not, failing to comply with court rulings. "The requirements for inclusion in the lists," Daum wrote, "are generally that one endanger others or national interests through their untrustworthy conduct, but remain unnervingly vague despite definitions in the relevant articles."
Particularly important are the limits on high-spending in Hubei’s article 30(5) and Zhejiang’s article 26(4) above, because their connection to the conduct which had one entered onto the list is most tenuous. Unlike keeping people out of the profession in which they violated rules and regulations, this type of provision seems more punitive or vindictive (you can’t have nice things if you were bad).
[…] China has always punished conduct that most believe should be legal, like speech. So, in considering potential issues in the emerging social credit system, I suggest looking at social credit in those areas where there is disagreement with China’s regulation more generally. It’s no surprise that to find that social credit doubles down in problematic areas.
[…] The impact of social credit can […] be seen [in the Internet Group Information Service Management Provisions]: Article 7 requires that “Internet group information service providers shall establish a user credit level management system, providing services corresponding to credit levels. “ (emphasis added).
[…] Decreasing the level of access to internet services, not as the result of a criminal or administrative ruling, but as the result of an undefined credit variable, seems to be very new and very dangerous. The credit in these rules likely doesn’t even reflect a nationally standardized system, but one created by the service provider who is trying to satisfy their own obligations under the rules. This is where the social credit system gets closest to the everybody-rating-everybody nightmare that depicted by so much media coverage of China’s Social Credit. To protect themselves from liability, everybody needs to keep others in check. There is little to lose by overzealously limiting services, but real consequences for failing to stop others who are breaking the rules. [Source]
Other schemes of varying sophistication exist alongside the provincial-level and national court schemes described in the post:
How China's supposedly hi-tech social credit system actually works in one of the official pilot cities, Rongcheng: almost no one in urban areas has heard of it. In villages, they have this ultra low-tech rating system. pic.twitter.com/M6S6jUi0NW
— Tom Hancock? (@hancocktom) January 16, 2018
Villagers annually nominate names in categories such as 'filial piety' 'clean and tidy' 'promotes wealth creation'. At lunar new year the votes are added up and points given out. pic.twitter.com/j4xxKCnXup
— Tom Hancock (@hancocktom) January 16, 2018
Villagers who do well in the vote get red stars next to their name and easier access to bank loans pic.twitter.com/R2rIdOcjkO
— Tom Hancock (@hancocktom) January 16, 2018
Villagers said it was basically a renaming of a 'virtue promotion' system introduced a decade ago. Few take much notice of it as punishments are rare.
— Tom Hancock (@hancocktom) January 16, 2018
Tired take: "China's social credit system is Confucian, because I've heard of Confucius and am pretty sure he was Chinese too."
Wired take: "The social credit system is an update to the 'ledgers of merit and demerit 功過格' b/c we're trapped in a dumber reprise of the late Ming." https://t.co/ucS5wExGJU— Brendan O'Kane (@bokane) January 16, 2018
Besides serving as a response to perceived breakdowns of morality due to a newly mobile population, increasing commercialization, and the rise of merchants at the expense of the literate classes, 功過格 had a system of frequent-flier points that let you upgrade your karma. pic.twitter.com/7RugVY6AW8
— Brendan O'Kane (@bokane) January 16, 2018
In a separate post over Christmas, Daum outlined some of the most common misrepresentations of social credit—that "the system will be mandatory by 2020," that "all Citizens will be given a ‘Citizen Score,’" and that "your score will dictate your place in society"—and noted the prevalence of coverage that "outright conflates or blurs the line between Social Credit and a number of private programs- especially [Alibaba’s] Sesame Credit."
Social credit is best understood as part of China’s growing surveillance apparatus. Like the cameras that now line every street and alley in Beijing, social credit adds something to security at the expense of privacy. Just as the cameras monitor our physical activity using AI and facial recognition, social credit is about using real-name registration systems and big data management to create records of our digital conduct. All aspects of our lives – transactional, educational, medical, legal, recreational, and consumer- leave a digital footprint, and social credit is about integrating and mobilizing that data to increase social and economic stability. Of course, it is the Party-State that maintains and mediates access to this data.
[…] Nothing in the discussion above is meant to suggest that Social Credit isn’t frightening. Like many other parts of China’s surveillance apparatus, it is very frightening. Even as it delivers some safety and security to many Chinese, it is all too clear from watching Xinjiang or other areas of unrest, that these systems can be quickly weaponized for harsh control. They can also be mobilized to identify and undermine individual without even upsetting the regional ecosystem.
[…] This article wasn’t meant to be about China so much as foreign coverage of China. China’s Social Credit is often used as removed way of discussing our own situation from a safe distance. This is, of course, also the role that science fiction like “Black Mirror” and Orwell’s 1984 has traditionally played, so it isn’t surprising to see them invoked here as well. We look at exotified foreign nations or speculative futures in order to reflect on our present, but what we take away from it likely says more about us than about the subject of our examination.
This is all fine, but it’s important to also remember that there are two stories worth exploring here; what is actually happening in China and what we fear is happening to ourselves. [Source]
The opening post in the "Giving Credit" series, which outlined the social credit scheme’s general principles, also emphasized this point:
China’s evolving ‘social credit’ system captures people’s imaginations. The idea of a monolithic data platform being used to judge us all resonates beautifully with rising anxieties about government collection of metadata, increasingly spot-on targeted advertising, and other horrors lurking in the enormous sea of data we store online. So, when China – justifiably associated with political, media, and social controls – reveals that it is aggregating information on ‘creditworthiness’ that will be used to assign rankings to all businesses, individuals, and other entities— it’s hard to blame folks for getting a little ahead of themselves.
Unfortunately, despite how close to home the issues involved are, many outside of China seem to view the story as uniquely Chinese — social credit as a wacky dystopian daydream from a faraway land. Parts of the system may be distinctly Chinese, but the system ultimately reflects China’s grappling with issues faced globally: how to positively utilize new information technology, the need for reliable economic data, and how personal privacy can best be sustained in all this. “Social Credit” is a Chinese formulation, but pieces of the system have been proposed elsewhere in one form or another for some time; and how each nation addresses these complex questions will be very revealing about the core values of that nations.
The consideration of China’s system can’t end with “China Bad”- which is an easy and common enough story – but should stimulate a discussion about how our laws and cultural norms struggle to keep pace with a changing information landscape, and if “Social Credit Bad”, how do other societies do better. These are not questions about the future, but are very much before us now.
[…] China may be able to more fully consolidate and act on it, but the data involved in Social Credit exists everywhere. Choices on its use are being made- by governments, by corporations, by consumers- both actively and through passive evolution. In watching China’s grand experiment unfold, we need to start not just assessing their system, but asking what our own use of data says about our priorities and values. [Source]
Lund University’s Stefan Brehm and Nicholas Loubere offered a similar warning at The Conversation this week:
The Chinese government’s ongoing attempts to create a social credit system aimed at rating the trustworthiness of people and companies have generated equal measures of fascination and anxiety around the world. Social credit is depicted as something uniquely Chinese – a nefarious and perverse digital innovation that could only be conceived of and carried out by a regime like the Chinese Communist Party.
[…] It is seen as a signal of a dystopian future, but one that could only exist in China’s authoritarian context. But China’s plans to “build an environment of trust” – as the Chinese government’s State Council puts it – using the data generated from digital activity is not uniquely Chinese. And the country’s experiments are a natural extension of a global trend where data is being used to control society.
[…] Social credit has the potential to cement and exacerbate existent power imbalances in societies, while simultaneously closing down spaces of resistance. The social credit system in development in China is a phenomenon that belongs to a global trend with a transformative potential that is just as threatening in Western democracies. It is but one harbinger of a likely common digital future marked by a shift away from the rule of law and towards rule by algorithms. [Source]
For an accessible overview of some of these issues in the West, see Zeynep Tufekci’s recent TED Talk, "We’re building a dystopia just to make people click on ads."
