Who 'Makes' the Rules? Whose Labels to Use?
Efforts toward representation & fairness in A.I. are worthwhile, yet they are unlikely to include sensitivity to Christian communities. (Plus '90s Christian pop references.)
by Scott H. Hawley
Visitors to Google Codelabs’ TensorFlow tutorial [1] will encounter two diagrams contrasting traditional computer programming with Machine Learning (ML):
“I guess the one thing that’s been bothering me the most is when I see us playing by the same rules that the world is using.”
This ties into current conversations about fairness in the development of automated decision-making systems, but first, let’s explore similarities between ML and Christianity regarding rules.Like ML, Christianity is presented through the letters of Paul as being different from following the rules in the Law of Moses. To use ML terminology, the Law was intended to be a metric, not an objective. The Law was a standard of behavior yet was “powerless” [2] to produce the goal of inner righteousness: “For no one can ever be made right with God by doing what the law commands. The law simply shows us how sinful we are” [3]. In Christian traditions, this adherence to rule-following is called legalism, an ancient example of Goodhart’s law:
“When a measure becomes a target, it ceases to be a good measure.” [4]
Like the rule-following artificial intelligence (AI) systems [5] made for business (missing reference) and consumer applications [6] before the “rise” of ML, a legalistic system suffers from deficits such as brittleness [7], lacking knowledge acquisition, and difficulty handling complex situations. Paul encourages believers to abandon rule-following and “live by the Spirit” [8], thereby making inferences via the Law put in our minds and written on our hearts [9] – or in ML parlance, encoded in the weights of our neural networks. Another ‘90s CCM title expounds,
*“It’s just a Spirit thing,
It’s just a holy nudge,
**It’s like a circuit judge in the brain…
It’s just a little hard to explain.”*
– “Spirit Thing,” Newsboys (1994)
“Hard to explain”: like many ML systems, the advantages come at the cost of explainability [10]. Having rules written out offers transparency [11] that sophisticated inference systems may lack. The “right to an explanation” in the EU’s GDPR [12] precludes [13] the use of complicated neural networks since decisions of who receives bank loans or is deemed to be “high risk” in criminal proceedings [14] are too important to be left to inscrutable models, many of which demonstrate unfairness in various forms referred to collectively as “bias.” As the recent conflict between Yann Lecun and Timnit Gebru [15] showcased, the notion that bias comes from training datasets is strong, but bias can enter at many stages of development involving a global supply chain [16] . Still, cases such as the lack of Black faces [17] in computer vision datasets or the use of historical data from mostly-male hiring practices [18] make datasets a good focus for our discussion. ML models trained on textual datasets learn to represent the biases of humans [19] . Microsoft’s “Tay” chatbot fiasco [20] showed how easy it is to create “racist AI” [21] without careful curation of data, yet even one of the most widely-used computer vision dataset is rife with derogatory labels for people [22]: “snob,” “slattern,” etc. These matter because ML systems are often structured as classifiers, “learning” these “ground truth” labels to apply them to new cases. Labels are the call-signs of classifications which then feed into policies. And as the saying goes, labels stick: to justify a discriminatory or violent policy, tie it to a label [23]. If someone labels you a Nazi and believes Nazis merit punching [24],… One ML researcher was recently labeled [25] a “bigot” for sharing reports of human rights abuses in China and his employment threatened because “people will feel unsafe.” Leaving aside the irony of who is making whom feel unsafe, we see that labels can be powerful [26] and contentious [27] . The choice of “whose labels to use?” is then key. This is easily seen in medicine, where labels by experts [28] should supersede those obtained by crowdsourcing [29] , yet this applies to moral judgments [30] as well. In the words of another ‘90s CCM song,
“To label wrong or right by the people’s sight is like going to a loser to ask advice.”**– “Socially Acceptable,” DC Talk (1992)
One important labeling task, sentiment analysis [31], seeks to classify content as expressing “positive” or “negative” sentiment for text such as tweets, movie reviews, or – as demands multiply for content moderation on social media [32] – hate speech. Usually models output a sentiment score between 0 and 1, converted to a (binary) classification via a threshold value, say, 0.5. A naively-constructed dataset will tend to yield bias by yielding lower scores [33] for expressions of “underrepresented” status in terms of race, gender, or religion. Another important text processing task involves forming word associations via “word [vector] embeddings” [34], which end up encoding human biases present in the text [19] , raising the question “Whose Truth is the ‘Ground Truth)?’” [35] To address problems of bias, there are efforts to increase diverse representation of people within datasets, within teams who create ML models and the voices amplified at conferences for industry and academia, in categories of race, gender, and sexual orientation, and… “Religion” is often conspicuously omitted from the list of categories, despite the underrepresentation of religious persons in the technology industry and academia. This is a powerful omission, as Christians are encouraged in Scripture to classify themselves not in ethnic terms [36] but rather by the cross-racial unity of the family of God [37]. Thus the classification scheme determines what forms representation and diversity take. As Kate Crawford pointed out to the Royal Society, “Systems of classification are themselves objects of power” [38] increasingly concentrated in the hands of the few creating AI systems. Crawford continues, “AI is rearranging power, and it’s about configuring who can do what with what and how knowledge itself works.” The preference for Christians and technology companies to transcend the diversity of backgrounds is common ground, yet diversity of beliefs brings with it inevitable conflicts [39], and whereas Christians are instructed by Jesus’ example to sit at the table with ‘problematic’ individuals [40] and even love those we find reprehensible [41] , the secular world is under no such compulsion [42], even proudly refusing to sit with [43] opponents. As Alistair McIntyre clarified in After Virtue [44] , the inconsistent use of values in modern pluralistic societies results from keeping the conclusions of centuries of moral reasoning while denying their basis, yielding unending spectacles [45] of moral positions asserted with passionate sincerity by avowed moral relativists. Other efforts to “fix” bias include de-biasing” ML models [46] – which amounts to “re-biasing,” [depending on what one means by “bias” [47]. De-biasing enforces symmetry (or invariance) of outputs to changes in inputs (e.g., male→female, black→white), yet “bias” can also mean “implicit assumptions.” One should be aware that requiring symmetry is a bias [48]; for example, de-biasing the “horrifying” result, “Man is to Computer Programmer as Woman is to Homemaker” corrects a historical error for “programmer” (e.g., the creator of programming was a woman) yet imposes an ideological bias against the gender role for “homemaker” that many – though certainly not all – Christians value. This is an assertion of will, as Hannah Fry says, “deciding what kind of world we want to live in,” begging the question of “who is ‘we’?” Examples of algorithmic anti-Christian bias don’t make news like those for race or gender, yet include automated (mis)labeling of traditional Christian views on sexuality as hate speech and worship videos as “harmful or false information.” This makes sense given the indifference or outright hostility toward Christians in academia, the press, and Silicon Valley – the latter meriting an episode of the HBO TV comedy! A few recent instances include: signers of the Southern Baptists’ statement on AI Ethics shying from listing company affiliations (an instance of “closeted conservatives”) because even privately acting according to one’s faith outside of work can get you “outed” and ousted, a prestigious panel on “science, religion, AI & ethics” featuring atheists but no believers (or coders?), and a research team showcasing anti-Catholic profanities as their premier text corpus example. Were it not for Biblical views on sexuality, “Christianophobia” might draw mainly from the long-discredited ‘conflict’ narrative about the harmony of faith and science (even though the ‘harder’ the science, the more Christians one finds) or stereotypes based on deplorable subgroups. The promotion of harmful Christian stereotypes by “progressive” organizations is at odds with their stated values, and not done to others (e.g., not Muslims) holding similar views on sexuality. Yet Jesus predicted this: “You will be hated by everyone because of me.” Should we expect otherwise? Claims of neutrality are belied when moderators proudly delete political posts they disagree with. The point is not about being pro-Trump or anti-Trump, but to highlight the dynamics of authority at work in tech companies’ control over the global flow of information, its classification, the resulting policies, and their effects. As Zeynep Tufecki recently stated, “The real question is not whether Zuck is doing what I like or not,…[it’s] why he’s getting to decide what hate speech is.” For ML-based content moderation, this returns us to “whose labels?” and “who’s at the table?” The “expert labeler” for Christians is God revealed in Scripture, in the person of Jesus, experienced via the Holy Spirit. Any efforts to create CCM-like alternative media, social platforms, or even ML models – and due to space we’ll skip the important question of who gets to apply the label “Christian” and to whom – must be done viewing God as the power apart from worldly politics. He holds the power of re-labeling: declaring the unloved as beloved, the guilty as innocent; renaming and affirming new identities. Holy Spirit can reset the “weights” in our minds’ neural networks, rewriting rules internalized in painful pasts. The Christian answer to the new ML paper “Can the Rules in a Deep Network be Rewritten?” is emphatically yes. The world itself will be regenerated. Thus we can perform “bi-directional” inference by looking backward at God’s faithfulness and forward via his promises as we focus our attention on Christ in the present, so we may (to put 2 Corinthians 3:18 into ML jargon) maximize our similarity measure with him, to ever-increasing capability. When working “at the table” with others, we should promote the many goals of progressive technical and academic groups which agree with Christian precursors such as justice for the poor, welcoming of foreigners, relief for the needy, stewardship of the environment, etc. As we seek to work diligently for powerful non-believing employers, we can emphasize this common ground even while having a separate basis for innovative thinking and partner together to transform the world in positive ways.