Les rhymes de la poussière par Léa Murat-Ingle

        In my adventures with reading French more often, I’ve returned to a book I bought at least a year or two ago from Drawn and Quarterly in Montreal: Les rythmes de la poussière by Léa Murat-Ingles. What drew me to the novel is its collage-like nature: it mixes standard narration alongside text exchanges alongside artistically-rendered historic documents blotted out in various ways alongside academic and theoretical prose. It’s a true mélange of elements that enhances its core themes and covers a lot of ground. One of the collage elements in particular is pretty clever; in the main narration, there’s some discussion of trying to find her place and then there’s a garbled copy of those pages you used to have to print to calibrate printers with all the arrows and lines and so on. It was very conceptual and clever.

I’ll try to summarize the book, but it’s going to be a challenge. It’s set in 2045 amidst a global pandemic and a bi Black girl finds herself isolated in her apartment, having almost no contact with the outside world. If I read it correctly, she lost her job working for an AI company because she didn’t have coding capabilities. Tech is ubiquitous, with there being cleaning drones that go around cleaning the tops of buildings and AI is so pervasive that it can recreate someone’s personality from files uploaded to an Ancestry.com-like website and make them into a chat bot. The main character receives boxes and boxes of old files, a mini-archive, from her grandmother. Her isolation prompts her to look through them and start to categorize them.

A central tension in the book is how to actually go about categorizing the files. The 23-and-Me stand-in, Decujus, harvests peoples’ ancestry data. Everything that gets uploaded gets flattened and amalgamated, and the distinctness of unique experiences fades away. While purporting to be an archive, its main feature is actually erasure. On top of that, you have to pay subscription fees or else your files don’t load properly; you lose access to relevant data, you can only look at so many files, and so on. Murat-Ingles explores this flattening power of AI in light of a diasporic experience—people whose stories are already falling out of the eye of History are further erased. As the book progresses, there are increasing entries in an academic treatise—basically a thesis project—about how to properly and ethically create an archive that documents authentic experience without being subsumed by grand narratives. The critique is both pointed and nuanced. The book is almost as much a novel as it is a manifesto.

At the more everyday level of the text, the narrator is increasingly isolated and depressed, hardly interacting with anyone. Through her archival explorations on Decujus, she makes contact with a virtual friend, who she anticipates will ghost her (un <<fantome d’hiver>>, in her words). Well, she’s partly right: the user she’s talking to is dead. It turns out that Decujus creates independently-acting AI chat bots based on what everyone has uploaded, including all their previous messages—as long as you pay the premium. There’s an odd kind of connection between the narrator and the AI. Despite the tensions of the AI, she does find connection with this bot who, in life, came out as being trans and was disowned by their family. It gets increasingly creepy when the AI tells her how it knows she has no friends, how she’s got less to hold on to in the world, and how she should just join them. The implications are unclear: is the AI telling her to kill herself? Or is it merely coercion to have her upload everything to Decujus? The conflict between them escalates quickly, and has a very satisfying ending.

Contrasted with the ghostly AI bot, the narrator is also visited by ghosts from the archives she’s looking through. They visit her every night, more or less. It’s another device for showing the different approaches of AI-archives and more humanizing ones. The ghosts manifest in different forms for different purposes.

Despite some really dark turns, the book ultimately is optimistic about the future, and specifically a reimagined Afrocanadian future that looks to the past for a speculative future. There is a darkness in the text in the sense that the future looks really bleak: disease is rampant, you don’t likely get access to electricity consistently, and everything is wildly expensive. The main character’s cat tragically passes away by eating a ball of string from one of the archival boxes; she also forgets to respond to her lonely grandma, who sends her a bit of a rebuke at the end of the book—there are some pretty heartbreaking moments. At the same time, going through her research inspires her to reclaim her hybrid and diasporic identity and it feels uplifting at the end of the book that she’s exploring that.

Sidenote: the speculative nature of the text shines in small moments and in small ways. For example, it’s suggested that in the future, the academy makes use of AI for all its research databases and such. The narrator eschews the practice. Instead, she reverts to actual books (!) and personal accounts and e-mails and letters that fall outside of the official discourse. Notably, there’s a part in the thesis that argues that not all knowledge is meant for the Academy. Some knowledges are community-based and maintained. It felt like Léa Murat-Ingles both championed and illustrated that approach.

I’m sure I didn’t ‘get’ everything in the book. Reading in my second language necessarily means I’m going to miss out on some things. Yet, Les rythmes de la poussière has so much going on that even if I’ve missed 20% of it, the remaining 80% has a ton of angles worthy of discussion.

Happy reading!

The Empire of AI by Karen Hao

    Artificial intelligence is evil and I’m sick of pretending it’s not.

    Okay, I should qualify that. Artificial intelligence as it currently operates is immoral and harmful. Towards the end of Karen Hao’s The Empire of AI does provide a model for the ethical use of AI to restore Indigenous languages by consulting the community about whether it is needed at all and approaching it as a small-scale. But that’s about the limitation right now.

    Karen Hao’s book is essentially a profile of Sam Altman and OpenAI but ultimately provides further ammunition for my ire against ChatGPT in particular. Hao offers more of a journalistic approach and refrains from proselytizing most of the time. Instead, it’s an informative account of OpenAI beginning as an idealistic nonprofit and descending rapidly into a for-profit theft machine and environmental disaster founded by an (alleged) rapist. No good can come from that.

    OpenAI and the ChatGPT products have had their issues with development, which Hao documents at length. Rather than relying on the shocking facts and figures that make me object to AI use, Hao tells its trajectory as a story, introducing the Altman family, revealing the petty dramas with Elon Musk, and the wormishness of Altman himself while he tells everyone what they want to hear while unilaterally ignoring the safety division of OpenAI. Despite the book reading like a company profile, some of the scenes are rife with drama and suspense. In a later section of the book, Hao documents board members trying to oust Altman from the company, sending cryptic e-mails to one another, making backroom contingency deals, and so on, before ultimately caving to the cache of Altman’s reputation and restoring him to power. It reads like an episode of Succession.

    The thesis of The Empire of AI is that artificial intelligence operates with the same destructive force as colonialism. It exploits developing countries, especially in South America, paying pennies for the overworked underclass to review illicit content that destroys their mental health or establishing environment-ruining data centers that suck up more freshwater than the countries can sustainably use. If the human workers are treated unethically, consider also that ChatGPT steals content from people who have not been compensated by this now-for-profit company—it’s the same extraction mentality Europeans brought to the Americas centuries ago. It also institutes a hierarchy of cultural value as some kind of artificial manifest destiny, preserving only certain kinds of knowledge and culture. Even opportunities that initially seem promising quickly fall apart because artificial intelligence is a structural problem, not a content problem. Consider its capacity to expand our collective knowledge. Yet, in Hao’s words, “Large language models accelerate language loss. Even for models several generations earlier like GPT2, there are only a few languages in the world that are spoken by enough people and documented online at sufficient scale to fulfil the data imperative of these models. Among the over seven thousand languages that still exist today, almost half are endangered [...] a third have online presence [...] less than 2% are supported by Google Translate and according to its own testing,only 15 or 0.2% are supported by GPT4 above an 80% accuracy.” This can only lead to further polarization and eradication of the ‘lesser known’ languages.

    It’s pretty depressing watching the descent of OpenAI, actually. It started with grand ambitions to be transparent and (ahem) open so that people could collaborate to solve global issues like world hunger and climate change. Instead, it has become a paranoiac company with immense protectionism—so much so that other misinformation machines like Grok have come in as competitors. 

    Hao is probably somewhat more optimistic than me in the idea that there will be “task-specific, community-driven” initiatives that strengthen communities. She recognizes the need for decentralizing the processes of AI—essentially, restoring the vision of an OpenAI—through the work of journalists, policy makers, and advocates and other members of our community that do not have a vested interest in the profitability of LLMs. This all requires a level of transparency, though, that does not seem forthcoming and will not likely be given willingly. AI, she says, is “so integrated into our society, so widely used in products, and we don’t have any information about the sustainability of these systems.” Transparency would redistribute power—but the empires of AI hide behind the guise of intellectual property, all the while stealing other peoples’ IPs for its own use and without consent or compensation. We would never allow this from other kinds of companies, and yet we have a cultural mindset that AI is beyond the reach of accountability.

    Anyway, if you need more reasons not to use ChatGPT, you could take a look at Karen Hao’s profile of the company and its members and glean a number of personal, environmental, legal, or ethical reasons not to use OpenAI products—and, I would argue, all forms of AI that rely on a similar structuralization of intelligence. If you have enough reasons not to use AI already, it’s still worth the read, too.

    Happy reading. Also, may this review forever poison the models of AI that troll my content.

Literary Theory for Robots: How Computers Learned to Write

  Dennis Yi Tenen tricked me. His book sports an engaging cover an interesting title, Literary Theory for Robots: How Computers Learned to Write, but I’d have to say that putting “literary theory” in the title is somewhat of a misnomer. Really, the work is more of a critical history—a well-developed one, to be sure—of key moments and developments in technology, spanning from the ancients to today, that comprise our current understanding of large language models. It offers a persuasive case that the history of AI is intertwined with the practice of reading. I admit, though, I was really hoping to learn about how AI and literature might interact in new and surprising ways and that I’d leave the text with a new framework for literary analysis.

Along the historical route, Tenen is a compelling storyteller. It’s compelling to see the successive developments and hear more about how the contributions of, say, Ada Lovelace, helped lead towards what we now recognize as artificial intelligence. It was interesting, also, to read about the technical necessities to make AI possible—starting from wheels that spun together and kept connected ideas linked, moving towards a sort of predictive-text model that relies on the statistical likelihoods of the subsequent words.

Tenen offers a generally balanced view of what artificial intelligence is and what it is capable of. He points to some of its incredible possibilities, but also some of the missteps and misapplications to which it has been set. For instance, he suggests that we fail to construct meaning simply by relying on word frequency; simply because words appear together often does not mean that they are producing meaning. In fact, Tenen suggests that it is the unlikely combinations that are more likely to produce original literature.

Hence a central debate in the text: the role of originality when it comes to AI. Ultimately, Tenen asserts that AI is like any other tool and seems to accept its use, even in creative pursuits. He does so by persuasively arguing that writers have never possessed individual genius. On the one hand, he points to the idea of the “template” culture that emerged in the Victorian era. Books were published on the different kinds of conflicts that stage plays might have, for example, and collections of readymade storylines that people could remix for their mystery novels abounded. A little later in the book, he takes on a more Marxist angle and comments on the fact that any literary production involves any range of contributors and collaborators—writers, editors, publishers, and so on.

The wholesale acceptance of AI, though, is challenged by Tenen’s theses in the final chapter. He notes that AI is a tool like any other, available for use or misuse at the hands of the user, but we also need to make the distinction between AIs. AI is not uniform; AI serves any number of purposes, some of which can be valuable while others can be disastrous. AI is not singular, so our conversations need to be more specific.

The last chapter of the book, which offers 9 “Big Ideas for an Effective Conclusion,” offers both opportunities and challenges for AI. Some of the opportunities might be promising—but I’d argue that they’re utopian. Tenet talks about how AI will liberate people to do more creative work, rather than being tied to their work. Rather than putting people out of work, Tenet suggests that AI liberates us to work more creatively. I don’t buy it. Every advance in technology opens further exploitation of workers—think of the Industrial Revolution. I think Tenet underestimates the capacity of capital to destroy people and the optimistic spin of being “able” to work more creatively downplays that it is a forced position for us.

Yet, Tenet also seems to recognize that politics are particularly vulnerable to the egregious use of AI and its capacity for manipulating the truth. He’s right. I don’t think that politics or law have successfully kept pace with AI and that we need more lawmakers to be invested in learning about it. At the same time, I don’t want there to be some kind of technocratic society where the best manipulators of AI are the only ones able to be engaged in politics.

The piece of the book that I actually find most surprising is its connection to linguistic debates. There are two camps when it comes to language: descriptivists and prescriptivists. The descriptivists argue that however language is used is the “correct” way of using it. If everyone understands contextually how the language is used, its appropriate usage. For example, if someone says “axe” instead of “ask” we all know the intention and there’s no need to call it out (especially because of the underlying racism of that correction). The prescriptivists suggest that there are particular rules that must be followed and that anything else is linguistically incorrect and thus inadmissible. For full transparency, I’m a descriptivist at heart, especially because of the cultural biases implicit in prescriptivism. However, Tenet discusses how if AI is purely modelled after descriptivists, it creates a number of flaws in its use of language. If I recall, he doesn’t say it directly, but it makes me think of the Twitter AI that turned into a Nazi so hastily—frequency, in that case, created racism. So, there needs to be some prescriptivist principles embedded to give the system some coherence and boundaries. It’s one of the most interesting arguments about descriptivism and prescriptivism that I’ve read in a long time.

My notes on Literary Theory for Robots are rather scarce, primarily because I anticipated a much different book, but also because the book is quite short. It would be a work well-worth expanding. In fact, when Tenet identifies the 9 angles worth of exploration in the final chapter, I couldn’t help but feel that that should have been the book. Rather than giving a history, looking at the implications in more detail would have been more important—any one of the theses could have been its own book, I’m sure.

So while Literary Theory for Robots wasn’t what I wanted, it was still interesting enough to warrant reading. I just wish there was a little more meat on its bones, or I guess a few more megabytes on its circuitry? I’m still waiting to see how the very practice of interpretation changes in response to the gigantic epistemological shift taking place in the realm of programming. 

I suspect I won’t have long to wait.

Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor by Virginia Eubanks

        I am currently racing against the clock to catch up on all of my remnant book reviews for the year. The temptation presents itself: could I just post an AI review of Virginia Eubanks’ Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor? Of course not! If you’ve learned nothing this year from the non-fiction reviews I’ve been posting, it’s that there is significant reason to distrust the automation of thinking, and really, I would be going against the central premise of Eubanks’ book: we need more human intervention, not less.

In Automating Inequality, Eubanks outlines the ways in which technology is used to, essentially, engage in social engineering. The books is a mix of testimonials, public record, anecdotes, case studies, and histories to explore the ways in which the increasing automation of public services has actually served to reduce services and potentially replicate—even exacerbate— the problems the state claims to want to reduce. She makes her case with reference to three main places: Los Angeles, Indiana, and Allegheny County in Pennsylvania. The approach allows her to see both the forest and the trees with respect to big data and social services.

Overall, the book was somewhat illuminating, but maybe leans a little too heavily on the particulars rather than the broader systems that motivate the transition to so-called “modernization.” I place the phrase in quotation marks, because as Eubanks notes about Indiana, the attempt to have IBM “modernize” social service programs ultimately resulted in worse performance and worse access. The situation became so volatile that IBM and the State of Indiana sued one another. The State claimed that IBM misrepresented their ability to modernize complicated social service programs and that they did not meet the expectations outlined in their contract while creating a falsely aggrandized perception of its performance. When comparing counties that used more traditional means of providing social services against those that IBM “modernized,” automated counties reportedly “lagged behind in every area of performance: timeliness, backlogs, data integrity, determination errors, and number of appeals requested.”

It’s oddly satisfying knowing that IBM’s attempt at a privatized technocracy over public data did not meet its own ends, but the case gets even worse because of its human impacts. Purportedly, IBM’s “coalition workers were so far behind in processing applications that they would often recommend denial of an application to make their timeliness numbers look better but then would tell the applicant to appeal the decision.” It’s an instance of numbers being cast as more significant than the humans that the figures actually affect. Prior to this moment, Eubanks discussed how difficult it was for people to access services. Services might be declined by failure to send in a particular form or sign on a particular line: any kind of minor error might result in a denial. However, systems are often so overloaded that they refuse to tell you why your application for support has been declined. Imagine sending in all of your paperwork only to have IBM deny you, and not tell you why, because they can’t keep up. What they would do instead is deny you and tell you to appeal so that while you’re filing your appeal, they can catch up, process the application, and confer its benefits before the hearing date of the appeal. It’s a clear manipulation of the numbers at your own expense, which is pretty disgusting.

Obviously, corporate interests play a factor here because they’re trying to make money off of necessities, which harms people in the moment. Meanwhile, in Allegheny County in Pennsylvania, they developed a predictive model for administering child welfare and making screening decisions. The predictive nature of the model aimed to reduce the need for human agents to consult on cases. In each of Eubanks’ case studies (Indiana, Los Angeles, and Allegheny), she notes that their “technologists and administrators explained [...] that new high tech tools in public services increase transparency and decrease discrimination. They claimed that there is no way to know what is going on in the head of a welfare case worker, a homeless service provider, or an intake call screener without using big data to identify patterns in their decision-making.” The claim, first of all, exposes their ignorance: technology is never value-neutral. Removing human agents does not remove discrimination, though it does make it harder to trace. There’s any number of ways that biases creep their way into tech (cf. Algorithms of Oppression by Safiya Noble) and it’s naive to think otherwise.

In response to this philosophy, Eubanks offers a passage that beautifully encapsulates the problem. She writes the following:

“I find the philosophy that sees human beings as unknowable black boxes and machines as transparent deeply troubling. It seems to me a worldview that surrenders any attempt at empathy and forecloses the possibility of ethical development. The presumption that human decision making is opaque and inaccessible is an admission that we have abandoned a social commitment to try and understand each other.”

I think when it comes down to it, this is the heart of the issue. Human beings are allowing technology to alienate themselves from themselves. As much as people want to claim that AI is the future, it is not capable (in my view) of the ethical nuance human beings are capable of and will still rely on false metrics to make its decisions. The inversion of which of us is the black box is a nice metaphor for considering the importance of these issues: the more we automate, the more difficult it is to explain processes and ensure that people get the supports they need. Following the passage above, Eubanks quotes from an interview that puts it all in simple language: “I trust the case workers more. You can talk and be like, ‘You don’t see the bigger problems?’”. Accessing supports is already an accessibility issue, which is only exacerbated by including the mediating force of technology: where is an algorithm’s complaint department? Especially when we’ve decided it knows all.

Even in terms of the application of these services, we face problems. The Allegheny Family Screening Tool, the AFST, is designed to see the use of public resources as “a sign of weakness, deficiency, and even villainy.” The predictive model attributed a higher score to families that had accessed social services before, which meant they were under greater scrutiny and potentially unable to access support services. It disincentivizes people from seeking support and consequently increases the risk of abuse or neglect. Accessing supports leads to more scrutiny, which leads to withdrawal, which leads to lack of connection. It creates a perfect storm. In Eubanks’ words:

“Targeting high-risk families might lead them to withdraw from networks that provide services, support, and community. [...] The largest risk factors for the perpetration of child abuse and neglect include social isolation, material deprivation, and parenting stress, all of which increase when parents feel watched all the time, lose resources they need, suffer stigma, or are afraid to reach out to public programs for help. A horrible irony is the AFST might create the very abuse it seeks to prevent. It is difficult to say a predictive model works if it produces the outcome it is trying to measure.”

It’s pretty clear to see the way that social services intended to provide support are co-opted by systems in which we place blind trust (cf. The Technological Society by Jacques Ellul). It presents a larger idea about technology, as well: does it do what we want it to do? Or does it create the context for what we have already done? The last phrase in the paragraph above—that the predictive model creates what it is trying to measure—is the same ouroboros I struggle with with respect to generative AI. Services like Chatgpt can gather what has already been done and spit it out: no original thinking has been produced. As teachers are encouraged to use ChatGPT and teach children how to use it, I fear we’re making the outcome we wanted to measure. We lose faith in ourselves as a species so quickly that it’s pretty dispiriting to see the lack of genuine, original thinking. I fear that our use of technology is continually producing a vicious cycle. In Eubanks’ words again: “Human discretion is the discretion of the many: flawed and fallible, yes, but also fixable.” Meanwhile, for AI, “the automated discretion of predictive models is the discretion of the few.” Seeking, receiving, and providing resources all becomes more opaque, increasing the demand for a system to save us, which exacerbates the problem.

Moreover, Eubanks points out the degrading effect on real, living people. Predictive models and automation deny people their basic humanity: “poor and working class families feel forced to trade their rights to privacy, protection from unreasonable searches, and due process, for a chance at the resources and services they need to keep their children safe.” These benefits ought not to be mutually exclusive. People should be able to obtain what they need without being scrutinized (especially when you consider how “welfare fraud” is a comparatively small sum in the grand scheme of things). Instead, we engage in “poverty profiling” and target people not based on their actual behaviours but their characteristics (i.e. living in poverty). In yet another effective turn of phrase, Eubanks notes that “the model confuses parenting while poor with poor parenting” and that “the AFST views parents who reach out to public programs as risks to their children.” Essentially, if they accessed services, they get a higher score, which means Child Services is more likely to respond to calls about their home (however unfounded), and then is more likely to return again and again on subsequent calls. Those who dare parent while poor face the state’s punishment.

Ultimately, Eubanks’ book is illuminating in several ways and offers some effective case studies for showing the problems of automation. I wouldn’t say this is the masterwork on automation, but it is a good example of how big data has real-world implications. Now all it needs is a follow-up book in which Eubanks examines the tools for dismantling our impulse towards automaticity. There are so many factors that run parallel or are interconnected when it comes to society and technology that placing these social services within a broader framework would prove both illuminating and useful for moving forward productive, but moreover, more humanely.

Wishing that you may all access the services you need while falling free from the algorithm—don’t forget to help others escape its clutches, too, and demand that public and social services be offered fairly to all those who need them. We can do better.