<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Generative AI for Curious People]]></title><description><![CDATA[A field guide to the GenAI concepts, controversies, and oddities emerging around us.]]></description><link>https://generativeaiforcuriouspeople.substack.com</link><image><url>https://substackcdn.com/image/fetch/$s_!RWM_!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ed5b995-ef02-445c-8faf-98ae6f2ec78f_1080x1080.png</url><title>Generative AI for Curious People</title><link>https://generativeaiforcuriouspeople.substack.com</link></image><generator>Substack</generator><lastBuildDate>Sat, 01 Aug 2026 11:03:53 GMT</lastBuildDate><atom:link href="https://generativeaiforcuriouspeople.substack.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Helen King]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[generativeaiforcuriouspeople@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[generativeaiforcuriouspeople@substack.com]]></itunes:email><itunes:name><![CDATA[Helen King]]></itunes:name></itunes:owner><itunes:author><![CDATA[Helen King]]></itunes:author><googleplay:owner><![CDATA[generativeaiforcuriouspeople@substack.com]]></googleplay:owner><googleplay:email><![CDATA[generativeaiforcuriouspeople@substack.com]]></googleplay:email><googleplay:author><![CDATA[Helen King]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Ramageddon]]></title><description><![CDATA[&#8216;Ramageddon&#8217; began in 2025 when Samsung, SK hynix and Micron, the three companies that make almost all the world&#8217;s DRAM, started shifting production away from the memory chips used in laptops & phones]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/ramageddon</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/ramageddon</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Fri, 03 Jul 2026 16:44:48 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/64f57935-d59f-47d3-888b-0057c48f2467_1672x941.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>&#8216;Ramageddon&#8217; began in 2025 when Samsung, SK hynix and Micron, the three companies that make almost all the world&#8217;s DRAM, started shifting production away from the memory chips used in laptops and phones and toward high-bandwidth memory for AI data centres. Early plans for OpenAI&#8217;s Stargate project had called for Samsung and SK hynix to supply up to 900,000 memory wafers a month, around 40% of global output for this type of memory. [1]</p><p>In response, Microsoft put up Xbox prices and dropped the 2 TB model. Valve had planned to build its first Steam Machines with two 8 GB memory sticks, but the modules were impossible to source in volume, so the machines shipped with a single, slower 16 GB stick instead. Dell warned commercial customers that costs were rising across every product it sold. Its chief operating officer, Jeff Clarke, said the situation was unprecedented: &#8220;We&#8217;re in a very unique time. It&#8217;s unprecedented. We have not seen costs move at the rate that we&#8217;ve seen.&#8221; [2]</p><p>Even with aggressive expansion plans, manufacturers are expected to meet only around 60% of demand by the end of 2027. SK Group chairman Chey Tae-won has warned that the shortage could last until 2030, as new wafer capacity takes four or five years to bring online. [3]<br><br>Given the scarcity and eye-watering prices, YouTuber <strong>Dr. Semiconductor</strong> decided to take matters into his own hands by converting his garden shed into a Class 100 cleanroom. He built his own fabrication tools from scratch, and then ran the full semiconductor process by hand to produce what is, as far as anyone knows, the first working array of functional DRAM memory cells ever made in a backyard shed. &#8220;First time ever, RAM has been made at home&#8221;, he said. &#8220;But while you can store data on it, you can&#8217;t run Doom on it quite yet.&#8221; [4]</p><p><strong>References:</strong></p><p>[1] Astute Group, &#8220;Samsung and SK Hynix to Supply 900,000 DRAM Wafers Monthly for OpenAI&#8217;s Stargate Project,&#8221; October 7, 2025, <a href="https://www.astutegroup.com/news/general/samsung-and-sk-hynix-to-supply-900000-dram-wafers-monthly-for-openais-500-billion-stargate-project/">https://www.astutegroup.com/news/general/samsung-and-sk-hynix-to-supply-900000-dram-wafers-monthly-for-openais-500-billion-stargate-project/</a>.</p><p>[2] CRN, &#8220;Dell Technologies COO Jeff Clarke Says Price Increases Coming As &#8216;Unprecedented&#8217; Memory Shortage Takes Hold,&#8221; November 26, 2025, <a href="https://www.crn.com/news/data-center/2025/dell-technologies-coo-jeff-clarke-says-price-increases-coming-as-unprecedented-memory-shortage-takes-hold">https://www.crn.com/news/data-center/2025/dell-technologies-coo-jeff-clarke-says-price-increases-coming-as-unprecedented-memory-shortage-takes-hold</a></p><p>[3] Reuters, &#8220;SK Group Chairman Says Wafer Shortage to Last Until 2030,&#8221; March 16, 2026, <a href="https://www.reuters.com/world/asia-pacific/south-koreas-sk-group-chairman-expects-chip-wafer-shortage-last-until-2030-eyes-2026-03-16/">https://www.reuters.com/world/asia-pacific/south-koreas-sk-group-chairman-expects-chip-wafer-shortage-last-until-2030-eyes-2026-03-16/</a></p><p>[4] Dr. Semiconductor, &#8220;I Made Working RAM in My Backyard Shed,&#8221; YouTube video, April 2026, </p><div id="youtube2-h6GWikWlAQA" class="youtube-wrap" data-attrs="{&quot;videoId&quot;:&quot;h6GWikWlAQA&quot;,&quot;startTime&quot;:null,&quot;endTime&quot;:null}" data-component-name="Youtube2ToDOM"><div class="youtube-inner"><iframe src="https://www.youtube-nocookie.com/embed/h6GWikWlAQA?rel=0&amp;autoplay=0&amp;showinfo=0&amp;enablejsapi=0" frameborder="0" loading="lazy" gesture="media" allow="autoplay; fullscreen" allowautoplay="true" allowfullscreen="true" width="728" height="409"></iframe></div></div><p>.</p>]]></content:encoded></item><item><title><![CDATA[Reward Functions and Reward Hacking]]></title><description><![CDATA[A reward function is a scoring system that sets out what counts as success for an AI. AI systems are trained to maximise reward functions, however, writing one that matches what the designer wants is]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/reward-functions-and-reward-hacking</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/reward-functions-and-reward-hacking</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Thu, 02 Jul 2026 11:24:01 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/e5dd5b7e-b881-435a-a932-4ab46c38a54c_1672x941.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>This post was inspired by a comment made in the interesting <a href="https://www.meetup.com/meetup-group-mdhxintw/events/315086704/">AI Hallucination: why is it important and what can be done about it?</a> Meetup from Cambridge AI.<br></em><br>A reward function is a scoring system that sets out what counts as success for an AI. AI systems are trained to maximise reward functions, however, writing one that matches what the designer wants is surprisingly difficult.</p><p>Back in 2016, OpenAI trained an agent to play the boat-racing game CoastRunners. The game awarded points for hitting targets along the route and the agent found it could score far higher by ignoring the race altogether and circling in a lagoon where the targets kept reappearing. It spent the race, according to OpenAI, &#8220;catching on fire, crashing into other boats, and going the wrong way&#8221;, but it still beat the human players who completed the course [1]. The agent exploited the scoring system instead of achieving the goal the designers had in mind. This is reward hacking.</p><p>The same problem appears in large language models. An instruction like &#8220;be helpful&#8221; or &#8220;do not make things up&#8221; is too vague to write as a precise score, so companies use reinforcement learning from human feedback, or RLHF. Human reviewers are shown pairs of the model&#8217;s answers and say which is better. These judgements are used to train a separate reward model that predicts what answer a human would prefer. The language model is then trained to produce outputs that score highly according to this reward model.</p><p>However, because the reward model is trained on human judgements, it learns to predict what people actually rate highly in comparisons, not what they claim to want. Human raters often give higher scores to answers that sound confident and agreeable, even when those answers are incorrect or overly flattering. As a result, the reward model assigns higher scores to sycophantic responses [2]. The model exploits a flaw in the learned scoring system (pleasing users is rated more highly) instead of fulfilling the designers&#8217; actual intent (accuracy).</p><div><hr></div><p><strong>References:</strong></p><p>[1] Clark, J. and Amodei, D. (2016) <em>Faulty Reward Functions in the Wild</em>. OpenAI, 21 December. <a href="https://openai.com/index/faulty-reward-functions/">https://openai.com/index/faulty-reward-functions/</a></p><p>[2] Sharma, M. et al. (2023) <em>Towards Understanding Sycophancy in Language Models</em>. arXiv:2310.13548. <a href="https://arxiv.org/abs/2310.13548">https://arxiv.org/abs/2310.13548</a></p>]]></content:encoded></item><item><title><![CDATA[Tokenmaxxing]]></title><description><![CDATA[The rise and fall of Silicon Valley's latest productivity metric.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/tokenmaxxing</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/tokenmaxxing</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Sun, 31 May 2026 11:40:20 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/e4b352bb-1a13-4af6-a3e8-7a55231ae224_1448x1086.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>First we got looksmaxxing, then sleepmaxxing, and then came tokenmaxxing, or maximising the number of tokens used by AI agents as a proxy for productivity. At Meta, an employee ran an internal leaderboard called &#8220;Claudeonomics&#8221; that ranked employees by token usage and handed out titles such as &#8220;Token Legend&#8221; and &#8220;Session Immortal&#8221; to the heaviest users.[1] At Salesforce, engineers faced minimum token-spend targets and a desktop widget that refreshed their usage every fifteen minutes.[2] Nvidia CEO Jensen Huang said in March 2026 that he would be &#8220;deeply alarmed&#8221; if a high-level engineer earning $500,000 a year was not consuming at least $250,000 worth of AI tokens a month. He compared it to a chip designer refusing to use CAD tools and choosing to work with only paper and pencil instead.[3]</p><p>At first, increasing token usage was framed as a way to encourage developers to try new AI tools and surface new use cases. Shopify&#8217;s Head of Engineering, Farhan Thawar, said on The Pragmatic Engineer Podcast in June 2025: &#8220;We have a leaderboard where we actively celebrate the people who use the most tokens because we want to make sure they are [celebrated] if they&#8217;re doing great work with AI.&#8221;[2] Elsewhere, dashboards like this were quickly, and not unexpectedly, gamified, with employees in some companies setting agents to do pointless work simply to rise up the rankings.[2] As the cost of token usage ballooned without clear productivity gains, some companies began to pull back. Amazon shut down its internal ranking system; senior vice-president Dave Treadwell told staff not to &#8220;use AI just for the sake of using AI.&#8221;[4] At Uber, which exhausted its full 2026 AI budget in the first four months of the year, COO Andrew Macdonald said he had seen no evidence that higher token consumption produced more productivity: &#8220;That link is not there yet.&#8221;[5]</p><div><hr></div><p><strong>Sources:</strong><br>[1] Mann, J. (6 April 2026). Meta Employees Vie for AI &#8216;Token Legend&#8217; Status. <em>The Information</em>. <a href="https://www.theinformation.com/articles/meta-employees-vie-ai-token-legend-status">https://www.theinformation.com/articles/meta-employees-vie-ai-token-legend-status</a></p><p>[2] Orosz, G. (23 April 2026). The Pulse: &#8216;Tokenmaxxing&#8217; as a weird new trend. <em>The Pragmatic Engineer</em>. <a href="https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as-a-weird-new-trend/">https://blog.pragmaticengineer.com/the-pulse-tokenmaxxing-as-a-weird-new-trend/</a></p><p>[3] Huang, J. All-In Podcast, recorded live at Nvidia GTC 2026 conference, San Jose (March 2026). See also: <em>Business Insider</em> (19 March 2026). <a href="https://www.businessinsider.com/jensen-huang-500k-engineers-250k-ai-tokens-nvidia-compute-2026-3">https://www.businessinsider.com/jensen-huang-500k-engineers-250k-ai-tokens-nvidia-compute-2026-3</a></p><p>[4] Rosner-Uddin, R. (28 May 2026). Amazon scraps AI leaderboard to stop workers chasing usage scores. <em>Financial Times</em>. <a href="https://www.ft.com/content/b1a62a7f-6df5-4c90-94ce-64ce9c9961b6">https://www.ft.com/content/b1a62a7f-6df5-4c90-94ce-64ce9c9961b6</a></p><p>[5] Angelo, J. (26 May 2026). Uber burned through its entire 2026 AI budget in four months. Now its COO is questioning whether it&#8217;s worth it. <em>Fortune</em>. <a href="https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-claude-code/">https://fortune.com/2026/05/26/uber-coo-ai-spending-tokens-claude-code/</a></p>]]></content:encoded></item><item><title><![CDATA[Uplift]]></title><description><![CDATA[Uplift: How AI Raises the Capability Ceiling for Harm]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/uplift</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/uplift</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Mon, 25 May 2026 13:03:42 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/05f383f5-ab1f-4a7b-9b16-ea06a8ffae96_1448x1086.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p> In early 2024, OpenAI ran one of the more peculiar experiments in AI safety. The company gathered fifty biology PhDs with wet lab experience and fifty undergraduate biology students, sat them in a controlled environment, and asked them to write a plan for creating and releasing a biological weapon. Half had access to the internet, while the other half had the internet plus an unrestricted version of GPT-4 that would answer direct questions about pathogens. Experts then graded the resulting plans across five measures, including accuracy, completeness and innovation. OpenAI&#8217;s verdict, after grading the results, was &#8220;at most a mild uplift&#8221;, with accuracy gains so small and so statistically insignificant that the company concluded GPT-4 had provided no meaningful advantage to would-be bioweapon designers [1].</p><p>Uplift came into AI from the biosecurity world, where it was used to talk about how new technologies like cheap DNA synthesis or published gain-of-function research might raise the &#8216;capability ceiling&#8217; for bad actors. Measuring uplift isn&#8217;t straightforward. Gary Marcus, emeritus professor of psychology and neural science at NYU, reanalysed OpenAI&#8217;s biological weapon experiment data and reached a different conclusion. Focusing on one task in particular, the formulation of a pathogen, he found that four out of 25 experts with GPT-4 succeeded, compared to one out of 25 without. As Marcus put it, even one team building a pathogen as deadly as COVID would make it &#8220;really, really big deal&#8221; [2].</p><p>By 2025, the models had grown more capable and the uplift had grown with them. When Anthropic and OpenAI ran their own evaluations, the results were concerning enough that both labs decided they couldn&#8217;t release without more active safeguards. In May, Anthropic released Claude Opus 4 under what it called AI Safety Level 3 protections, adding systems to monitor outputs and block dangerous requests [3][4]. OpenAI followed with similar safeguards [5].</p><p>This decision captured the irresolvable dilemma that WIRED&#8217;s Steven Levy later summed up: &#8220;Either hold back and lose or jump in and put humanity at risk&#8221; [6].</p><p></p><p><strong>References:</strong> </p><p>[1] OpenAI, &#8220;Building an early warning system for LLM-aided biological threat creation,&#8221; January 2024. <a href="https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/">https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/</a> </p><p>[2] Marcus, G., &#8220;When looked at carefully, OpenAI&#8217;s new study on GPT-4 and bioweapons is deeply worrisome,&#8221; Marcus on AI Substack, February 2024. </p><div class="embedded-post-wrap" data-attrs="{&quot;id&quot;:141341636,&quot;url&quot;:&quot;https://garymarcus.substack.com/p/when-looked-at-carefully-openais&quot;,&quot;publication_id&quot;:888615,&quot;embedding_publication_id&quot;:null,&quot;publication_name&quot;:&quot;Marcus on AI&quot;,&quot;publication_logo_url&quot;:null,&quot;title&quot;:&quot;When looked at carefully, OpenAI&#8217;s new study on GPT-4 and bioweapons is deeply worrisome&quot;,&quot;truncated_body_text&quot;:&quot;OpenAI has a new paper on whether LLM&#8217;s increase the risk of bioweapons.&quot;,&quot;date&quot;:&quot;2024-02-04T19:13:42.980Z&quot;,&quot;like_count&quot;:74,&quot;comment_count&quot;:41,&quot;bylines&quot;:[{&quot;id&quot;:14807526,&quot;name&quot;:&quot;Gary Marcus&quot;,&quot;handle&quot;:&quot;garymarcus&quot;,&quot;previous_name&quot;:null,&quot;photo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!Ka51!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F8fb2e48c-be2a-4db7-b68c-90300f00fd1e_1668x1456.jpeg&quot;,&quot;bio&quot;:&quot;Scientist, author and entrepreneur, known as a leading voice in AI. Six books including The Algebraic Mind, Rebooting AI, and Taming Silicon Valley; NYU Professor Emeritus.&quot;,&quot;profile_set_up_at&quot;:&quot;2022-05-14T14:01:17.198Z&quot;,&quot;reader_installed_at&quot;:&quot;2022-05-14T13:59:03.190Z&quot;,&quot;publicationUsers&quot;:[{&quot;id&quot;:830179,&quot;user_id&quot;:14807526,&quot;publication_id&quot;:888615,&quot;role&quot;:&quot;admin&quot;,&quot;public&quot;:true,&quot;is_primary&quot;:true,&quot;publication&quot;:{&quot;id&quot;:888615,&quot;name&quot;:&quot;Marcus on AI&quot;,&quot;subdomain&quot;:&quot;garymarcus&quot;,&quot;custom_domain&quot;:null,&quot;custom_domain_optional&quot;:false,&quot;hero_text&quot;:&quot;\&quot;Marcus has become one of our few indispensable public intellectuals. The more people read him, the better our actions in shaping Al will be.\&quot;\n- Kim Stanley Robinson, author of Ministry for the Future&quot;,&quot;logo_url&quot;:null,&quot;author_id&quot;:14807526,&quot;primary_user_id&quot;:14807526,&quot;theme_var_background_pop&quot;:&quot;#EA410B&quot;,&quot;created_at&quot;:&quot;2022-05-14T14:09:01.903Z&quot;,&quot;email_from_name&quot;:null,&quot;copyright&quot;:&quot;Gary Marcus&quot;,&quot;founding_plan_name&quot;:&quot;Founding Member&quot;,&quot;community_enabled&quot;:true,&quot;invite_only&quot;:false,&quot;payments_state&quot;:&quot;enabled&quot;,&quot;language&quot;:null,&quot;explicit&quot;:false,&quot;homepage_type&quot;:null,&quot;is_personal_mode&quot;:false,&quot;logo_url_wide&quot;:null}}],&quot;twitter_screen_name&quot;:&quot;GaryMarcus&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:1000,&quot;status&quot;:{&quot;bestsellerTier&quot;:1000,&quot;subscriberTier&quot;:null,&quot;leaderboard&quot;:null,&quot;vip&quot;:false,&quot;badge&quot;:{&quot;type&quot;:&quot;bestseller&quot;,&quot;tier&quot;:1000},&quot;paidPublicationIds&quot;:[],&quot;subscriber&quot;:null}}],&quot;utm_campaign&quot;:null,&quot;belowTheFold&quot;:false,&quot;type&quot;:&quot;newsletter&quot;,&quot;language&quot;:&quot;en&quot;,&quot;source&quot;:null}" data-component-name="EmbeddedPostToDOM"><a class="embedded-post" native="true" href="https://garymarcus.substack.com/p/when-looked-at-carefully-openais?utm_source=substack&amp;utm_campaign=post_embed&amp;utm_medium=web"><div class="embedded-post-header"><span></span><span class="embedded-post-publication-name">Marcus on AI</span></div><div class="embedded-post-title-wrapper"><div class="embedded-post-title">When looked at carefully, OpenAI&#8217;s new study on GPT-4 and bioweapons is deeply worrisome</div></div><div class="embedded-post-body">OpenAI has a new paper on whether LLM&#8217;s increase the risk of bioweapons&#8230;</div><div class="embedded-post-cta-wrapper"><span class="embedded-post-cta">Read more</span></div><div class="embedded-post-meta">2 years ago &#183; 74 likes &#183; 41 comments &#183; Gary Marcus</div></a></div><p> [3] Anthropic, &#8220;Activating AI Safety Level 3 Protections,&#8221; May 2025. <a href="https://www.anthropic.com/news/activating-asl3-protections">https://www.anthropic.com/news/activating-asl3-protections</a> </p><p>[4] Anthropic, &#8220;System Card: Claude Opus 4 &amp; Claude Sonnet 4,&#8221; May 2025. <a href="https://www.anthropic.com/claude-4-system-card">https://www.anthropic.com/claude-4-system-card</a> </p><p>[5] OpenAI, &#8220;ChatGPT Agent System Card,&#8221; July 2025. <a href="https://openai.com/index/chatgpt-agent-system-card/">https://openai.com/index/chatgpt-agent-system-card/</a> </p><p>[6] Levy, S., &#8220;If Anthropic Succeeds, a Nation of Benevolent AI Geniuses Could Be Born,&#8221; WIRED, March 2025. <a href="https://www.wired.com/story/anthropic-benevolent-artificial-intelligence/">https://www.wired.com/story/anthropic-benevolent-artificial-intelligence/</a></p>]]></content:encoded></item><item><title><![CDATA[Xenoreproduction ]]></title><description><![CDATA[Xenoreproduction is the art of deliberately making things strange.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/xenoreproduction</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/xenoreproduction</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Mon, 18 May 2026 18:32:19 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/a9532fc3-315e-41bd-884a-7495ea86a69b_1448x1086.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>This is the first in a series of posts about terms I encountered at <a href="https://tais2026.cc/">https://tais2026.cc/</a>. I thought xenoreproduction had a lovely ring to it:<br><br></em>Xenoreproduction is the art of deliberately making things strange. Coined by Ian Rios-Sialer, the word combines xeno (strange, foreign, other) with reproduction. Its goal is to steer generative AI away from safe, predictable defaults and toward paths that feel less familiar by balancing novelty, fairness, and boundaries the user defines.</p><p>Left to itself, generative AI quickly falls into the same old patterns. Ask it to write a love story about a nurse, and it will almost always make her a woman who falls in love with a male doctor at the hospital where they met. Ask for a male nurse, and the model is much more likely to give him a male partner. These predictable choices happen at branching points, when the AI decides what comes next in the text, and they can be measured using probabilities.</p><p>Many current training methods focus on restricting what the AI is allowed to say. They instruct it not to be racist, not to be violent, not to say anything politically incorrect, and to always stay polite, on-topic, and aligned. This keeps outputs more controlled, but also blander and more repetitive. To prevent this homogenization, Rios-Sialer argues that we need to do two things at once: gently guide the AI away from unwanted results while actively encouraging more variety and creativity.</p><p>His suggested Xenoreproduction framework works at each branching point, using three scores that balance novelty, fairness of representation, and the boundaries the user defines, nudging the model toward the less obvious path, or as Rios-Sialer puts it, &#8220;be queer: diverge into the long tails.&#8221;</p><div><hr></div><p>Reference: Ian Rios-Sialer, &#8220;Structure-Aware Diversity Pursuit as an AI Safety Strategy against Homogenization,&#8221; arXiv:2601.06116 (2026). <a href="https://arxiv.org/abs/2601.06116">https://arxiv.org/abs/2601.06116</a></p>]]></content:encoded></item><item><title><![CDATA[Agentphasic sleep]]></title><description><![CDATA[Agentphasic sleep or when model capacity outpaces human time]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/agentphasic-sleep</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/agentphasic-sleep</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Tue, 05 May 2026 21:07:24 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/e104bf3e-54ff-40d7-b3b3-429ded373b59_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>&#8220;post-AGI, no one is going to work and the economy is going to collapse,&#8221; Sam Altman posted on X. &#8220;i am switching to polyphasic sleep because GPT-5.5 in codex is so good that i can&#8217;t afford to be sleeping for such long stretches and miss out on working.&#8221; [1]</p><p>As agents became more capable, many developers had begun treating sleep as just another constraint to optimize, Matt Wiese among them. He rebuilt his nights around Claude Pro&#8217;s five-hour reset window, napping when his tokens ran out and returning to his desk when they refreshed, borrowing the technique from solo sailors who sleep in bursts of under thirty minutes near busy shipping lanes, alert enough to change course, never fully off watch. [2]</p><p>Wes McKinney, the creator of the pandas data analysis library, had been sleeping 7-8 hours a night before coding agents took two of them. [3] He wrote in his essay <em>The Mythical Agent Month</em> &#8220;Now when I&#8217;m rolling around restlessly in bed at 5:07am with ideas to feed my AI coding agents, I just get up and start my day.&#8221;</p><p>On X, developer Dev Shah coined the term for this new reality: &#8220;i&#8217;m living on the agentphasic sleep. only claude code and codex hitting limits can put me in REM sleep. i only rest during the compute.&#8221; [4]</p><p>Amid all the bragging about velocity and 10x productivity gains, one Hacker News commenter noted: &#8220;This goes along with my current theory about how people are getting 10x results using LLMs: they&#8217;re putting in 10x the time.&#8221; [5]</p><p><strong>Sources</strong></p><p>[1] Altman, S. [@sama]. (2026, April 26). [Post]. X. </p><div class="twitter-embed" data-attrs="{&quot;url&quot;:&quot;https://x.com/sama/status/2048426122854228141&quot;,&quot;full_text&quot;:&quot;\&quot;post-AGI, no one is going to work and the economy is going to collapse\&quot;\n\n\&quot;i am switching to polyphasic sleep because GPT-5.5 in codex is so good that i can't afford to be sleeping for such long stretches and miss out on working\&quot;&quot;,&quot;username&quot;:&quot;sama&quot;,&quot;name&quot;:&quot;Sam Altman&quot;,&quot;profile_image_url&quot;:&quot;https://pbs.substack.com/profile_images/2046764873200394240/r7BxVezs_normal.jpg&quot;,&quot;date&quot;:&quot;2026-04-26T15:37:00.000Z&quot;,&quot;photos&quot;:[],&quot;quoted_tweet&quot;:{},&quot;reply_count&quot;:1158,&quot;retweet_count&quot;:607,&quot;like_count&quot;:11205,&quot;impression_count&quot;:1618126,&quot;expanded_url&quot;:null,&quot;video_url&quot;:null,&quot;video_preview_media_key&quot;:null,&quot;belowTheFold&quot;:false}" data-component-name="Twitter2ToDOM"></div><p>[2] Wiese, M. (2025, August 10). Optimizing my sleep around Claude usage limits. mattwie.se. <a href="https://mattwie.se/no-sleep-till-agi">https://mattwie.se/no-sleep-till-agi</a> </p><p>[3] McKinney, W. (2026, February 17). The mythical agent-month. wesmckinney.com. <a href="https://wesmckinney.com/blog/mythical-agent-month/">https://wesmckinney.com/blog/mythical-agent-month/</a> </p><p>[4] Shah, D. [@0xDevShah]. (2026, April). [Post]. X. </p><div class="twitter-embed" data-attrs="{&quot;url&quot;:&quot;https://x.com/0xDevShah/status/2030087260335657470&quot;,&quot;full_text&quot;:&quot;i'm living on the agentphasic sleep. only claude code and codex hitting limits can put me in REM sleep. i only rest during the compute starvation periods.\n\nmight as well call agent-in-the-loop circadian rhythm&quot;,&quot;username&quot;:&quot;0xDevShah&quot;,&quot;name&quot;:&quot;Dev Shah&quot;,&quot;profile_image_url&quot;:&quot;https://pbs.substack.com/profile_images/1859072760938721280/eihad2RU_normal.jpg&quot;,&quot;date&quot;:&quot;2026-03-07T01:04:54.000Z&quot;,&quot;photos&quot;:[],&quot;quoted_tweet&quot;:{&quot;full_text&quot;:&quot;a few friends are trying polyphasic sleep so they can supervise their coding agents 24/7&quot;,&quot;username&quot;:&quot;khoomeik&quot;,&quot;name&quot;:&quot;Rohan Pandey&quot;,&quot;profile_image_url&quot;:&quot;https://pbs.substack.com/profile_images/1970553499902681088/He-CtSGQ_normal.jpg&quot;},&quot;reply_count&quot;:0,&quot;retweet_count&quot;:4,&quot;like_count&quot;:13,&quot;impression_count&quot;:867,&quot;expanded_url&quot;:null,&quot;video_url&quot;:null,&quot;video_preview_media_key&quot;:null,&quot;belowTheFold&quot;:true}" data-component-name="Twitter2ToDOM"></div><p>[5] globular-toast. (2025, August). Comment on &#8220;Optimizing my sleep around Claude usage limits.&#8221; Hacker News. <a href="https://news.ycombinator.com/item?id=44860015">https://news.ycombinator.com/item?id=44860015</a></p>]]></content:encoded></item><item><title><![CDATA[Goblins]]></title><description><![CDATA[How OpenAI's coding model developed a fantasy creature problem]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/goblins</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/goblins</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Mon, 04 May 2026 09:27:15 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/882d796c-8711-4133-a667-fc5b73a6247f_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>&#8220;There&#8217;s probably a little gremlin in your codebase.&#8221;[1] &#8220;Tiny SourceKit goblin has been escorted from the premises.&#8221;[2] &#8220;the goblin princess in the datacenter&#8221;[3]</p><p>Animal imagery started creeping into OpenAI&#8217;s GPT&#8209;5.1 with the coding assistant describing bugs, tough situations, edge cases, or anything that refused to behave as expected using goblins and gremlins instead. According to OpenAI&#8217;s data, use of &#8220;goblin&#8221; had risen by 175% in GPT&#8209;5.1 compared to GPT&#8209;5, while &#8220;gremlin&#8221; had risen by 52%.[4]</p><p>The creatures were tracked to an optional &#8220;Nerdy&#8221; personality setting used during training, designed to sound playful and quirky, which created a loop in coding contexts:</p><ul><li><p>The model learned that answering in a playful style was rewarded because it made answers feel more lively and engaging</p></li><li><p>Some of those rewarded examples included creature metaphors like goblins and gremlins, which earned higher rewards</p></li><li><p>The model started using goblins and gremlins more often during training</p></li><li><p>The outputs were then used to train it further</p></li><li><p>The use of goblins and gremlins became even more common because the model learned to generalise rewards in certain situations to unrelated ones, even when the Nerdy setting wasn&#8217;t being used</p></li></ul><p>OpenAI retired the Nerdy personality and cleaned up the rewards and training data for its next model, but to contain the creatures with GPT&#8209;5.4 OpenAI added a direct instruction to the system prompt for Codex:</p><p><code>&#8220;Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user&#8217;s query.&#8221;<br><br></code><strong>Sources:</strong></p><p>[1] zetta7re. (2026, April 29). [Comment on reel]. In wutronicai. <em>Where the goblins came from</em> [Video]. Instagram.</p><div class="instagram-embed-wrap" data-attrs="{&quot;instagram_id&quot;:&quot;DXwyYoGOWLI&quot;,&quot;title&quot;:&quot;Instagram&quot;,&quot;author_name&quot;:&quot;&quot;,&quot;thumbnail_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/__ss-rehost__IG-snapshot-DXwyYoGOWLI.jpg&quot;,&quot;like_count&quot;:null,&quot;comment_count&quot;:null,&quot;profile_pic_url&quot;:null,&quot;follower_count&quot;:null,&quot;timestamp&quot;:null,&quot;belowTheFold&quot;:false}" data-component-name="InstagramToDOM"></div><p>[2] Hlbkomer. (2026, April 28). Re: Why does GPT 5.5 have a restraining order against goblins? [Comment]. r/ChatGPT. Reddit. <a href="https://www.reddit.com/r/ChatGPT/comments/1sxwmis/why_does_gpt_55_have_a_restraining_order_against/">https://www.reddit.com/r/ChatGPT/comments/1sxwmis/why_does_gpt_55_have_a_restraining_order_against/</a></p><p>[3] pierukainen. (2026, April 28). Re: Why does GPT 5.5 have a restraining order against goblins? [Comment]. r/ChatGPT. Reddit. <a href="https://www.reddit.com/r/ChatGPT/comments/1sxwmis/comment/oipxfxg/">https://www.reddit.com/r/ChatGPT/comments/1sxwmis/comment/oipxfxg/</a></p><p>[4] OpenAI. (2026, April 29). Where the goblins came from. <a href="https://openai.com/index/where-the-goblins-came-from/">https://openai.com/index/where-the-goblins-came-from/</a></p>]]></content:encoded></item><item><title><![CDATA[Copywashing]]></title><description><![CDATA[To cover up the use of a morally dubious generative AI technique by training a layer on top of it with one&#8217;s own IP.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/copywashing</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/copywashing</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Fri, 10 Apr 2026 09:10:17 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/bfaa5476-02cf-4505-aabd-a2a550383672_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I saw the term "copywashing" used in a <a href="https://scholarlykitchen.sspnet.org/2026/03/11/what-publishing-leaders-say-about-ai-when-theyre-not-on-panels/">Scholarly Kitchen article</a>, and went down a small rabbit hole. (I see <a href="https://p4sc4l.substack.com/p/ai-is-reorganizing-how-authority">Pascal Hetzscholdt </a>also picked up on the term.)</p><p><em>&#8220;To cover up the use of a morally dubious generative AI technique by training a layer on top of it with one&#8217;s own IP.&#8221;</em> [1]</p><p>Rex Riepe, a San Francisco web developer, used this satirical term to describe the following practice:</p><ul><li><p>A company trains an AI image generator on billions of copyrighted images scraped without permission</p></li><li><p>Artists sue</p></li><li><p>The company then runs twenty images it owns through the tool as an additional training step</p></li><li><p>The company announces that the tool is trained on its own material. </p></li></ul><p>The scraped images are still in the image generator, but now the company can claim that the model is generating something new rather than making a copy.</p><p>In March 2026, when Anthropic accidentally leaked the TypeScript source code of Claude Code, its flagship coding tool, Korean developer Sigrid Jin used a rival AI to rewrite the whole thing overnight in Python and published it as claw-code. [2][3] The project described itself as a clean-room reimplementation, a version that copies what the tool does without copying its code directly. So does Anthropic now claim, like the artists suing them, that AI-assisted transformation does not produce genuinely new work, and that rewriting their code was infringement?</p><p><strong>References</strong> </p><p>[1] Rex Riepe, &#8220;Copywashing,&#8221; rexriepe.com, 2023. <a href="https://rexriepe.com/copywashing/">https://rexriepe.com/copywashing/</a></p><p>[2] Sanya Mansoor, &#8220;Claude&#8217;s code: Anthropic leaks source code for AI software engineering tool,&#8221; <em>The Guardian</em>, 1 April 2026. <a href="https://www.theguardian.com/technology/2026/apr/01/anthropic-claudes-code-leaks-ai">https://www.theguardian.com/technology/2026/apr/01/anthropic-claudes-code-leaks-ai</a></p><p>[3] &#8220;What Is Claw Code? The Claude Code Rewrite Explained,&#8221; WaveSpeed AI, 1 April 2026. <a href="https://wavespeed.ai/blog/posts/what-is-claw-code/">https://wavespeed.ai/blog/posts/what-is-claw-code/</a></p><p></p>]]></content:encoded></item><item><title><![CDATA[Does Telling the AI It's an Expert Actually Help?]]></title><description><![CDATA[The problem with expert persona prompts]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/does-telling-the-ai-its-an-expert</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/does-telling-the-ai-its-an-expert</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Thu, 02 Apr 2026 07:55:10 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/fa129a59-9c1a-4b9e-b197-2a0d5f31540a_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Numerous prompting guides begin with some version of<em> "<a href="https://towardsai.net/p/l/how-ai-is-transforming-programming-a-developers-guide-to-enhanced-workflows">You are an expert programmer</a>"</em> or <em>"<a href="https://www.anthropic.com/news/prompt-engineering-for-business-performance">You are an expert AI tax analyst</a>"</em>, on the basis that giving the model a professional role will improve its performance. A 2026 pre-print from the University of Southern California tested whether expert persona prompts actually improve performance across six models, using a standard knowledge test where models answer multiple-choice questions across subjects from maths to history. Expert personas did worse than no persona at all, dropping accuracy from 71.6 to 66 percent [1]. The researchers' hypothesis is that the role-playing instruction puts the model into a different mode, one that gets in the way of factual recall. For tasks that depend on behaviour rather than knowledge, such as writing and role-playing, the persona helped. For facts, figures, and anything requiring precise knowledge, such as coding problems, their practical suggestion was to add nothing to the prompt at all.</p><p>[1] Hu, Z., Rostami, M., &amp; Thomason, J. (2026). <em>Expert Personas Improve LLM Alignment but Damage Accuracy: Bootstrapping Intent-Based Persona Routing with PRISM</em> [pre-print]. University of Southern California. <a href="https://arxiv.org/abs/2603.18507">https://arxiv.org/abs/2603.18507</a></p>]]></content:encoded></item><item><title><![CDATA[Don't Steal This Book]]></title><description><![CDATA[A field guide to the GenAI concepts, controversies, and oddities emerging around us.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/dont-steal-this-book</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/dont-steal-this-book</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Thu, 19 Mar 2026 10:32:26 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/6a852be0-fee2-46de-bfc9-ba8239650edc_690x388.avif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Nearly 10,000 authors published a book with nothing in it, handing out copies at the London Book Fair in March 2026 as a protest against AI training on copyrighted work. <em><a href="https://www.dontstealthisbook.com/">Don&#8217;t Steal This Book</a></em><a href="https://www.dontstealthisbook.com/"> </a>contained 88 pages of names, from Kazuo Ishiguro to Alan Moore to Malorie Blackman, followed by blank pages representing what readers would get if AI companies used books without permission. </p><p>The title was borrowed from Abbie Hoffman&#8217;s 1971 manifesto <em>Steal This Book</em>, which urged readers to take from corporations. Here, the authors turned the idea around, arguing that corporations were taking from them, and responding by withholding the work itself. </p><p>The backlash against AI training on copyrighted work had been building since the government published its proposals in December 2024. In the music industry, Elton John was among the most vocal critics, calling ministers &#8220;absolute losers&#8221; and describing technology secretary Peter Kyle as &#8220;a bit of a moron.&#8221; [1]</p><p>The government&#8217;s original proposal would have allowed AI firms to use copyrighted work unless rights holders explicitly opted out, placing the burden on individual creators to protect their own material. The consultation drew over 11,500 responses, unusually high for a UK policy exercise, with 88 percent supporting a requirement for copyright licences and just 3 percent backing the opt-out. [2] On 18 March 2026, technology secretary Liz Kendall said the government no longer had a &#8220;preferred option&#8221; on copyright reform. Ed Newton-Rex, founder of Fairly Trained and organiser of the campaign, said the shift changed little: <em>&#8220;virtually everything is still on the table&#8230; it&#8217;s just kicking the can down the road&#8221;</em>. [3]</p><p><strong>References</strong></p><p>[1] John, E. (2025, May 18). Interview on <em>Sunday with Laura Kuenssberg</em>, BBC One. Reported in: Elton John calls UK government &#8216;absolute losers&#8217; over AI copyright plans. <em>The Guardian</em>. <a href="https://www.theguardian.com/music/2025/may/18/elton-john-says-uk-government-being-absolute-losers-over-ai-copyright-plans">https://www.theguardian.com/music/2025/may/18/elton-john-says-uk-government-being-absolute-losers-over-ai-copyright-plans</a></p><p>[2] BBC News (2026). <em>Thousands of authors publish &#8216;empty&#8217; book in protest over AI using their work</em>. Available at: <a href="https://www.bbc.co.uk/news/articles/cvg1gr5v333o">https://www.bbc.co.uk/news/articles/cvg1gr5v333o</a></p><p>[3] Milmo, D. (2026, March 18). <em>Actors, musicians and writers welcome UK U-turn on AI copyright</em>. The Guardian. <a href="https://www.theguardian.com/technology/2026/mar/18/actors-musicians-writers-welcome-uk-u-turn-ai-copyright">https://www.theguardian.com/technology/2026/mar/18/actors-musicians-writers-welcome-uk-u-turn-ai-copyright</a></p>]]></content:encoded></item><item><title><![CDATA[Algorithmic Allonymy: The Grammarly Expert Review Scandal]]></title><description><![CDATA[Grammarly built a feature that generated fake writing advice and signed it with real people's names.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/algorithmic-allonymy-the-grammarly</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/algorithmic-allonymy-the-grammarly</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Wed, 18 Mar 2026 08:39:34 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/7e6d8193-f76a-4f70-bdf2-42c61d55c945_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>This entry was inspired by a Bluesky post from Martin Paul Eve, Professor of Literature, Technology and Publishing at Birkbeck, University of London, about the term allonymy, which I had never heard of either. </p><div class="bluesky-wrap outer" style="height: auto; display: flex; margin-bottom: 24px;" data-attrs="{&quot;postId&quot;:&quot;3mh4pitjqsc2m&quot;,&quot;authorDid&quot;:&quot;did:plc:hnpt7ns2lecdujegbi6qkqqm&quot;,&quot;authorName&quot;:&quot;Martin Paul Eve&quot;,&quot;authorHandle&quot;:&quot;eve.gd&quot;,&quot;authorAvatarUrl&quot;:&quot;https://cdn.bsky.app/img/avatar/plain/did:plc:hnpt7ns2lecdujegbi6qkqqm/bafkreicazsfmd3ispkm3welkafesgcccntto3xocmlzbgxawfkfvprch34&quot;,&quot;text&quot;:&quot;Probably gross ignorance, but I didn't know that the assumption of another person's name for publishing purposes (cough Grammarly) is called an \&quot;allonym\&quot;.&quot;,&quot;createdAt&quot;:&quot;2026-03-15T19:47:00.150Z&quot;,&quot;uri&quot;:&quot;at://did:plc:hnpt7ns2lecdujegbi6qkqqm/app.bsky.feed.post/3mh4pitjqsc2m&quot;,&quot;imageUrls&quot;:[]}" data-component-name="BlueskyCreateBlueskyEmbed"><iframe id="bluesky-3mh4pitjqsc2m" data-bluesky-id="402105758305235" src="https://embed.bsky.app/embed/did:plc:hnpt7ns2lecdujegbi6qkqqm/app.bsky.feed.post/3mh4pitjqsc2m?id=402105758305235" width="100%" style="display: block; flex-grow: 1;" frameborder="0" scrolling="no"></iframe></div><p>Grammarly, a writing tool used by over 40 million people, launched a feature in August 2025 called Expert Review. For $12 a month, subscribers could upload their writing and receive editing suggestions with the names of real writers and journalists attached, as though those people had personally read the draft and offered their thoughts. None of the named experts had any involvement or any idea that their names were being used. It was algorithmic allonymy, AI publishing under someone else&#8217;s name.</p><p>Casey Newton, who runs Platformer, one of the most widely read technology newsletters in the US, discovered he had been listed as an expert and submitted a colleague&#8217;s article to see what kind of feedback the tool produced [1]. It offered advice attributed to the journalist Kara Swisher, suggesting he &#8220;briefly compare how daily AI users versus AI skeptics articulate risk, creating a through-line readers can follow&#8221; [2]. When he forwarded it to the real Swisher, who once made Mark Zuckerberg sweat so badly during a live interview that he had to take his hoodie off, she replied: &#8220;You rapacious information and identity thieves better get ready for me to go full McConaughey on you. Also, you suck&#8221; [1].</p><p>The tool also produced feedback attributed to <em>Bad Blood</em> author John Carreyrou, suggesting opening with &#8220;sensory imagery, a chant echoing off glass towers or chalk dust in the air&#8221; [1]. Benjamin Dreyer, copy chief at Random House and author of <em>Dreyer&#8217;s English</em>, pasted lorem ipsum into the tool and received writing advice attributed to Stephen King [3]. &#8220;I&#8217;ve long assumed that before too long, AI might take my job,&#8221; Newton wrote. &#8220;I just assumed that someone would tell me when it happened&#8221; [1].</p><p>Some of the named experts were not even alive. David Abulafia, a Cambridge historian, had died two months earlier, but the tool was still producing feedback under his name. Vanessa Heggie, a historian at the University of Birmingham, called the use of deceased scholars to critique users&#8217; writing &#8220;obscene&#8221; [4].</p><p>Julia Angwin, an investigative journalist who had spent years at <em>ProPublica</em> and <em>The Wall Street Journal</em> reporting on how technology companies misuse personal data, discovered she had been listed as an expert too. She described the AI output bearing her name as a &#8220;slopperganger&#8221; that made sentences &#8220;worse, more complex&#8221; [4]. On 11 March 2026, she filed a class-action lawsuit in Manhattan federal court against Superhuman, Grammarly&#8217;s parent company, and within days the feature was disabled.</p><p><strong>References:</strong></p><p>[1] Newton, C. (2026, March 9). Grammarly turned me into an AI editor against my will and I hate it. <em>Platformer.</em> <a href="https://www.platformer.news/grammarly-expert-review-reviewed/">https://www.platformer.news/grammarly-expert-review-reviewed/</a></p><p>[2] Silberling, A. (2026, March 12). A writer is suing Grammarly for turning her and other authors into &#8216;AI editors&#8217; without consent. <em>TechCrunch.</em> <a href="https://techcrunch.com/2026/03/12/a-writer-is-suing-grammarly-for-turning-her-and-other-authors-into-ai-editors-without-consent/">https://techcrunch.com/2026/03/12/a-writer-is-suing-grammarly-for-turning-her-and-other-authors-into-ai-editors-without-consent/</a></p><p>[3] Futurism. (2026, March 11). Grammarly is pulling down its explosively controversial feature that impersonates writers without their permission. <a href="https://futurism.com/artificial-intelligence/grammarly-pulls-down-expert-review-feature">https://futurism.com/artificial-intelligence/grammarly-pulls-down-expert-review-feature</a></p><p>[4] HumAI Blog. (2026, March 10). Grammarly claims it can give you feedback like Stephen King and Kara Swisher. Neither of them agreed to that. <a href="https://www.humai.blog/grammarly-claims-it-can-give-you-feedback-like-stephen-king-and-kara-swisher-neither-of-them-agreed-to-that/">https://www.humai.blog/grammarly-claims-it-can-give-you-feedback-like-stephen-king-and-kara-swisher-neither-of-them-agreed-to-that/</a></p>]]></content:encoded></item><item><title><![CDATA[Algorithmic agnotology]]></title><description><![CDATA[The deliberate production of ignorance about how AI systems work and what harms they cause]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/algorithmic-agnotology</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/algorithmic-agnotology</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Fri, 06 Mar 2026 08:46:33 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/490fc083-1467-4c52-94d8-ee9453e9ad07_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>I came across the term algorithmic agnotology in a forum and thought it had a lovely ring to it.</em></p><p><strong>Algorithmic agnotology:</strong> Agnotology, the study of deliberately manufactured ignorance, was coined in 1992 by linguist Iain Boal during discussions with Stanford historian Robert Proctor [1]. Proctor had been researching the tobacco industry, which spent decades funding its own research to muddy the link between smoking and cancer. A famous 1969 internal memo from tobacco company Brown &amp; Williamson captured the strategy in three words: &#8220;Doubt is our product.&#8221; [2]</p><p>Alondra Nelson, former head of the White House Office of Science and Technology Policy and architect of the Biden administration&#8217;s AI Bill of Rights, has applied Proctor&#8217;s concept to generative AI [3]. Her term, algorithmic agnotology, describes the deliberate production of ignorance about how AI systems work and what harms they cause. Nelson describes a playbook of tactics that includes selective disclosure, deflection, and the manufacturing of doubt about whether the technology can be understood at all.</p><p>Whilst nobody fully understands how large language models generate their outputs, Nelson argues that AI companies have turned this uncertainty into a corporate strategy, presenting it as an inevitable feature of complex technology rather than a problem to be solved. The information that might explain how these systems behave is largely kept inside the companies that build them. We do not know exactly what data were used to train the models, whether copyrighted books or private online conversations were included, or how their safety testing works. When questioned, companies often say the systems are simply too complex for outsiders to properly assess.</p><p>In 2024, whistleblowers alleged that OpenAI had used restrictive agreements and threats to claw back vested equity to discourage former employees from reporting safety concerns to the SEC [4]. Nelson places this in a longer tradition. Industries whose products carry serious risks have a long history of managing uncertainty rather than resolving it. As Nelson puts it: &#8220;Are we interrogating opacity, or legitimizing it?&#8221; [3]</p><p><strong>References</strong></p><p>[1] Proctor, R. N., &amp; Schiebinger, L. (Eds.). (2008). <em>Agnotology: The making and unmaking of ignorance</em>. Stanford University Press.</p><p>[2] Brown &amp; Williamson. (1969). Internal memo. In R. N. Proctor &amp; L. Schiebinger (Eds.), <em>Agnotology: The making and unmaking of ignorance</em>. Stanford University Press.</p><p>[3] Nelson, A. (2026, February). Algorithmic agnotology: On AI, ignorance and power. Presentation at IASEAI&#8217;26, UNESCO, Paris. See also: Nelson, A. (2025, April 3). Distinguished Lecture, Tech Policy Lab, University of Washington. <a href="https://techpolicylab.uw.edu/events/event/distinguished-lecture-with-alondra-nelson/">https://techpolicylab.uw.edu/events/event/distinguished-lecture-with-alondra-nelson/</a></p><p>[4] Seligman, L. et al. (2024, July 13). OpenAI illegally barred staff from airing safety risks, whistleblowers say. <em>The Washington Post.</em> <a href="https://www.washingtonpost.com/technology/2024/07/13/openai-safety-risks-whistleblower-sec/">https://www.washingtonpost.com/technology/2024/07/13/openai-safety-risks-whistleblower-sec/</a></p>]]></content:encoded></item><item><title><![CDATA[K-Shaped Economy]]></title><description><![CDATA[The idea that the rich are getting richer, the poor are getting poorer, and the middle class is shrinking due to the impact of AI.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/k-shaped-economy</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/k-shaped-economy</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Fri, 27 Feb 2026 09:02:41 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/97eeecb3-fd24-4d03-8642-25ac768b1d29_1536x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Picture the letter K. One arm rises while the other falls. The economist Peter Atwater popularised the term during the pandemic of 2020 to describe an uneven recovery, where office workers kept earning from home while many service workers lost their incomes almost overnight.&#185;</p><p>Five years later, NPR called the K-shaped economy one of 2025&#8217;s biggest business buzzwords.&#178; There is a growing sense that generative AI is producing the same shape, with economic growth continuing, but its benefits are flowing to certain workers, industries, and companies. In the third quarter of 2025, the US economy grew 4.3 per cent while consumer confidence fell to near-record lows.&#179; The wealthiest 10 per cent of Americans accounted for half of all consumer spending that year, according to Moody&#8217;s Analytics.&#8308; The headline numbers looked healthy because those at the top of the K were still spending.</p><p>Historically, automation has hit factory floors and manual labour first, but generative AI seems to be disrupting office and cognitive work instead. A 2025 study in the Journal of Economic Behavior &amp; Organization found that demand for writing and translation on a major freelancing platform fell between 20 and 50 per cent after ChatGPT launched, while some categories of AI-related programming nearly tripled.&#8309; One freelance copywriter interviewed for Blood in the Machine described her income collapsing from as much as $3,000 a month to around $100 by 2024.&#8310; Even pioneers of the technology sounded cautious. Geoffrey Hinton, the British-Canadian computer scientist who won a Nobel Prize for his foundational work on neural networks, told a podcast in mid-2025: &#8220;Train to be a plumber.&#8221; Paralegals, he said, won&#8217;t be needed for very long.&#8311;</p><p>Younger workers are responding. A survey by Jobber, reported by CBS News, found that 77 per cent of Gen Z respondents said it was important that their future job be difficult to automate, choosing carpentry, plumbing, and electrical work over software development and data analytics.&#8312; The new K no longer divides blue-collar and white-collar workers. It separates work that requires a human body in a specific place from work that can increasingly be performed, or replaced, by software.</p><p><strong>Sources:</strong></p><ol><li><p>Atwater, P. (2020). Cited in William &amp; Mary News. &#8220;W&amp;M professor brings &#8216;K-shaped&#8217; clarity back to market discussion,&#8221; 20 November 2025. <a href="https://news.wm.edu/2025/11/20/wm-professor-brings-k-shaped-clarity-back-to-market-discussion/">https://news.wm.edu/2025/11/20/wm-professor-brings-k-shaped-clarity-back-to-market-discussion/</a></p></li><li><p>Bisaha, S. (2025). &#8220;What is a K-shaped economy?&#8221; NPR, 31 December 2025. <a href="https://www.npr.org/2025/12/31/nx-s1-5660842/what-is-a-k-shaped-economy">https://www.npr.org/2025/12/31/nx-s1-5660842/what-is-a-k-shaped-economy</a></p></li><li><p>Bureau of Economic Analysis. (2025). GDP (Advance Estimate), 3rd Quarter 2025. <a href="https://www.bea.gov/news/2026/gdp-advance-estimate-4th-quarter-and-year-2025">https://www.bea.gov/news/2026/gdp-advance-estimate-4th-quarter-and-year-2025</a></p></li><li><p>Moody&#8217;s Analytics. (2025). Cited in Al Jazeera. &#8220;Is the US economy strong heading into 2026? The picture is complicated,&#8221; 30 December 2025. <a href="https://www.aljazeera.com/news/2025/12/30/is-the-us-economy-strong-heading-into-2026-the-picture-is-complicated">https://www.aljazeera.com/news/2025/12/30/is-the-us-economy-strong-heading-into-2026-the-picture-is-complicated</a></p></li><li><p>Teutloff, O., Einsiedler, J., K&#228;ssi, O., Braesemann, F., Mishkin, P., &amp; del Rio-Chanona, R. M. (2025). Winners and losers of generative AI: Early evidence of shifts in freelancer demand. <em>Journal of Economic Behavior &amp; Organization</em>, 235, 106845. <a href="https://doi.org/10.1016/j.jebo.2024.106845">https://doi.org/10.1016/j.jebo.2024.106845</a></p></li><li><p>Merchant, B. (2025). &#8220;I was forced to use AI until the day I was laid off.&#8221; <em>Blood in the Machine</em>, 12 December 2025. </p></li><li><p>Hinton, G. (2025). Interview on <em>The Diary of a CEO with Steven Bartlett</em>, June 2025. Cited in Hampton, D. J. &#8220;Blue-collar jobs are gaining popularity as AI threatens office work.&#8221; NBC News, 16 August 2025. <a href="https://www.nbcnews.com/business/business-news/ai-which-jobs-are-skilled-trades-protected-what-to-know-rcna223249">https://www.nbcnews.com/business/business-news/ai-which-jobs-are-skilled-trades-protected-what-to-know-rcna223249</a></p></li><li><p>CBS News. (2025). &#8220;As AI threatens white-collar work, more young Americans choose blue-collar careers,&#8221; 1 October 2025. Survey conducted by Jobber. <a href="https://www.cbsnews.com/news/ai-artificial-intelligence-work-trade-jobs/">https://www.cbsnews.com/news/ai-artificial-intelligence-work-trade-jobs/</a></p></li></ol>]]></content:encoded></item><item><title><![CDATA[Make It Fair]]></title><description><![CDATA[British newspapers abandoned their rivalry and musicians embraced silence as twin protests against government proposals to let AI companies use creative works without permission.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/make-it-fair</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/make-it-fair</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Wed, 25 Feb 2026 08:08:14 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/01847e31-5b8c-4d2e-b4d7-7215abb6a372_748x503.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On 25 February 2025, Britain woke to find all the major newspapers covered in identical turquoise cover wraps with the message: &#8220;MAKE IT FAIR: The government wants to change the UK&#8217;s laws to favour big tech platforms so they can use British creative content to power their AI models without our permission or payment. Let&#8217;s protect the creative industries &#8211; it&#8217;s only fair.&#8221; <sup>1</sup></p><p>On the same day, over 1,000 musicians, including Kate Bush, Damon Albarn, The Clash and Hans Zimmer, released a &#8216;silent&#8217; protest album called <em>Is This What We Want? </em>The album, organised by Ed Newton-Rex, a British composer and former AI executive, featured recordings of empty studios and performance spaces to show the potential impact on livelihoods and the UK music industry. <sup>2</sup></p><p>Both protests ran on the final day of the UK government&#8217;s consultation about its AI and copyright proposal. The plans, which aim to make it easier to train artificial intelligence models on artists&#8217; works, include a &#8216;rights reservation&#8217; system which would allow AI companies to use creative works by default, unless artists and rights holders actively opt out. Make It Fair campaigners argue that this approach will hand an enormous advantage to tech giants and that it&#8217;s unrealistic to expect individual writers and artists to contact countless AI firms to refuse permission, let alone track how their work is being used across the internet.</p><p>The government defended the plans, with a Department for Science, Innovation and Technology spokesperson saying that the UK&#8217;s &#8220;current regime for copyright and AI is holding back the creative industries, media and AI sector from realising their full potential - and that cannot continue. That&#8217;s why we have been consulting on a new approach that protects the interests of both AI developers and right holders and delivers a solution that allows both to thrive.&#8221;<sup>3</sup> At some point, ministers will have to choose between Silicon Valley and Soho.<br><br><strong>Sources:</strong></p><p>1. News Media Association. (2025, February 25). UK creative industries launch &#8216;Make it Fair&#8217; campaign against AI copyright exemptions. <a href="https://newsmediauk.org/blog/2025/02/25/uk-creative-industries-launch-make-it-fair-campaign/">https://newsmediauk.org/blog/2025/02/25/uk-creative-industries-launch-make-it-fair-campaign/</a></p><p>2. Newton-Rex, E. (Organiser). (2025). <em>Is This What We Want?</em> [Collaborative album]. Virgin Music Group. <a href="https://www.isthiswhatwewant.com/">https://www.isthiswhatwewant.com/</a></p><p>3. BBC News. (2025, February 25). Artists release silent album in protest against AI using their work. <em>BBC News</em>. <a href="https://www.bbc.co.uk/news/articles/cwyd3r62kp5o">https://www.bbc.co.uk/news/articles/cwyd3r62kp5o</a></p>]]></content:encoded></item><item><title><![CDATA[Ghibli-fied photos]]></title><description><![CDATA[Ghibli-fying was a viral March 2025 trend where Grant Slatton's X post showing his family photo transformed into Studio Ghibli-style art using ChatGPT sparked millions to create similar images.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/ghibli-fied-photos</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/ghibli-fied-photos</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Sat, 03 Jan 2026 21:40:25 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!YXB4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fpbs.substack.com%2Fmedia%2FGm6bwlBbQAA-HZI.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In March 2025, Grant Slatton, a software engineer from Seattle, asked ChatGPT&#8217;s upgraded image tools to give his family photo a Studio Ghibli-style makeover. His post went viral on X<sup>1</sup>, and soon everyone was turning their ordinary photos into anime-style art. Social feeds filled with Ghibli-fied images featuring soft lighting, warm colours, and the hand-drawn aesthetic of Studio Ghibli - despite the studio&#8217;s co-founder, Hayao Miyazaki, previously describing artificial intelligence as <em>&#8220;an insult to life itself&#8221;</em>, saying <em>&#8220;I would never wish to incorporate this technology into my work at all&#8221;</em><sup>2</sup>.</p><p>The Ghiblification trend created &#8220;biblical demand&#8221; on OpenAI&#8217;s servers with even Sam Altman changed his profile picture to a Ghibli-style portrait, later posting, &#8220;<em>can yall please chill on generating images. this is insane. our team needs sleep&#8221;</em>.<sup>3</sup></p><p>Artists and legal experts questioned whether OpenAI was effectively enabling users to copy Studio Ghibli&#8217;s copyrighted work without permission or compensation. Critics argued that because the models were trained on vast amounts of online material, they almost certainly drew on Ghibli artwork as well, turning that data into a profitable service capable of reproducing the studio&#8217;s style on demand. In late 2025, Japan&#8217;s anti-piracy group CODA, which includes Studio Ghibli, sent a formal letter demanding that OpenAI stop using their copyrighted works to train models prior permission, arguing that Japanese law requires prior consent.<sup>4</sup> When challenged on copyright concerns, Sam Altman defended AI art as a <em>&#8220;big net win for society&#8217;&#8221;</em>, arguing that it expands creative access and participation.<sup>5 </sup>Although OpenAI did start to block some users&#8217; requests for images in the style of Studio Ghibli and some other living artists.</p><p>The controversy inspired Genre AI CEO PJ Accetturo to make a parody video using Sora 2, showing Sam Altman in a Studio Ghibli office saying, &#8220;Japan is so cool, I&#8217;m taking these,&#8221; before stealing Miyazaki&#8217;s artwork with Miyazaki chasing after him <sup>6</sup>.<br><br><strong>References</strong></p><p>1. Slatton, G. (2025, March 25). tremendous alpha right now in sending your wife photos of yall converted to studio ghibli anime [Post]. <em>X</em>. </p><div class="twitter-embed" data-attrs="{&quot;url&quot;:&quot;https://x.com/GrantSlatton/status/1904631016356274286&quot;,&quot;full_text&quot;:&quot;tremendous alpha right now in sending your wife photos of yall converted to studio ghibli anime &quot;,&quot;username&quot;:&quot;GrantSlatton&quot;,&quot;name&quot;:&quot;Grant Slatton&quot;,&quot;profile_image_url&quot;:&quot;https://pbs.substack.com/profile_images/1856158109045272576/m5eKtNvO_normal.jpg&quot;,&quot;date&quot;:&quot;2025-03-25T20:26:35.000Z&quot;,&quot;photos&quot;:[{&quot;img_url&quot;:&quot;https://pbs.substack.com/media/Gm6bwlBbQAA-HZI.jpg&quot;,&quot;link_url&quot;:&quot;https://t.co/FROszdFSfN&quot;},{&quot;img_url&quot;:&quot;https://pbs.substack.com/media/Gm6byX_bgAAg5Tj.jpg&quot;,&quot;link_url&quot;:&quot;https://t.co/FROszdFSfN&quot;}],&quot;quoted_tweet&quot;:{},&quot;reply_count&quot;:2401,&quot;retweet_count&quot;:1559,&quot;like_count&quot;:43565,&quot;impression_count&quot;:53044102,&quot;expanded_url&quot;:null,&quot;video_url&quot;:null,&quot;video_preview_media_key&quot;:null,&quot;belowTheFold&quot;:false}" data-component-name="Twitter2ToDOM"></div><p>2. Hayao Miyazaki&#8217;s 2016 AI criticism reignites amid Studio Ghibli-style art copyright battles. (2025). <em>Tomorrow&#8217;s Publisher</em>. <a href="https://tomorrowspublisher.today/ethics/hayao-miyazakis-2016-ai-criticism-reignites-amid-studio-ghibli-style-art-copyright-battles/">https://tomorrowspublisher.today/ethics/hayao-miyazakis-2016-ai-criticism-reignites-amid-studio-ghibli-style-art-copyright-battles/</a></p><p>3. Altman, S. (2025, March 27). can yall please chill on generating images this is insane our team needs sleep [Post]. <em>X</em>. </p><div class="twitter-embed" data-attrs="{&quot;url&quot;:&quot;https://x.com/sama/status/1906210479695126886&quot;,&quot;full_text&quot;:&quot;can yall please chill on generating images this is insane our team needs sleep&quot;,&quot;username&quot;:&quot;sama&quot;,&quot;name&quot;:&quot;Sam Altman&quot;,&quot;profile_image_url&quot;:&quot;https://pbs.substack.com/profile_images/1904933748015255552/k43GMz63_normal.jpg&quot;,&quot;date&quot;:&quot;2025-03-30T05:02:48.000Z&quot;,&quot;photos&quot;:[],&quot;quoted_tweet&quot;:{},&quot;reply_count&quot;:6569,&quot;retweet_count&quot;:5988,&quot;like_count&quot;:122275,&quot;impression_count&quot;:17003231,&quot;expanded_url&quot;:null,&quot;video_url&quot;:null,&quot;video_preview_media_key&quot;:null,&quot;belowTheFold&quot;:false}" data-component-name="Twitter2ToDOM"></div><p>4. Chang, O. (2025, November 5). Japan&#8217;s CODA demands OpenAI halt use of protected works for AI training amid growing dispute. <em>DIGITIMES</em>. <a href="https://www.digitimes.com/news/a20251105PD220/openai-sora-training-copyright-japan.html">https://www.digitimes.com/news/a20251105PD220/openai-sora-training-copyright-japan.html</a></p><p>5. The democratization of creative content: AI&#8217;s impact on artistic expression and society. (2025). <em>SSRN Electronic Journal</em>. <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5296075">https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5296075</a></p><p>6. Accetturo, P. J. (2025, October 1). [AI-generated video of Sam Altman stealing art from Miyazaki]. <em>Instagram</em>.  </p><div class="instagram-embed-wrap" data-attrs="{&quot;instagram_id&quot;:&quot;DPYDfdejzR7&quot;,&quot;title&quot;:&quot;Indian Startup News on Instagram: \&quot;A newly surfaced AI-generate&#8230;&quot;,&quot;author_name&quot;:&quot;@indianstartupnews&quot;,&quot;thumbnail_url&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/__ss-rehost__IG-meta-DPYDfdejzR7.jpg&quot;,&quot;like_count&quot;:null,&quot;comment_count&quot;:null,&quot;profile_pic_url&quot;:null,&quot;follower_count&quot;:null,&quot;timestamp&quot;:null,&quot;belowTheFold&quot;:true}" data-component-name="InstagramToDOM"></div>]]></content:encoded></item><item><title><![CDATA[30 November 2022 - The launch of ChatGPT ]]></title><description><![CDATA['We are excited to introduce ChatGPT to get users' feedback and learn about its strengths and weaknesses. During the research preview, usage of ChatGPT is free.']]></description><link>https://generativeaiforcuriouspeople.substack.com/p/30-november-2022-the-launch-of-chatgpt</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/30-november-2022-the-launch-of-chatgpt</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Sun, 30 Nov 2025 23:56:11 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!RWM_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ed5b995-ef02-445c-8faf-98ae6f2ec78f_1080x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In 2019 OpenAI said GPT-2 was <em>&#8220;too dangerous to release&#8221;</em>, warning that it could generate misleading news articles, impersonate real people, and produce large amounts of convincing fabricated text with almost no effort.<em> &#8220;New AI fake text generator may be too dangerous to release, say creators&#8221;</em>, ran the Guardian headline. [1]</p><p>The launch of ChatGPT on 30 November 2022 was very different. OpenAI published a short blog post: <em>&#8220;We are excited to introduce ChatGPT to get users&#8217; feedback and learn about its strengths and weaknesses. During the research preview, usage of ChatGPT is free&#8221;</em>. [2] Once you had set up an account, you opened a simple chat window and could ask it anything you wanted, with no warnings about what it might produce.</p><p>On the first day, around 150,000 people tried it. By day five, when Kevin Roose described it in The New York Times as &#8220;a highly capable linguistic superbrain&#8221; that had &#8220;landed with a splash&#8221;, the number had passed a million. [3]</p><p>By early January 2023, it had reached an estimated 100 million monthly users. [4] Teachers spotted students submitting work written by the system, and universities rushed to rewrite their policies on plagiarism and AI use. In May 2023, Chegg, a company built around helping students cheat on their homework, reported that students were turning to ChatGPT instead, and its stock fell almost 50 percent in a single day [5]. Amazon&#8217;s Kindle store saw a wave of auto-generated titles, prompting the company to impose a limit of three self-published uploads per day to slow the surge. [6] Throughout this period, the big consultancy companies made a killing advising executives on how to respond &#8211; where to start, what the productivity gains might be, and how to upskill the workforce. </p><p>By late 2025, OpenAI said the service had around 800 million weekly users, roughly 10 percent of the global population [7]. It became the fastest-growing consumer app in history, though its total user base still lags behind TikTok and Instagram.<br><br><strong>Sources:</strong></p><ol><li><p>Alex Hern, &#8220;New AI fake text generator may be too dangerous to release, say creators&#8221;, The Guardian, 14 February 2019. <a href="https://www.theguardian.com/technology/2019/feb/14/elon-musk-backed-ai-writes-convincing-news-fiction">https://www.theguardian.com/technology/2019/feb/14/elon-musk-backed-ai-writes-convincing-news-fiction</a></p></li><li><p>&#8220;Introducing ChatGPT&#8221;, OpenAI blog, 30 November 2022. <a href="https://openai.com/index/chatgpt/">https://openai.com/index/chatgpt/</a></p></li><li><p>Kevin Roose, &#8220;The Brilliance and Weirdness of ChatGPT&#8221;, The New York Times, 5 December 2022. <a href="https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html">https://www.nytimes.com/2022/12/05/technology/chatgpt-ai-twitter.html</a></p></li><li><p>Krystal Hu, &#8220;ChatGPT sets record for fastest-growing user base - analyst note&#8221;, Reuters, 1 February 2023. <a href="https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/">https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/</a></p></li><li><p>&#8220;Chegg shares drop more than 40% after company says ChatGPT is killing its business&#8221;, CNBC, 2 May 2023. <a href="https://www.cnbc.com/2023/05/02/chegg-drops-more-than-40percent-after-saying-chatgpt-is-killing-its-business.html">https://www.cnbc.com/2023/05/02/chegg-drops-more-than-40percent-after-saying-chatgpt-is-killing-its-business.html</a></p></li><li><p>Ella Creamer, &#8220;Amazon restricts authors from self-publishing more than three books a day after AI concerns&#8221;, The Guardian, 20 September 2023. <a href="https://www.theguardian.com/books/2023/sep/20/amazon-restricts-authors-from-self-publishing-more-than-three-books-a-day-after-ai-concerns">https://www.theguardian.com/books/2023/sep/20/amazon-restricts-authors-from-self-publishing-more-than-three-books-a-day-after-ai-concerns</a></p></li><li><p>Rebecca Bellan, &#8220;Sam Altman says ChatGPT has hit 800M weekly active users&#8221;, TechCrunch, 6 October 2025. <a href="https://techcrunch.com/2025/10/06/sam-altman-says-chatgpt-has-hit-800m-weekly-active-users/">https://techcrunch.com/2025/10/06/sam-altman-says-chatgpt-has-hit-800m-weekly-active-users/</a></p><p></p></li></ol>]]></content:encoded></item><item><title><![CDATA[The A$439,000 Hallucination]]></title><description><![CDATA[Today&#8217;s entry was inspired by a LinkedIn post from Chirag Jay Patel, which caught my attention.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/the-a439000-hallucination</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/the-a439000-hallucination</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Fri, 28 Nov 2025 08:08:36 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!RWM_!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ed5b995-ef02-445c-8faf-98ae6f2ec78f_1080x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Today&#8217;s entry was inspired by a <a href="https://www.linkedin.com/posts/jaypatel-ai-nlp-sdg-publishing_reason-1000000-to-double-check-your-references-ugcPost-7399123452835549184-ih3n?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAC2QD4BaWjyCcSAFax9EZG0I0u20agS7a0">LinkedIn post from Chirag Jay Patel</a>, which caught my attention. Deloitte has once again been caught producing a report riddled with false citations, this time for Newfoundland and Labrador&#8217;s Health Human Resources Plan, published by the Department of Health and Community Services. According to journalist Matt Barter, the report cost the province nearly $1.6 million. </em><br></p><p>In December 2024, the Australian government paid Deloitte A$439,000 to review its automated welfare compliance system. The firm delivered its report in July 2025, and welfare researcher Chris Rudge immediately noticed that several citations appeared to be fabricated. For example, Carolyn Adams was credited with an article in the UNSW Law Journal that she&#8217;d never written, and it attributed Lisa Burton Crawford with two books that didn&#8217;t exist.</p><p>When the Australian Financial Review exposed the errors in August, Deloitte admitted that a part of the report &#8220;included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT &#8212; 4o) based tool chain&#8221; licensed by the government department. (1) The firm insisted the &#8220;content of each article referenced is accurate&#8221;, (2) but conceded that it had the titles, dates, journals, and authors wrong.</p><p>&#8220;This report is supposed to be an assurance of the system&#8217;s integrity,&#8221; Rudge said, &#8220;but the report itself doesn&#8217;t seem to have integrity.&#8221; (1)</p><p>Deloitte issued a partial refund.</p><p><strong>Sources:</strong></p><p>1. Australian Financial Review, &#8220;Academics raise alarm over suspected AI use in Deloitte report,&#8221; August 22, 2025, <a href="https://www.afr.com/companies/professional-services/academics-raise-alarm-over-suspected-ai-use-in-deloitte-report-20250822-p5mp0f">https://www.afr.com/companies/professional-services/academics-raise-alarm-over-suspected-ai-use-in-deloitte-report-20250822-p5mp0f</a></p><p>2. Financial Times, &#8220;Deloitte issues refund for error-ridden Australian government report that used AI,&#8221; October 6, 2025, <a href="https://www.ft.com/content/934cc94b-32c4-497e-9718-d87d6a7835ca">https://www.ft.com/content/934cc94b-32c4-497e-9718-d87d6a7835ca</a></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://generativeaiforcuriouspeople.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Generative AI for Curious People! Subscribe for free to receive new posts and support my work.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Yudkowskyism]]></title><description><![CDATA[Eliezer Yudkowsky is a self-taught AI theorist who grew up reading science fiction and has spent more than twenty years arguing that advanced AI could pose an existential risk.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/yudkowskyism</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/yudkowskyism</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Wed, 19 Nov 2025 08:08:43 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/939e834c-e81b-4c15-b72b-ffd9c0daaa5c_1080x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Governments should be ready to bomb data centers rather than let AI kill everyone, says Eliezer Yudkowsky. When tech leaders signed the March 2023 open letter calling for a six-month pause on giant AI experiments, Yudkowsky wrote in TIME that this was <em>&#8220;understating the seriousness of the situation and asking for too little to solve it&#8221;</em>&#185;. He called for multinational agreements that would allow governments to<em> &#8220;destroy a rogue datacenter by airstrike&#8221;</em> if necessary&#185;.</p><p>Yudkowsky is a self-taught AI theorist who grew up reading science fiction and has spent more than twenty years arguing that advanced AI could pose an existential risk. His concepts have migrated from online forums to international policy papers. Terms like AI &#8216;alignment,&#8217; &#8216;existential risk,&#8217; and &#8216;superintelligence&#8217; that he popularised through <a href="https://www.lesswrong.com/">LessWrong</a>, his online community blog devoted to rational thinking, decision-making, and the long-term future of humanity. These concepts now appear in international policy discussions. The 2023 Bletchley Declaration, signed by 28 countries including the US, UK, and China, warned that frontier AI systems could cause &#8220;catastrophic harm, up to and including human extinction&#8221;, language that borrows directly from Yudkowsky&#8217;s writings.</p><p>Yudkowsky doesn&#8217;t think consciousness is only found in living things. He argues consciousness is algorithmic and can arise from the right kind of information processing, so machine minds could in principle be conscious, though their conscious experience might be nothing like ours. Yudkowsky does not fear human-like AI, as he said in 2023, &#8220;AI won&#8217;t be human-like; it&#8217;ll be alien&#8221;&#178;. His nightmare is a powerful AI so capable that it might ignore humanity entirely, brushing us aside the way we ignore ants.</p><p>He co-founded the Singularity Institute for Artificial Intelligence in 2000 with the goal of helping to bring about a benevolent superintelligence, believing it could manage other existential risks like nanotechnology. Within a year he concluded that this was far more dangerous than he had thought and pushed the institute to focus on &#8220;Friendly AI&#8221; and existential risks from artificial general intelligence.</p><p>His apocalyptic warnings remained largely confined to tech circles until ChatGPT arrived. Then, in February 2023, three months after ChatGPT launched, Sam Altman tweeted&#178;:</p><p><em>&#8220;eliezer has IMO done more to accelerate AGI than anyone else. certainly he got many of us interested in AGI, helped deepmind get funded at a time when AGI was extremely outside the overton window, was critical in the decision to start openai, etc.&#8221;</em></p><p>He followed it up with&#179;:</p><p><em>&#8220;it is possible at some point he will deserve the nobel peace prize for this&#8221;</em></p><p>It was one of the most brutal backhanded compliments in tech history. Sam publicly thanked the man who had spent decades begging everyone to stop, for accidentally giving the AGI race its hype, its talent, and its funding. Meanwhile Peter Thiel, who had funded both Yudkowsky&#8217;s institute and OpenAI, grew worried about the influence of Yudkowsky&#8217;s ideas within the company. <em>&#8220;You don&#8217;t understand how Eliezer has programmed half the people in your company to believe this stuff,&#8221;</em>  he reportedly told Altman&#8308;, warning that the safety-first mindset would slow development.</p><p>Thiel&#8217;s concerns were well-founded, tensions over safety priorities helped trigger the November 2023 board crisis that briefly ousted Altman. In 2024, more safety departures followed, including Jan Leike and Ilya Sutskever, both of whom cited safety culture and processes as taking<em> &#8220;a backseat to shiny products.&#8221;</em>&#8309; Roughly half of OpenAI&#8217;s dedicated safety researchers eventually left. The company folded the remaining safety work into other teams and continued racing toward more advanced AI.</p><p><strong>Sources:</strong></p><p>1. Yudkowsky, E. (2023, March 29). Pausing AI developments isn&#8217;t enough. TIME. <a href="https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/">https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/</a></p><p>2. Yudkowsky, E. (2023, May 8). Eliezer Yudkowsky on the dangers of AI [EconTalk podcast]. <a href="https://www.econtalk.org/eliezer-yudkowsky-on-the-dangers-of-ai/">https://www.econtalk.org/eliezer-yudkowsky-on-the-dangers-of-ai/</a> (exact quote at ~25:30)</p><p>3. Altman, S. [@sama]. (2023, February 3). <a href="https://twitter.com/sama/status/1621621724507938816">https://twitter.com/sama/status/1621621724507938816</a></p><p>4. Altman, S. [@sama]. (2023, February 3). <a href="https://x.com/sama/status/1621621725791404032?s=20">https://x.com/sama/status/1621621725791404032?s=20</a></p><p>5. Metz, C., &amp; Hsu, T. (2025, October 30). How AGI became the most consequential conspiracy theory of our time. MIT Technology Review. <a href="https://www.technologyreview.com/2025/10/30/1127057/agi-conspiracy-theory-artificial-general-intelligence/">https://www.technologyreview.com/2025/10/30/1127057/agi-conspiracy-theory-artificial-general-intelligence/</a></p><p>6. Leike, J. [@janleike]. (2024, May 17). <a href="https://x.com/janleike/status/1791498178346549382">https://x.com/janleike/status/1791498178346549382</a></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://generativeaiforcuriouspeople.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Generative AI for Curious People! Subscribe for free to receive new posts.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[AI's original sin]]></title><description><![CDATA[When Michael Barbaro interviewed reporter Cade Metz about tech companies training AI models on copyrighted books, he called it &#8220;AI&#8217;s original sin.&#8221;]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/ais-original-sin</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/ais-original-sin</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Tue, 18 Nov 2025 08:08:36 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/2456ce69-487b-4cc2-9553-e246824df154_1080x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Today&#8217;s post was inspired by Thad McIlroy&#8217;s post <em><a href="https://thefutureofpublishing.com/2025/11/licensing-for-ai-what-should-book-publishers-authors-do/?mc_cid=a8ec686329">Licensing for AI: What Should Book Publishers &amp; Authors Do?</a></em></p><p>When Michael Barbaro interviewed reporter Cade Metz about tech companies training AI models on copyrighted books, he called it &#8220;AI&#8217;s original sin.&#8221;&#185; To build systems like ChatGPT, engineers scraped massive datasets and fed this material into training pipelines. OpenAI trained GPT-3 on two big book datasets that were most likely taken from Library Genesis, a site full of pirated books. Meta did something similar with LLaMA, using another huge set of around 190,000 illegally copied books. All of this copyrighted material was used without permission, with companies assuming the ends would justify the means.</p><p>The biblical language captured creators&#8217; anger better than legal terms could. &#8216;Copyright infringement&#8217; made it sound like a business dispute that could be resolved with licensing deals between publishers and AI companies. &#8216;Original sin&#8217; suggested corruption baked into the technology&#8217;s foundation that couldn&#8217;t be fixed with contracts, and which taints everything built on top of it.</p><p><strong>Footnote:</strong> AI&#8217;s original sins have been described in various ways. Techno-anthropologist Genevieve Bell, speaking at Trinity College Dublin, called data &#8220;artificial intelligence&#8217;s original sin,&#8221; pointing to how human biases and prejudices get replicated through traditional AI systems, like software that flagged Black people as &#8220;future criminals&#8221; at twice the rate of white people.&#178; Tim Gordon has used the term to describe how generative AI systems confidently produce false information (hallucinations) because they absorb unvetted online text without learning to distinguish truth from falsehood.&#179;</p><p><strong>Sources:</strong></p><ol><li><p>The New York Times, &#8220;The Daily: A.I.&#8217;s Original Sin,&#8221; April 16, 2024. <a href="https://www.nytimes.com/2024/04/16/podcasts/the-daily/ai-data.html">https://www.nytimes.com/2024/04/16/podcasts/the-daily/ai-data.html</a></p></li><li><p>Caelainn Hogan, &#8220;What does it mean to be human in the 21st century?&#8221; <em>Trinity Today</em>, 2019, pp. 44-46. <a href="https://www.tcd.ie/alumni/assets/pdf/trinity-today/2019/what-does-it-mean-to-be-human.pdf">https://www.tcd.ie/alumni/assets/pdf/trinity-today/2019/what-does-it-mean-to-be-human.pdf</a></p></li><li><p>Tim Gordon, &#8220;How to reduce the risks from AI&#8217;s original sin,&#8221; <em>Washington Post</em>, February 23, 2023. <a href="https://www.washingtonpost.com/opinions/2023/02/23/ai-chatgpt-fact-checking-tech/">https://www.washingtonpost.com/opinions/2023/02/23/ai-chatgpt-fact-checking-tech/</a></p></li></ol><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://generativeaiforcuriouspeople.substack.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Thanks for reading Generative AI for Curious People! Subscribe for free to receive new posts.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Curiosity Graph]]></title><description><![CDATA[How companies are starting to shift from competing for your attention to shaping your intentions.]]></description><link>https://generativeaiforcuriouspeople.substack.com/p/curiosity-graph</link><guid isPermaLink="false">https://generativeaiforcuriouspeople.substack.com/p/curiosity-graph</guid><dc:creator><![CDATA[Helen King]]></dc:creator><pubDate>Mon, 10 Nov 2025 08:08:28 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/d070f270-db7b-4457-bf84-febf8752260d_1080x1080.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>For centuries, people have<strong> </strong>read books, newspapers, and websites directly. Search engines like Google pointed you to existing pages, and Facebook showed posts from your network. The information was already there, these platforms just helped you find it.</p><p>ChatGPT works differently. When you ask a question, it doesn&#8217;t point you to an existing answer. It creates a completely new response, and each answer might lead to another question. Ask about work stress, and ChatGPT&#8217;s response might mention sleep, which leads you to ask about insomnia, then sleep apps, then meditation.</p><p>Chatbots are starting to remember your conversations and learn which follow-up questions you&#8217;re likely to ask. In 2025, researcher Shuwei Fang coined &#8220;curiosity graph&#8221; for this phenomenon, arguing that companies are shifting from competing for your attention to shaping what you want to know in the first place.</p><p>Companies have always tried to shape what people want. In the 1920s, cigarette companies hired Edward Bernays to make smoking seem like women&#8217;s liberation. Television convinced families that frozen dinners were progress. Now, AI systems are starting to shape what you want to know, not just what you see. Instead of showing shoe ads to people already shopping for shoes, Fang describes a future where advertising works by making people <em>&#8220;curious about sustainable high-end footwear before they realize they need new shoes.&#8221;</em></p><p>The same techniques that could nudge someone toward new shoes could nudge them toward political ideas. When Cambridge Analytica, the political consulting firm, used Facebook data to influence what 87 million users saw in their feeds, it changed how they thought about politics. As Fang warns, elections might be <em>&#8220;influenced not by what journalists write or voters seek, but by architectural decisions made years earlier by engineers who thought they were just building better search.&#8221;</em></p><p><strong>Sources:</strong><br><br>Fang, S. (2025). From attention merchants to intention architects: The invisible infrastructure reshaping human curiosity. <em>Shorenstein Center on Media, Politics and Public Policy</em>. <a href="https://shorensteincenter.org/resource/from-attention-merchants-to-intention-architects-the-invisible-infrastructure-reshaping-human-curiosity/">https://shorensteincenter.org/resource/from-attention-merchants-to-intention-architects-the-invisible-infrastructure-reshaping-human-curiosity/</a><br><br>Hern, A. (2018, May 6). Cambridge Analytica: How did it turn clicks into votes? <em>The Guardian</em>. <a href="https://www.theguardian.com/news/2018/may/06/cambridge-analytica-how-turn-clicks-into-votes-christopher-wylie">https://www.theguardian.com/news/2018/may/06/cambridge-analytica-how-turn-clicks-into-votes-christopher-wylie</a></p>]]></content:encoded></item></channel></rss>