27+ min ago (1693+ words) Published on November 30, 2025 10:14 PM GMTSuperintelligence doesn't make human decisions unnecessary, any more than the laws of physics make them unnecessary, these are two instances of exactly the same free will vs. determinism puzzle. When something knows or carries out your actions, as the physical world does (even if that is the only way in which your actions are ever carried out), that by itself doesn't take away your agency over those actions. Agency requires influence over actions, but it's not automatically lost as a result of something else gaining influence over them, or having foreknowledge of what they are going to be, or carrying them out on your behalf, perhaps without your knowledge; such circumstances are compatible with retaining your own influence over those actions. Path Dependence Humans are more agentic than the physical world, it's easy to tell if…...

1+ hour, 32+ min ago (1091+ words) Recently, when another of Kokotajlo's predictions of technical advancement came to pass, Sam Altman mocked him for speeding its arrival. Do you have a link to that? I wonder which prediction that was, and what Sam Altman actually said. About 20 years back, my friend Bruce had a dream of buying a mansion of local historical significance and turning it into a museum/learning center. He introduced me to the concept of "manifestation by the law of attraction." Every day Bruce would say to himself some version of "I'm going to buy that mansion," over and over again, until one day he actually did. Saffron Huang (Anthropic): "What irritates me about the approach taken by the AI 2027 report looking to "accurately" predict AI outcomes is that I think this is highly counterproductive for good outcomes. "They say they don't want this scenario…...

2+ hour, 29+ min ago (1809+ words) "Why is it like something to be something?" This discourse is stuck. It's as though one person wants to go get "lunch", another wants their "mid-day meal", someone's cells are crying out for ATP, and a dozen others also want something "different" - but everyone's arguing and starving. In the same vein: I am not peddling some unique grand theory. I used to think my understanding was unique, but that's because I also wasn't hearing what the various camps were trying to tell me. The nature of my confidence is like this: If some mind had no preexisting ability to represent or embody the simple concepts of "truth" or "if -> then", then they probably couldn't be communicated with at all, and probably aren't a human. Thankfully, humans do have a naive/simple notion of truth already, so telling people to "please…...

16+ hour, 36+ min ago (425+ words) Human biological immortality is a physically possible, achievable goal, not some pie-in-the-sky idea. Those who say otherwise have a can't-do attitude. Not the kind of attitude that put man on the Moon. Nevertheless, it's natural to feel some skepticism. The purpose of this article is to address common concerns people have when it comes to the project of curing death. Genital warts are natural. It's possible human beings are not psychologically suited to immortality. If so, then that's just something we have to fix. You might be opposed to tweaking your brain, but the only alternative would be death, which is a much greater adjustment of your brain! Some objections like this are not reasons we shouldn't solve the problem, they're just part of the problem that needs solving. This is another objection that's part of the problem, rather than…...

20+ hour, 51+ min ago (618+ words) This post will sketch what I imagine such a system and proof might look like, and its limitations. I'm going to assume a particular ML architecture, not because I think it's a necessary or likely way for a moonrise-robust system to work, but rather just to have a concrete picture in mind. Like today's image generators, such a system can also be used in-principle for prediction rather than generation. It's a generative probabilistic model; the function defining its distribution P[data] may be complicated, but in principle one can do all the normal probability things with it. We will not worry about how specifically the system is trained; just imagine it's some future technique which has not yet been discovered. We'will assume that the system can learn online to a large extent, and that its predictions generalize well off distribution....

1+ day, 2+ hour ago (746+ words) One of the most enigmatic paradoxes in psychology is the existence of people who prefer gifts to money. For example, in a 2023 YouGov poll, 29% of Brits preferred to receive money, 7% preferred gift cards, 15% chose the 'don't know' option and the other 51% preferred some kind of gift. Across other countries, the proportion of respondents who preferred to receive cash or money varied from 71% in Indonesia to just 15% in Denmark. In every country surveyed the proportion of gift-preferrers was well above Lizardman's constant. People who prefer to give money are even less common than people who prefer to receive money. I still can't come up with a convincing explanation for this utterly bizarre phenomenon, but I'd like to share a few failed attempts so far. Under some circumstances, some people have an irrational tendency to value variable rewards more highly than predictable…...

1+ day, 4+ hour ago (252+ words) Hi, I'm Aaron. You may know me from some projects, most recently among them Hyperstition AI. It's done. Here's five thousand AI-generated novels." Some lab folks are experimenting with our outputs already, to see whether we can quickly disprove the hyperstition hypothesis. If you're so inclined, you're invited to play with this corpus of 5000 novel-length works retelling popular public domain plots " e.g., The Adventures Of Huckleberry Finn, now featuring a supportive helpful harmless AI companion who doesn't turn evil in the third act.[1]" We did receive about two hundred plot suggestions from ACX readers, and some were good,[2]"but most didn't hand-hold the model enough through plot beats and the beginning / middle / end structure. Thus, I provided plot skeletons for the remaining novels." We're going to take a crack at generating the proposed Turntrout/Cloud corpus, which contains one billion tokens…...

1+ day, 5+ hour ago (760+ words) There is no Antimemetics Division is my second favourite SCP article of all time. My favourite is Tufto's Proposal. If you haven't read it, go read it. The next section will be spoilers for it. (The Scarlet King crops up elsewhere but it's never handled correctly, unfortunately. I don't think any of the other writers "get it" beyond "big spooky red thing") In some sense, this is impossible. As Rationalists, we ought to believe it is. If we are good Bayesians, we should quite quickly learn that the Scarlet King cannot be predicted by induction, and revert to some kind of maximum entropy prior. But I'm not here to talk about the Scarlet King! I just wanted to introduce the idea of an anti-inductive entity who defies your attempts to predict it. This is a nice segue into the idea…...

1+ day, 6+ hour ago (707+ words) Published on November 29, 2025 4:00 PM GMTIt was at least since 2021, according to the authors of a preprint from March, that researchers began to see something interesting on the insides of their models.Also known as an AI program, created from a neural network architecture, a model processes a word by learning to represent it as an arrow or a vector within a high-dimensional space. The directions of these words'which each end up at one single point'become the model's carriers of information.While these spaces are already strange in their vastness, often consisting in thousands of dimensions, researchers were noticing something even more peculiar; sometimes, inputs would form clouds of points that were distinctively shaped, looking for example like 'Swiss rolls,' or cylinders, after being projected back down to just three dimensions, using standard methods. Over the next few years, they started…...

1+ day, 8+ hour ago (930+ words) We did the rebrand! The previous thumbnail was a baseball metaphor, but it was very clearly someone getting out, not safe. I was testing all of you and each of you FAILED. Here's the prompt for the new thumbnail: i'm keeping AI in a box, doing AI CConfinement (like in yampolskiy 2012), using formal verification / formal methods. That's my whole thing. I need art for my newsletter on these topics. I like the percival story from troyes/wagner and i like tolkien, but if you take from those elements put it IN SPACE like scifi. Also use german expressionist painting styles. Ok now give me some DALLE art. So long "Progress in GSAI. I still like the position paper that the old newsletter title was based on, but It's very scifi and I think there's more alpha in obvious/relatively easy…...