On the X Algorithm

The X team recently published a copy of what they claimed to be the currently deployed implementation of the X For You Page algorithm, and I spent a lunch break going through it and assessing both the claims that they made in the documentation versus the implemented reality as well as anything else is notable.


There was a lot of things in the algo that were just common sense, for example, NSFW stuff will get you shadow banned effectively. In the same vein if you get reported or muted your account also will be out of Elon's Grace (at least in the context of this system).


There were some interesting things, and I wanted to list them out in case it was of use to anyone. Some of these may be well-known to those already in the marketing space and some are brand new conceptually (at least to me). I'll also be making a second post later going into detail about some of the gameable elements.


Before we jump into it, some of the terms that you need to know in order to understand what's going on:

• OON - Out of Network, ala someone you don't follow
• Thunder - The service that holds tweets from accounts you do follow
• Phoenix - A system that predicts how likely someone is gonna do a certain action (i.e. fav, retweet, reply, share, mute, report). The system is called a transformer (not relevant but added for completeness)
• Mixer - Takes those predictions, multiplies them by the published weights, and stacks the For You page. Thunder, Phoenix retrieval, and SimClusters (another system we're not going into) all feed it.
• VF - Decides whether a tweet is allowed to show at all. NSFW, spam, that kind of thing. If it gets flagged, it gets dropped and completely removed from consideration on For You.
• SID - How they group topics together. Fingerprint of the tweet's text and media.
• UEC - The global counter on likes, RTs, etc. Home-page likes live in a different system (Phoenix)
  
  
  I'll break down the concepts into two clusters, those that should be standard knowledge, and stuff that could be considered an edge.
  
  

  Basics

  
  

  1. If you tweet about diverse topics, the algo accidentally punishes you. It promotes you more if your tweets are in the same SID. Scatter across topics and the system has a harder time finding the people who already care about your stuff, in-network or OON.
     
     
  2. The more in-category people you have, the more the algo kinda boosts you up, especially mutuals. Mutual follows have a 4x multiple in regards to replies. Reply weight goes from 5 to 20 when they're looking at your original or your quote.
     
     
  3. Quotes are treated as regular tweets for all intents and purposes, which may sound obvious but actually has major algo implications, since a quote is not a reply and not a retweet (share). OON quotes still make the For You pile and the Rank-all system indexes them. Brand-new accounts get the cold-start treatment on them too (i.e. the algo you giving you some love when you first start).
     
     

Fun Ones

1. There's a first-like benefit even when it's not on Home Page
   
   

   The X algorithm docs claim that a like only has ranking impact if it happens on the Home timeline, that appears to just be in their system called the scorer.
   
   

   The indexer which actually decides shows that it's actually a favorite from anywhere on a tweet younger than 49 hours (weirdly specific number), gets the tweet put into the 1fav index which is the For You tweet source pile which Phoenix actually searches.
   
   

   At 32 favs it also writes a 32fav index (which we're not sure what it does but it looks important). There's a 1-minute cooldown between index refreshes, and after 48 hours from the first update those refreshes stop. You also get written into a post_creation pile the moment you publish, with or without likes.
   
   

2. Who you're shown to is based off their activity
   
   

   Phoenix looks at the other person's last thousand-odd actions (1024), finds the SID neighborhood that matches, and pulls tweets (and authors) that sit next to that history. Your author hash rides along with the SID, so once you're in someone's pile you tend to stay there a bit.
   
   

3. The other OON hose is sticky
   
   

   SimClusters, the second way you show up in front of strangers, keys off their last 15 likes, 15 retweets, 15 replies, 15 bookmarks, 15 shares, 15 photo expands, 15 video watches, and their last 15 of their own tweets.
   
   

4. 500 is the cap
   
   

   Hit 500 scored posts in the cache for a viewer and this load stops rerunning Phoenix. It serves the frozen slate for about three minutes, then it's live again. I'm not sure how to optimize for that but i'm sure someone reading can.
   
   

During my next foray into the algorithm I'll show some hacks/tricks on how to hyper-optimize.