Staggering Stats to Start your Saturday
I love an alliterative title.
When I’m not on Substack being the voice of a generation, I daylight as a Web Analyst (well, I’m really more of an Engineer of sorts, but that’s what it says at the bottom of my work emails, so I guess I’m stuck with it). Naturally, this has cultivated in me a great respect of data. It flows in my veins. Every morning, I kneel before a shrine of bar charts, pie graphs, and a candlelit Blu-Ray of Moneyball. Some say it’s overkill, but Brad makes me feel safe.
So, I see a lot of stats, but every now and then I see something that makes me “AWOOGA” at full volume while my eyes extend three feet out of my head. There are some charts that leave a lasting impression. Some happy, some sad - these are the ones that piqued my interest recently.
Benchmark Saturation
If you’re not familiar with what an AI benchmark is, it’s essentially a test for AI models. They assess skills like software engineering, visual reasoning, speech recognition, etc. As tech progresses, benchmarks become ‘saturated’, meaning we hit diminishing returns on progress. We make our way from a score of 20% to 90%, but then things slow down a bit, and we need to make newer, more difficult tests to progress. Still, once a demanding benchmark has hit that ‘wall’ (or, perhaps, thick sludge), it’s at least an indication that the domain is being lost to machine dominance - just like chess and girlfriends did.
“Mum, she’s not a ‘doll’, she’s a pediatrician, and you will treat her with respect!”
So, saturation is not a new thing. We’ve had benchmarks for many years, and they’ve grown more difficult as AI gets more sophisticated. You want to know what’s really wild though? The speed at which we’re saturating these benchmarks is accelerating. A lot.
Benchmarks are falling very quickly these days. Take ARC-AGI for example. A notoriously difficult visual reasoning benchmark, the highest score achieved on it in January of last year was only a few percent, but by December we achieved 87.5%. In one year! Progress that once took decades is now being made in months.
One unbeaten benchmark that I think is of particular interest is OSWorld. It’s designed to test an AI model’s ability to do ordinary every day things. It tests agents, who go out and complete tasks, rather than chatbots, that just reply with answers. Agents, I imagine, are the next frontier. We know that AI can write competent code, and knows more facts than any one person does, but its utility will really begin to show when you can comfortably give it complex, open-ended tasks like a human. We want to be able to tell it to have that report on our desks by Friday, and slam our fist on the desk like we’re J. Jonah Jameson.
The sort of tasks in OSWorld are fairly simple for us. In fact, they are literally derived from real world computer use cases, and humans can achieve 72% on it. However, the highest AI score currently achieved on it is ~20% from Claude.
It’s at this point that people say “See? It’s an idiot! It can’t even use a GUI, send an email, or build a report. AI is just the next NFT fad.”. But as we spoke about earlier, that 20% percent will not sit for long. In fact, it’s not crazy to think AI will beat us at this test by the end of the year (at least if ARC-AGI in anything to go by). It’s quite daunting that this is a rate of progress we have to get used to now. Actually, it could be a rate of progress that passes us by, while a faster, much more scary rate of progress replaces it in a mere 6 months.
Animal Consumption
A sad one, with a bit of ranting, but it’s important. So, if you don’t read it, I’ll come to your house and smush your face against the screen until you do. Sorry, you signed away your face smushing rights in the terms and conditions of your subscription (right next to the butt slapping clause).
Anytime. Anywhere.
We eat a lot of animals - enough that it’s pretty hard to count. Estimates vary, but these days we think it’s something like 80 billion land animals a year, with aquatic animals in the trillions. To put that into perspective, about 100 billion humans have ever existed, and we churn through that much flesh in a matter of weeks or months.
While you’ve probably heard that statistic before, it only sinks in when you see it visually. Big numbers are hard to grasp. Once they pass a few hundred thousand, my brain just goes, “Okay, I get it - big number!” and tunes out. What really brings it home is to see it. Here’s a graph comparing the number of human deaths each year with the number of farmed land animal deaths.
I had to only include farmed land animals or it’d else just be a red circle, and you’d think this was about Japan or something.
This is why I get a little bit annoyed when people act like I’m a over scrutinizing puritan for talking about animal rights. I think on some level people think eating meat might be wrong, but it’s wrong in the same way that lying is. It’s the sort of thing that we ought not do, but hey - we’re only human!
But eating animals isn’t like that. At all. We’re talking about causing more suffering to other species than all of humanity has ever experienced every couple of years - and it’s for products we don’t even need. I haven’t lived a perfect life. Like anyone, I’ve done things I’m not proud of; but if you produced a list of my wrongdoings in order of badness, and had to add keeping a dog in a cage for 6 months, cutting off its tail, ripping out its teeth, and cutting it’s head off - it would certainly go to the top of the list! I imagine (and hope) it would be the same for anyone reading this. Yet, that’s only a fraction of the harm a single year of meat eating causes.
You might be inclined to look at the above graph and think “Well, it’s so enormous, what is one person’s changed habits going to do?”, but that’s confused. The reason it’s so enormous is because each individual’s contribution to this is also massive. If somehow all meat consumption was provided by one cow being slaughtered a year, each of us would be getting meat for the cost of an 8 billionth of a cow life. If that were the case, I’d eat meat too! However, your average American eats 7000 animals in a lifetime, 99% of them living in what can only be described as torture chambers (which you can read about here SFW).
We often turn ourselves into knots over things that are comparatively very minor. Do we use paper straws and recycle? Do we say antiquated words? Are we using our “platforms” appropriately in response to whatever news story took over the zeitgeist this week? However, doesn’t this all feel a bit ridiculous when looking at the above graph? How does our decision regarding this not totally dwarf all other moral dilemmas of our age? The only answer I hear is an appeal to species difference, but as I’ve said before, that’s a dead end.
State of the World
Okay, let’s end on a happy one. If you look at the news, things look pretty bad. Everyday there’s a shooting, a bombing, or a Jake Paul fight - it’s relentless. However, there is a bit of an illusion going on, and it’s one that I fell for pretty dramatically.
Just a few years ago, I was what you’d describe as a Doomer. There are some problems, like national debt, biodiversity loss, and ocean acidification, that don’t really seem to have a solution. Well, maybe they do, but the solution would require a level of cooperation that humanity shows time and time again it’s incapable of. So, it was easy to think that things are taking a downturn. I was terminally online, and spent a decent portion of each day imagining what collapse would look like. What happens if all the fish die, or crops start to fail? How long do we have left until fossil fuels run out and we’re left in a medieval feudal state? I foresaw a future where I was standing over a burning oil drum, rubbing my hands together, trying to sell my supple body for a can of old beans.
I think I was wrong about that now, and don’t worry nearly as much. In large part because I think we’re about to make super intelligent AI, and no one really knows what will happen after that. It’s such an enormous shake up that I don’t feel confident extrapolating so much anymore. However, looking back, I can also see I fell victim to bias.
Many of us tilt toward pessimism for a few reasons. As kids, we didn’t read about bad things online, so it feels like instability is new - even though it’s not. Additionally, the news only surfaces the bad stuff because it gets more clicks than good stuff, so if you flip through the internet, you’ll more likely see an article prophesying disaster than success.
We all know this, and feel that while other people are silly rubes for falling for this ruse, we’ll rise above - but then everyone thinks that. What’s so special about you and me that we can overcome human psychology so easily? We’re human, just like everyone else, and so the barrage of negative news is going to have an effect on our world view. It’s no surprise to me that young people - the ones that spend the most time online - are also the ones that are most likely to be pessimistic about the future.
Yet, if you look at the data, a very different picture emerges from the one on the TV screen. There are a lot of issues that we’re going in the right direction on. Extreme poverty is on its way to being eradicated.
That’s incredible! It’s more than halved in my lifetime alone. It’s also not the only metric that’s improving:
Life Expectancy
Literacy
War and Peace
Our world in data reads like the utopian origin story. Yet, it doesn’t get much coverage, which I think is a shame. If people my age were as well acquainted with this as we are with all the awful shit, we may have bit more fight in us. I get why we sometimes feel like things are falling apart, but in many ways they’re just starting to build up - and that’s pretty cool!
Lightning Round
Here are some other pieces of data that I think are cool or noteworthy. Have a good weekend, everyone! I’m gonna go play Final Fantasy VIII for 9 hours.
Weight Loss drugs are pretty good these days! In the above study, the placebo wasn’t even doing nothing - it was diet and exercise. It’s promising that we could start making a serious dent in obesity rates.
The cost of sequencing the human genome is a fraction of what it used to be. This helps with all sorts of stuff, like cancer research, or building disease resistant crops.
Not particularly surprising, but it’s crazy how quickly new platforms grow now. This shows how long it took each platform to hit 1M users. Threads is number one, but I feel like it’s cheating because it basically just copied Instagram users. Still, ChatGPT totally crushed Facebook and Spotify. Household names enter our homes a lot quicker than they used to.
really well written!
I’m in the 2 minute warning stage of life, so here is an idea; all these platforms seem to bloom like cancer. Has any programmer, or AI itself taken on the missions of:
1. Curing cancer
2. Ending hunger in the third world
3. Ending war
That’s my input for now