My mother grew up speaking Arabic, but only a few words have found their way into my lexicon; namely terms of endearment like “gulbi”, which means “my heart”, and the occasional “yallah” as a real catch all. While I struggled with elementary Spanish, my sister is a polyglot. She speaks Spanish like a local and is regularly mistaken for one.

Me and Duolingo exist like passing ships in the night. We reconnect every few months when I realize how little Spanish I remember, only for me to flounder on airport or grocery store vocab.

Recently, I had a moment of “thank God I paid attention in Spanish class”.

This August, I found myself in a relatively isolated area of Portugal, waiting 7 hours for a tow truck to get our unable-to-start rental car, with 1% left on the one phone between the two of us traveling together. Luckily, the small surf beach was a camper’s paradise, mostly for Spanish-speaking visitors. I mustered up every bit of confidence I had and went van to van, asking any camper to lend us a phone charger, using my broken Spanish.

I think I probably got out some version of “Tu tienes un cord por mi teléfono? El teléfono tiene nada! Cero! Tienes energía para mi teléfono?”. It was enough to get a lovely Spanish family to hand over a portable charger, which gave us just enough battery to get through the day.

I was thankful for elementary school Spanish, my one quarter of Spanish required to graduate college, and some undeserved confidence. After recovering from the bleak day, I wondered: what will this scenario look like in a few years, when we all have tools that make languages feel like a fraction of the barrier they are today?

The fact that a single human can learn multiple languages is fascinating. Just look to children — “kids really can learn the whole variety of human languages. And that’s for an interesting reason, which is that languages are evolved to be learned by kids. So a language can’t survive as a language if it’s not learnable”, says cognitive scientist Michael Frank. Languages are inherently learnable and structured, making them perfect for large language models to be able to adapt to them.

ChatGPT supports 59 languages. However, model performance still relies heavily on the amount of data available online, meaning that languages left out of the digital ecosystem are harder for ChatGPT to perform tasks on. Given the variance in performance by language, more AI-generated content will appear online in a handful of languages, createing a feedback loop where high-resource languages keep getting richer, while lower-resource ones risk further marginalization.

OpenAI is highly motivated to get its products into the hands of non-English-speaking users for this reason. Both OpenAI and Perplexity recently announced offering their AI tools to users in India for free — a data-acquisition play to improve model performance and user experience in their native languages before eventually introducing paid tiers.

There’s a dream world in which tools like ChatGPT actually enable preservation of languages. Preserving a language will require input from native speakers training models, especially if there is very little training data available online. As a 6th grader, we were required to take Latin, which was deemed a positive for my future SAT scores but useless for conversation. If there had been a way to practice Latin on-demand, powered by AI, I’m sure I’d remember far more today than I do. A tool like Speak would have done wonders for my skills, providing an AI-powered tutor that could converse with me 24/7, for a fraction of the cost of a tutor.

Beyond preservation, AI opens up opportunity for communication across languages in real-time. Tools like Toby or Apple’s native translation stack will provide economic value, like hiring a non-English speaker for an upcoming work project or opening up opportunities for folks who have been previously been limited to offering services in a single language. More of these tools will move to our phones, so it’s clear why Apple is making a play here.

I’d be curious to track the percent of English speakers worldwide over time, with post-ChatGPT as the reference point for potential inflection. I’d bet we will see more English spoken worldwide as a result. We definitely won’t lose languages altogether — geography, culture, evolving dialects, and unevenly distributed access to technology ensure they’ll persist even in an age of AI. What we may lose is the traditional way we learn them. Learning a language in the future will be faster and cheaper, but also easier to avoid as real-time translation becomes common practice.

Two weeks after my dire moments in Portugal, Apple launched Live Translation.

In Messages, Live Translation can automatically translate a user’s response as they type and deliver it in the recipient’s language. During FaceTime calls, users can follow along with live translated captions while still hearing their friend or family member’s voice. And during a phone call, the translation is spoken out loud in real time. Users can access Live Translation on AirPods with an all-new gesture by simultaneously pressing both AirPods stems; by saying “Siri, start Live Translation;” or even from the Action button on iPhone, which will allow them to hear in their preferred language.

While the demos feel a bit clunky, the result is that the gap between hardware and software providing value for multi-lingual experiences is closing.

We’re moving rapidly towards a future where an AirPod in your ear can break any barrier that language has been in the past. I could speak in Arabic with my grandfather. I could order a glass of wine in Italian.

But, perhaps an over-reliance on AI wouldn’t have helped me when I was service-less, phone-less and without a car in Portugal. Looking back, I’m forever grateful I paid enough attention in elementary Spanish to get through that day. But perhaps AI would also accelerate my ability to learn a new language, so I would have had far more confidence, landing the right words the first try instead of stumbling through.

I’ve always viewed language learning as an art. We’ve culturally tied being multi-lingual as a desirable goal. Perhaps we’ll lose some of that art over time. However, the opportunity outweighs that art — democratizing language learning, reducing costs, economic opportunity for non-English speakers, and preservation of old languages. A world where the barriers of language start to break down is one that I’m eager to be a part of.

Subscribe now

Two things have occurred. First - life happened! Since February, I’ve been on over 20 flights, spent lots of time with family, ran the Boston marathon, moved into a new apartment, bought a new domain for this blog and enjoyed some weekend trips out of NYC. A second thing plagued my ability to write — I worry that there are no new ideas. This is driven by the fact that the cost of ideas is going to zero.

I believe we are experiencing two things at once — the biggest Cambrian explosion in ideas of all times (anyone can have an idea, just chat with an LLM!) alongside a great creative consolidation (how can you have a good idea when everyone has access to the same machine?). The cost of an idea is going to zero, but the cost of a good, new idea is rising. Great ideas are getting more expensive.

Subscribe now

The creative consolidation started with social media

Last year, I read Filterworld. The book explores how social media algorithms have led to the progressive flattening of culture. Social media platforms can both connect communities across the globe, while also being built on algorithms that reward sameness. Users crave perpetual fluency (ease in processing visual stimulus). We want content we can understand and absorb, quickly and easily, so sameness just breeds more engagement which breeds more sameness.

Algorithms flattened culture on a global scale. In 2015, if you were in NYC or Madrid or Seoul, every coffee shop had white subway tile and a cursive neon sign hanging behind the baristas. These common threads of culture appeared everywhere, as the sharing culture encouraged this sameness beyond borders.

Generating a flattened culture

Fast forward to today — AI is the most disruptive catalyst to technical change that we’ve seen since the semiconductor, or maybe fire. The large language models (LLMs) that power ChatGPT and the likes exist as a snapshot in time. When a large language model is trained, it creates a generalized understanding of the world. LLMs by their nature are supposed to predict the most probable next word based on patterns detected in its training data. However, this means LLMs naturally favor the average. Outliers are statistically suppressed. They don’t generate novelty, but rather an average of all novelty that has existed in the window of its training. LLMs are perpetually stuck in the ideas and attitudes of their training data. LLMs are a mechanism to generate flattened culture, picking the familiar over the fringe. Culture, once layered and strange, will be optimized for fluency in an AI world.

We can’t ignore the magic!

So, good ideas are expensive and we’re heading towards a cultural convergence, propelled by LLMs. It may be a pessimistic view. However, I believe it encourages us to set our bar higher, to push our thinking, and to resist the urge of average.

I remember vividly taking a test in high school that included a reading about visiting the Grand Canyon. The writer argued that for each visitor that saw the Grand Canyon after the first person to ever see it had an experience that was marginally worse, less exciting, and less unique than everyone that came before them. An experience, once magical to a few, has become normalized to the many. With each new fantastical experience that new technology is creating for us, somehow our bar for jaw dropping, impressive, world changing, and unique continues to get higher. This will push us to new, optimistic heights.

I wrote this piece up last weekend, when apparently mine and Sam Altman’s writer’s block ended at the same time. His latest piece, the Gentle Singularity, discusses the rising bar:

Already we live with incredible digital intelligence, and after some initial shock, most of us are pretty used to it. Very quickly we go from being amazed that AI can generate a beautifully-written paragraph to wondering when it can generate a beautifully-written novel; or from being amazed that it can make live-saving medical diagnoses to wondering when it can develop the cures; or from being amazed it can create a small computer program to wondering when it can create an entire new company. This is how the singularity goes: wonders become routine, and then table stakes.

It’s not a bad thing! Any single idea may diminish in value, but good ideas will be rewarded. Good ideas will still emerge from the fringe, the novel, and the nuanced. We will have to encourage outliers, even when LLMs may push us towards an average. Artificial intelligence is not a replacement for creativity. Ideation, powered by creativity, obscurity, and wonder, will be encouraged.

For a long time, technical people in the startup industry have made fun of “the idea guys”; people who had an idea and were looking for a team to build it. It now looks to me like they are about to have their day in the sun.

The cost of any idea is going to zero. But the cost of good, novel, and new have never been more valuable. The same systems that will flatten culture may also open it up. When ideas become free, the barrier to enter is lower, but the ceiling is farther away. The world is truly opening up. Anyone can write that novel, start that business, or make that movie. Ideas are cheap, but now the tools to bring them to life are powerful and accessible. We can bring the jumble of ideas in our head into reality in a way we haven’t been able to before AI. In a world where ideas are plentiful, we’ll get to focus on the novel, the unusual, the different, and the obscure. I am excited for this reality.

Subscribe now

Thanks for reading. This Substack is now available at day-to-data.com as well as on the Substack platform. Making things official around here. Looking forward to the next one.

After a brief hiatus, and a fast start to the new year, it’s great to be back to writing. The last two months have been nothing short of crazy in the tech world - from Stargate to DeepSeek, alongside new models and endless promises of how AI will reshape how we work and even how we think about being human.

If there’s one thing we can all agree on, it is that technology is moving at a speed that feels hard to comprehend. There’s a popular benchmark for LLMs called GPQA which consists of multiple choice, Ph.D-level questions. In June 2023, GPT-4 achieved 31%, which is just a bit better than random guessing (25%). Just over a year later, with the release of OpenAI’s o1, model performance surpassed expert level outcomes, scoring a 76% against the benchmark. TLDR; in a year, we went from a kid doing multiple choice questions for fun to an LLM that is outperforming human experts. Dario Amodei said it best, calling it the “compressed 21st century” something like 50 years of research may materialize in 5 years. It’s incredible, terrifying, and mind-numbing all at once.

This past December, I had my own realization of just how far technology has progressed, with a very different, yet somehow more tangible evolution of innovation.

The world before flash drives and floppy disks

In the days leading up to Christmas, I sat with my grandparents in their living room. I grew up in a family of academics and have normalized piles of essays and other tidbits of academia lying around the house. We were in the depths of a conversation when my grandfather was inspired to surface a folder of trinkets he has kept in their basement for decades.

First out of the folder, he handed me old “punched cards” that he had kept from his days of using IBM mainframes while working at the University of Wisconsin-Milwaukee. In the same folder, he had floppy disks, CD ROMs, and flash drives — all remnants of the evolution of technology he saw unfold. So today, you’re in for a brief history lesson on the history of data storage. To get to the punched card though, we have to start with making fabric.

From Babbage to IBM

The story starts with a Frenchman. In the early 1800s, Joseph Marie Jacquard was trying to find a faster way to make fabric. He used large cards with holes that when organized together would represent the desired pattern to be woven into fabric. Pins would push thread through the holes of the punched cards, and be prevented from pushing thread through areas where there were not holes, with a shuttle moving back and forth along the loom, resulting in a patterned piece of fabric.

The result was the Jacquard loom, which caused the price of fabric and time spent on textile production to plummet. Jacquard’s punched cards became a physical representation of binary code — punch vs. no punch dictated the actions of the loom, to thread the needle or not to thread the needle.

A few decades later in the 1830s, Charles Babbage, better known as the father of the computer, and Ada Lovelace, aka the world’s first computer programmer began to imagine ways to use punched cards to represent not just binary outcomes, in the form of a 0 or 1, but any type of data. Babbage’s Analytical Machine, which used punched cards as the inputs and outputs of his number crunching machine, unfortunately never came to fruition. However, Babbage and Lovelace’s work led directly to Herman Hollerith and, ultimately, the founding of IBM.

Herman Hollerith & the punched card tabulator

After teaching at MIT, Herman Hollerith left to pursue a statistics role at the U.S. Census. In the late 1800s, counting census results took almost a decade, just in time for another census. Given his knowledge of both Babbage & Lovelace’s punched cards and the complexity of data in the census, Hollerith got to work building a solution.

He went on to invent an electric-powered counting machine. In 1888, Hollerith entered his machine into a contest put on by the U.S. Census for a solution to automating data tabulation and counting, where the winning machine earned a contract to process the 1890 census. For a data preparation task, Hollerith’s machine took 5.5 hours, outperforming his nearest competitor who took 44.5 hours. The 1890 U.S. Census was the first to be counted by Hollerith’s electric-powered counting machine and took only two years, versus taking eight years the previous decade. Hollerith’s tabulation machines ended up completing censuses across Italy, Russia, Norway, Puerto Rico, the Philippines, the US, and several other countries.

In 1898, Hollerith founded a business called the “Tabulating Machine Company” to commercialize his counting technology. A decade later, after pressure from the government to reduce the cost of his computing services and time winding down on his patented technology, Hollerith sold his business to a financier who merged Hollerith’s business with two others, resulting in a new company called the “Computing-Tabulating-Recording Company”. In 1924, the business was renamed the International Business Machines Corporation. And IBM was born.

The “modern” punched card

The punched card truly skyrocketed with the creation of IBM. In the mid 1950s, 20% of IBM’s revenue came from punched card sales. And for almost 50 years, IBM’s punch cards held most of the world’s stored data.

To “write” a program, each card was one line of code, roughly equivalent to 80 bytes (my 256GB computer I’m writing this on would require 3.2 billion punched cards to store an equivalent amount of data). As cards grew in complexity, computer programmers continued to tackle more complex problems, ranging from the Census, to science, to complex mathematics, and more.

The 1965 IBM 1130, similar to what my grandfather learned how to use punched cards on, is a far cry from Hollerith’s counting machine from nearly a century prior. The 1130 could be rented for less than $1K/month to be used by researchers, mathematicians, scientists, accountants, and more.

The era of infinite storage

Now, storing massive amounts of digital data is table stakes. iPhones and computers easily have 256 GB of storage. I store over 50K photos on my cell phone. The hundreds of photo albums that sit in 5 bookshelves in my grandparent’s basement could fit on a flash drive no bigger than my thumb. We’ve come a long way from the days of the punch card.

Quick technical note: storage & memory are actually two different things. Storage is used for keeping longer term data, whereas memory is essentially faster, temporary storage for data that needs to be readily accessible at any given time. When it comes to your computer, you may have a device with 256GB of storage, but only 8GB of memory, which is purposeful given memory’s temporary nature. If you’re a Mac user, you can click ‘About This Mac’ under the Apple icon to find out your memory!

I left my grandparent’s house with a pile of punch cards that I plan to frame and hang up in a spot around my apartment. When I arrived home after break, I showed them to my roommate, who said, “you should write about these on Day to Data!”. So here we are. A bit of history, a bit of technology, and just a slice of the innovation that’s helped get us to what we’re accustomed to today.

Thanks for reading this month’s article. More history and deep dives ahead. And a reminder that if you’re thinking about starting something new, curious about startups, or just want to jam on ideas, shoot me a note on LinkedIn — I’d love to chat.

Subscribe now

SC24

For a bit on Supercomputing — the conference took place in Atlanta during the third week in November. The massive exhibition hall brought together over 18k attendees, with a tilt towards academics and folks working in the HPC ecosystem (data center owners and operators at computing companies). I arrived on Wednesday and could noticeably tell I was joining at the mid-point, only due to the raspy, lost voices of folks at their booths. It was a bustling atmosphere, full of curious folks that I was happy to be amongst.

A hardware world

As someone who spends most of her time in software, Supercomputing threw me out of my comfort zone. I spent my time navigating through booths full of server racks, cooling tanks, and power systems. It was pretty incredible.

The big companies had pulled out all the stops. Nvidia had a booth with 20+ employees demoing the state of the art GPUs and what folks are using H100s and Blackwell to build. Vertiv had their modular data centers out for demoing. Among the hardware, there were even gelato stations, coffee carts, and tacos.

There was a few major themes that came out of SC24 for me, both from sessions I attended and conversations I had with other attendees. To highlight a few…

• Data centers at the edge: Companies like DUG are deploying shipping container sized data centers, complete with power, cooling, and connectivity, in locations on the edge. Interest appeared to be largely from folks in research (think trying to do computationally heavy work in a remote location).

• Energy efficiency is top of mind: across several of the panels was a shared sentiment that as exciting as AI is, it is putting pressure on companies to stay in check with their energy consumption. Read more of my thought’s on this in a past article here.

• High performance computing is for more than just AI: While AI was the darling of the event, powerful hardware is being used for traditional machine learning, from mathematics to medicine, and robotics to agriculture.

• The national labs are not messing around: an impressive showing came from the folks at the national labs like Livermore and NREL. Livermore launched their exascale supercomputer, El Capitan, which has recently been declared the world’s fastest supercomputer.

• Honorable mention — Bitcoin mining centers turning into AI hubs: Yep. Bitcoin mining centers are trying to pivot their operations to host more AI workloads.

Far outshining all these topics was one theme — computers are getting really hot, and we need to do something about it. Let’s dig into what’s happening in the data center and the companies racing to cool things down.

Things are getting hot

Just as your laptop heats up as it’s overworked with processes, all computing hardware will generate heat while it’s running.

GPUs and CPUs today will have parts that help regulate temperatures, similar to the fans that you can hear as your laptop overheats. In addition, data centers use air cooling to keep servers cool, setting up “hot aisles” where the computers push their excess heat, and “cold aisles” where cold air is pulled into the server racks. As you can see in the image below, the floors are raised to deliver cold air from all angles.

This set up has been sufficient, but most definitely won’t work for the next decade with AI workloads. Why’s that? The power density of GPUs is increasing. GPUs are delivering more computational power (typically measured in teraflops, which is essentially how many operations a GPU can perform in one second) per unit of physical area or volume of the chip. While this means that for the same size GPU, you get more processing power, it results in a host of challenges. The main implications are keeping these systems cool and complexity of manufacturing. I’ll save my thoughts on the ladder for another day.

We’re hitting the power wall

Nvidia’s H100 demands about 700w per processor. Their latest Blackwell will pull in about 1kW. On a single processor scale, this might not seem huge, but when you’re filling a data center with thousands of these processors, the power difference is real. Blackwell’s rollout is reportedly slowing due to overheating in the 120kW racks. Per Tom’s Hardware, “these problems have caused Nvidia to reevaluate the design of its server racks multiple times, as overheating limits GPU performance and risks damaging components. Customers reportedly worry that these setbacks may hinder their timeline for deploying new processors in their data centers.“

And we need solutions

Nvidia’s Blackwell can only achieve peak performance if liquid cooled. Everyone’s in need of solutions to address the overheating problem. There’s a few flavors of how cooling can work, including but not limited to:

• Air cooling: typically room based or rack based (as described above with hot and cold aisles)

• Underfloor cooling: raised floors to give room for ducts to deliver cool air to servers from below

• Direct to chip liquid cooling: Delivers coolant directly to the CPU or GPU heat sinks via tubing.

• Immersion cooling: Entire servers are submerged in a non-conductive coolant fluid. Can be single phase (where coolant stays liquid) or two phase (where coolant goes between liquid and vapor)

The first two types are table-stakes, but the second two are where the industry is heading. Each type of cooling has its own slew of necessities from the data center, like increased access to water or different maintenance requirements.

There are a few companies worth having on your radar that are building for the cooling market.

Submer: The Barcelona-based company recently raised 50M euros to scale their immersion cooling systems. As shown below, Submer builds tanks where servers sit in a bath of coolant. The tanks are big and require a re-thinking of data center architecture before they are installed, due to different water, fluid and space requirements. Submer makes their own immersion fluid, but is interoperable with those from Shell or ExxonMobil.

CoolIT: the company, acquired by KKR and Mubadala for $270M in May 2023, provides cooling systems that deliver liquid coolant directly to processors that are generating heat. As shown below at right, they build “coldplates” which sit in a low profile on top of a processor. CoolIT also builds rack level solutions, in the form of liquid-to-air coolant distribution units, shown below at left.

I’d suspect that these are two companies that you haven’t heard of, that are building incredible hardware and will be benefiting tremendously from the AI boom.

We’ve got a lot to build

There’s a lot of value to be captured in software thanks to AI. However, building that software requires a massive physical infrastructure built out in order to meet demand. Supercomputing was a view into the world that’s behind the scenes — the processors, wires, cooling tanks, and more that underpin the software-level success everyone’s running after. And we’re just at the beginning.

Subscribe now

Thank you for reading this month’s Day to Data. Just a slice of what I’m thankful for this Thanksgiving week - thanks for being a part of it! See you all in December.

Amodei writes in his piece to contemplate “what a world with powerful AI might look like if everything goes right”. Amidst a throng of pessimists and skeptics, much of which is merited, there is a path to a world where AI’s impact yields tremendous upside.

Dario’s piece delightfully coincided with the announcement of the Nobel Prize for Chemistry being awarded to David Baker of Baker Labs, and Demis Hassabis and John Jumper of Google DeepMind, for their groundbreaking research on protein structure, heavily enabled by AI.

Today, I’m going to discuss AlphaFold - what is does and why it matters - and a rose-colored perspective on a future where humans and AI can benefit tremendously from the speed of progress. We are just at the precipice, and we’re exploring this juncture through one of biology’s toughest problems.

Subscribe now

The protein folding problem

Proteins have been around for an estimated 3.7B years. Proteins start as a string of amino acids, which fold into a precise shape that enables specific functionality. The possible folding patterns are nearly infinite; Levinthal’s paradox suggests that testing every configuration by brute force would take longer than the age of the universe.

Understanding protein structures and their folding process can help scientists debunk protein behavior and function. This knowledge can get us closer to curing diseases that are results of anomalies in the folding process, like Alzheimer’s or Sickle Cell disease, as well as creating target drugs that work with a protein’s shape.

To understand the shape of a single protein, it could take hundreds of thousands of dollars and years of a researcher’s time - but what if it didn’t?

CASP & some healthy competition

There are three questions that scientists have wanted to understand about proteins that encompass the “protein folding problem”:

1. What is the folding code?

2. What is the folding mechanism?

3. Can we predict a protein’s structure, solely on its amino acid sequence?

In 1994, Professor John Moult and Professor Krzysztof Fidelis founded the Critical Assessment of Protein Structure Prediction (CASP) to inspire research teams every two years to test their protein folding prediction methodology.

These methodologies are scored using the Global Distance Test (GDT)— the measure of similarity between two protein structures. In 2018, a brilliant team from Google’s DeepMind entered the 13th CASP competition, and their AlphaFold algorithm ended up beating the previous competition’s winning algorithm by almost 50%. It was an unprecedented outcome.

“We were the best team in the world at a problem the world was not very good at” — John Jumper, CASP 13 winning team member and Director at Google DeepMind

At the following competition in 2020, AlphaFold 2 achieved another record breaking score of 87.0 GDT. AlphaFold was no longer just a computer experiment, but a leap towards better medicine, healthier patients, next-gen biology, and an improved understanding of our world.

How AlphaFold works

AlphaFold follows two other impressive algorithms released by the Google team — AlphaGo and AlphaZero. AlphaGo most famously was an algorithm taught to play Go, a highly complex, strategic game that takes place on a 19x19 board. In 2016, AlphaGo beat Lee Sedol, a worth renowned Go player, in an incredible moment for the DeepMind team.

Biology, and the rules of physics at its core, is pretty beautiful in my opinion. The data about the world around us is full of patterns that are hard to interpret to the human eye, but that computers and algorithms can uncover across massive amounts of data.

It’s clear that AlphaFold 2 is learning something implicit about the structure of chemistry and physics. It sort of knows what things might be plausible. It’s learned that through seeing real protein structures, the ones that we know of. But one of the innovations we had was to do something called self-distillation, which is to get an early version of AlphaFold 2 to predict lots of structures—and to predict the confidence level in those predictions.— Demis Hassabis, CEO of DeepMind

From a one dimensional sequence of amino acids, AlphaFold predicts what the 3D protein shape is going to be, using solely machine learning methods, versus expensive and time consuming methods like X-ray crystallography or magnetic resonance spectroscopy.

AlphaFold was trained on the Protein Data Bank. At a very high level…. I’ll do my best to walk you through it. Using neural networks, the model takes an amino acid sequence as an input, and uses that to search against comparable proteins. Those comparable proteins and structural protein data are fed into an architecture called an EvoFormer, purpose built for this protein folding problem, which is then input to a structure module that predicts 3d atomic coordinates. The model then refines itself, taking one predicted structure and using it as input in the next iteration of prediction, helping the network get smarter with each turn. The output of many turns of this cycle is a set of predicted 3d atomic coordinates for resulting protein from a given amino acid sequence. Here’s a great video with a more technical breakdown of how the AlphaFold algorithm works.

The future with AlphaFold

AlphaFold left a remarkable impression on the research community. They’ve also since released further improved models, AlphaFold 2 and AlphaFold 3.

“AlphaFold changed the game,” Guo said. “In this [2022] competition, almost all groups used AlphaFold as a key component of their systems to make their predictions.” — CASP 15 winning team member, Zhiye Guo

AlphaFold’s impact doesn’t stop at medicine. By understanding protein folding, we can tackle problems like:

• Helping fight malaria, that can improve the efficacy of the malaria vaccine

• Supporting work towards a plastic polluting enzyme

• Predicting the structure of PINK1, a protein that impacts those with Parkinson’s

Getting closer to solving these questions, and the hundreds of other that AlphaFold has gotten us a bit closer to cracking, is a world I am excited to live in.

The “compressed 21st century”

Dario Amodei coined a phrase that really stuck with me in his piece — the “compressed 21st century”.

My basic prediction is that AI-enabled biology and medicine will allow us to compress the progress that human biologists would have achieved over the next 50-100 years into 5-10 years. I’ll refer to this as the “compressed 21st century”: the idea that after powerful AI is developed, we will in a few years make all the progress in biology and medicine that we would have made in the whole 21st century. — Dario Amodei

It’s a good reminder of the value in investing in resilient and adaptable companies, capable of being improved by rapid technological advancement.

We are standing on the brink of a compressed century—a period that might condense decades of progress into just a few years. Like the leap from steam engines to electricity, which took 100+ years, the speed of AI suggests we’re in for a massive acceleration of progress. We are just getting started.

Subscribe now

Thanks for reading October’s Day to Data (though a few days late!). October was a busy month, full of running the Chicago Marathon, enjoying an unseasonably warm NYC fall, and cooking new meals at home! The next few months include a trip to Atlanta for Supercomputing and Vancouver for NeurIPS - if you’ll be in town for either, shoot me a message. See you all later this month.

The video ends with three takeaways:

1. Don’t have debts rise faster than incomes.

2. Don’t have incomes rise faster than productivity (because eventually you will become uncompetitive)

3. Do all that you can to raise your productivity, because in the end that’s all that matters.

Today, we’re focusing on the last one — productivity, AI’s attempts to raise it, and questions left for what’s to come.

How is productivity measured?

Productivity is a key underlying metric to our economy.

Productivity measures how efficiently goods and services can be produced by comparing the amount of economic output with the amount of inputs (labor, capital, etc.) used to produce those goods. Policymakers are interested in productivity because productivity growth is generally the most consequential determinant of long-term economic growth and substantive improvements in individual living standards. — the Congressional Research Service

Productivity is fundamentally calculated as the GDP (gross domestic product) per worker. The United States has largely outranked countries like France, the UK, Canada, and China in this statistic (see graph below for trends since 1991).

AI has promised, across nearly every industry, that we can do more with less, hence improving economic productivity. Each person you hire will generate more positive business outcomes or you can generate the same amount of outcomes with fewer people. One may assume this new paradigm will improve our productivity, but this hasn’t always been the case with major technological shifts.

Subscribe now

The 1970s productivity decline

Economists and technologists have always wanted to better understand the connection between tech innovation and economic impact. In the latest print of Nate Silver’s “The Signal and the Noise”, he writes about the 1970s and the moments where new technology actually inhibited our productivity:

Many thought the computer boom of the 1970s would dramatically impact the nation’s productivity. Instead the era “produced a temporary decline in economic and scientific productivity,” wrote Silver.

“You can see the computer age everywhere but in the productivity statistics.” said economist Robert Solow in 1987

Silver goes on to explain — “The 1970s were the high point for ‘vast amounts of theory applied to extremely small amounts of data’”. He summarizes: We face danger whenever information growth outpaces our understanding of how to process it.

We are amidst an age of outpaced information growth. However, when we look to the 1960s-1970s, while productivity may have declined, the period was a breeding ground for some of the most lucrative and long-lasting businesses in technology today — semiconductors. Fairchild was founded in 1957. Intel was founded in 1968. Through periods of great innovation and misunderstood information may come a short term decline in productivity, followed by longer term periods of sustained growth.

The immediate productivity outcomes of AI

AI has shifted what productivity can look like. As coined by Sarah Tavel of Benchmark, the new call for AI startups is to “sell work, not software”.

Sarah Tavel's Newsletter

AI startups: Sell work, not software

For the past 25 years, application software startups have had a singular focus: increasing company and employee (including developer) productivity. This looked like building software that increased productivity at the employee level, increased collaboration across employees and teams, and/or enabled better oversight and management at the leadership leve…

Read more

3 years ago · 221 likes · 22 comments · Sarah Tavel

I recently listened to a podcast with Bret Taylor, CEO of Sierra and chairman of the board at OpenAI. He tee’d up the pain behind a use case that’s been quickly addressed by many startups, including his, around customer support.

The cost of having direct conversations with all of them [your customers], it's tremendously expensive. Usually, that involves building a contact center, either in-house or with a BPO or an outsourced firm. It probably means that you're overstaffing it to handle peak seasons. And it involves huge amounts of training for the workforce in that call center because every time you have a new product or new process, you need to train everybody, and it becomes a huge cost for your company. — Bret Taylor, Invest Like the Best

Now with AI, setting up a call center can be as easy as setting up any other SaaS tool. Thanks to text-to-voice models from companies like ElevenLabs and AI call companies like Bland, a typical cost center for businesses can become a flexible, scalable, AI-driven solution. The efficacy, accuracy and cost at scale are still in question. Many companies are bundling these technologies, selling the work of a call center, not software.

With $17M of fresh funding this week, Hippocratic AI is building agents for a far more expensive field — healthcare. Their team is building secure, healthcare specific agents for roles across pharma, dental, and more. Reporting shows that $82.7B is spent on healthcare admin between payers and providers - which is a fraction of the broader administrative burden on the $4.5T+ healthcare market. Hippocratic can chip away at this preposterous number. More selling work, not software.

There is massive economic value at stake. These are step change level improvements. We are amidst a gold rush in value creation. A senior GS economist estimated that “AI will ultimately automate 25% of all work tasks and raise US productivity by 9%”, thought AI adoption remains nascent.

It’d be naive to not present some skepticism — in a recent Upwork survey of 2.5k full-time workers, 77% who use generative AI in their jobs said it has added to their workload and is hampering their productivity due to time being spent review AI generated output and getting up to speed with new technology.

A reasonable reaction would be to fear job loss and a concentration of value and resources among those able to invest in and shape AI. I share a perspective well stated by a friend of mine Kojo who wrote that startups should aim to “sell aspiration, not efficiency”:

You don't want to fire 20% of your developers; you want to build 20% more features. You don't want to fire 20% of your SDRs; you want to close 20% more deals. The desire to do more with new tools is an aspirational quality deep in our psyche. The most compelling AI products tap into this mentality and lead with aspiration. — Kojo Osei

This won’t be cheap

An estimated $1T is expected to be poured into AI over the next decade.

The trillion dollars pouring in is for everything from hiring top researchers to making commitments with renewable energy providers to power data centers. AI is causing many to rethink the entire compute supply chain in a dramatic way — not just power and energy, but critical minerals like copper, the availability of water used for cooling data centers, and manufacturing capacity to produce everything needed. We are gobbling up resources and shelling out capital in the process.

• Taiwan Semiconductor Manufacturing Company (TSMC) is investing $65B in building their Arizona fabs for chip manufacturing, creating over 6k jobs.

• Google is utilizing a pool of fast-vesting stock grants worth up to millions per person to entice its lucrative AI talent at DeepMind not to leave for competitors.

• Amazon invested $4B into Anthropic, a foundation model provider, to bolster their position as a preferred cloud for AI developers.

• US VC funding jumped to $55.6B in Q2 2024 from $37.8B, largely driven by investments in xAI, $6B to Elon Musk’s company, and Coreweave, $1.1B to the “AI hyperscaler”.

• This week, Microsoft announced its participation in the Global Artificial Intelligence Infrastructure Investment Partnership, alongside BlackRock and others, to bring together $100B to invest in energy infrastructure for AI workloads.

• Larry Page, founder of Google, reportedly has said that he would rather have Google (1.94T market cap) go bankrupt than miss out on the value that could come from AI.

This level of activity in the market demonstrates what many believe — there is simply too much opportunity to leave dollars left unspent.

However, the focus on spend may be incorrect. Arguably, the more relevant metric is dollars spent versus company revenues:

“Cloud computing companies are currently spending over 30% of their cloud revenues on capex, with the vast majority of incremental dollar growth aimed at AI initiatives. For the overall technology industry, these levels are not materially different than those of prior investment cycles that spurred shifts in enterprise and consumer computing habits. And, unlike during the Web 1.0 cycle, investors now have their antenna up for return on capital. They’re demanding visibility on how a dollar of capex spending ties back to increased revenues, and punishing companies who can’t draw a dotted line between the two.” — Eric Sheridan, Sr. Equity Research Analyst at Goldman Sachs

Will it be worth it? Let’s look to the Bitter Lesson

In this new world, two things seemingly appear to rule: compute and data — and hence, capital. As Rich Sutton proposed in the Bitter Lesson, “the biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin… the only thing that matters in the long run is the leveraging of computation”.

More compute & more data will result in a higher performing model. A better model will mean more reasoning and understanding, which will enable tackling of tasks to truly shift the productivity levels of workers of all skillsets. And all of this takes a tremendous amount of capital.

One of the best phrases to describe the way I feel amidst all the AI noise shockingly comes from C.S. Lewis, who wrote, “isn't it funny how day by day nothing changes but when you look back, everything is different?”. While today our lives may not feel supercharged by AI just yet, the effects will begin to ripple down, and how we work, live, communicate, and connect will be fundamentally different in the future.

So back to Ray Dalio… This is a moment of productivity explosion. We are doing all we can to increase productivity, because in the end, that’s all that matters.

More thoughts to come. I’d love to chat with any folks pondering similar questions and hear your take. Otherwise, thanks for reading today’s post. I’ll be back next month - turns out longer form, a bit more researched writing is more my speed. See you in October!

Subscribe now

Barely at all. Until 2023.

In 2023, projected new energy demand skyrocketed, nearly tripling the 2022 forecast. I was at a dinner with some investors a few weeks ago and shared this tidbit. I was met with a response so dramatic I had to fact check myself to make sure I was remembering correctly. 

Subscribe now

The catalyst behind much of this demand increase is data centers. In Northern Virginia alone, they need “the equivalent of several large nuclear power plants” to meet the electricity demand coming from data centers. Data centers have been around for decades, but in the past few years, it’s become clear that we need plenty more of them.

Behind the promised outcomes of AI is a bit of a dirty secret - it is almost unbelievable how much power it requires to not only train powerful large language models but to continue to serve end uses to billions of users around the world compared to previous methods of search and web indexing.

Even on a small scale, the increase in power required is colossal - a ChatGPT query is 6-10x as power intensive as a traditional Google search (prior to their integration of Gemini into search). If the ~200M users ChatGPT users were only sending one query a day on average, it’s dizzying how much demand the platform is putting on our fading energy resources.

On a larger scale, all 2,700 data centers in the US accounted for more than 4 percent of the country’s total electricity consumption, predicted to jump to 6% by 2026. Google’s data center electricity consumption alone jumped 17% in 2023. It’s been estimated that power required to train GPT-4 was equivalent to that of powering 1300 homes for a year (this is just for training - more impact comes from inference).

Power markets are responding. PJM, one of the largest electric grid operators in the US, had their power capacity auction at the end of July. Capacity in this instance means acquiring supply for resources on the grid to meet demand in the future. PJM goes and procures power supply from generators (i.e. one hydroplant commits to providing 1,000 megawatts of power, another for 15k megawatts) to match the demand they expect for the following three years, then a price is set for the power, with a higher price signaling the need for more energy to be produced. At the end of July, PJM’s price jumped to an all-time high, of $269.92/megawatt-day, up from $28.92/megawatt-day at the last auction. The signal was clear — “the market is sending a price signal that should incent investment in resources," PJM CEO said.

Hyperscalers are getting greedy

In order to feed the beast, there’s a frenzy erupting over precious energy resources across several key players — hyperscalers (primarily Amazon Web Services, Azure, and Google Cloud), power providers, and real estate owners with premium access to the grid.

It is estimated that hyperscalers will capture 50% of the data center market in the next 5 years. Microsoft, Amazon and Google’s businesses have become more dependent on the build out of data centers than ever before. In 2022, Microsoft Intelligent Cloud accounted for 38% of revenue, bringing in ~$75B of $198.27B of total Microsoft revenue. In 2023, it jumped to ~46% of total revenue for Microsoft.

In a world where GPUs are seemingly the most valuable resource, our energy needs may have found a resource that may just be even more valuable over the next decade - land. And not just any land, but land with premium access to power.

The hyperscalers are paying up for the hottest commodity it appears. At the end of 2023, Microsoft paid a family in Wisconsin nearly $70M for over 400 acres of land (which was previously valued by the town at $600K), complete with a pumpkin farm included in transaction. Microsoft has continued to gobble up land in Wisconsin, owning nearly 1200 acres as of this week, to support their plans to build a mega-AI data center hub in the state.

With the impending scale of this build out, hyperscalers are also buying up energy commitments to meet AI demand as well as their carbon-free goals. On May 1, 2024, Microsoft signed the largest corporate power purchase agreement in history with Brookfield Asset Management when they committed to development of 10.5 gigawatts of renewable energy. As of January 2024, Amazon had been the world’s largest corporate purchaser of renewable energy, with more than 500 solar and wind projects globally generating enough power for 7.2 million US homes each year.

Subscribe now

The future is distributed

Many are looking to new infrastructure to provide relief to the strained system — virtual power plants may play a role. Virtual power plants aggregate energy that has been produced across distributed generators, such as solar panels on an apartment complex and a wind turbine on someone’s farm. The Department of Energy predicts that 22.5% of peak load demand will be met by virtual power plants by 2030. Going distributed could mean a massive shift in our existing energy markets in a multitude of ways, some best described by Azeem Azhar of Exponential View, quoted here:

• “A geographical redistribution of economic power: Regions with abundant renewable resources, such as windy coastlines or sunny deserts, could become new industrial hubs.”

• “Seasonal energy arbitrage: Businesses will adapt to seasonal energy patterns. Imagine OpenAI running its AI model training during peak solar months, or Tesla producing batteries when wind power is cheap.”

• “Increased grid resilience: A network of distributed energy sources is less vulnerable to large-scale outages, improving overall energy security.”

Exponential View

🌅 Dawn of the distributed age

Two weeks ago, the Economist wrote a special edition on the “dawn of the solar age” in which they argued solar will become the dominant source of power, changing the global energy landscape. They are not wrong (as all EV readers would have known for years), but it’s much more than that… it’s the dawn of distributed energy…

Read more

2 years ago · 81 likes · 4 comments · Nathan Warren and Azeem Azhar

We’ve been wrong about this before.

In the dawn of the internet age, there was a gold rush, that was likely similar to the AI gold rush we’re amidst right now. There was an urgent demand to meet the energy needs that the new technology would require. In May 1999, Forbes published an article titled “Dig more coal — the PCs are coming”.

However, much of this expected demand didn’t come to fruition. Take a look at the chart below from the Congressional Research Service regarding energy consumption showing little to no growth in energy consumption from 2000 (after the Forbes article was written) to 2019 (pre ChatGPT).

This lack of growth was a response to improvements in both energy conservation (using less) and energy efficiency (finding more effective methods for energy use). Energy efficiency is a good reminder of Jevon’s Paradox —

In the long term, an increase in efficiency in resource use will generate an increase in resource consumption rather than a decrease.

This can be applied to what we are seeing in energy — we’ve improved efficiency, but that just means users are ready to use more energy. So what if we over-respond to this energy dilemma on our hands? When asked about over-investment in AI on an earnings call, Sundar Pichai, CEO of Google, claimed that the risk of missing out on the benefits of AI outweighs any belief that they are over-investing. For those drawing parallels to the cloud computing build-out, and predicted energy demand that came with it, Jenny Grimberg from Goldman Sachs asked Brian Janous, prev VP of Energy at Microsoft and now founder of Cloverleaf Infrastructure, for his outlook:

Is our current power grid enough?

It’s unlikely our grid alone will be able to support our energy future. One writer, Justin Etheredge, said it best —

The power grid is the largest, most complex, and most important machine ever constructed. It is the network that enables everything we have in modern society, and yet we are scarcely aware of it until it stops working. It is quite literally the platform that our lives are built on.

We’ve got to be aware of it before it stops working. We are at the precipice of a new era, perhaps one where acute awareness of the grid and our need for renewable energy becomes the norm.

Thanks for reading this month’s Day to Data. I’ve been spending a lot of this summer digging into newer themes and areas outside of my comfort zone - power being one of them - and writing has been on the back burner. Day to Data is on summer vacation, but will be back to regular programming in September. See you all then!

The idea that we have preferences, whether we’ve offered them directly or they’ve been been inferred, that enable our favorite systems to learn more about us and recommend something back to us is a fundamentally simple idea, but one that has powered hundreds of powerful use cases from finding your partner on Hinge to your favorite neighborhood spot on Yelp.

LLMs (large language models) and VLMs (vision language models) have fundamentally changed the way we can build recommendation systems and today I want to spend some time thinking about how this underlying tech will power the next generation of recommenders.

It’s a pleasure to collaborate with Neel Kulkarni on this article. An email from Neel landed in my inbox a few months ago, full of his thoughts on this space. As we’ve jammed on ideas since, his insights have been crucial to putting this piece together. Now for an intro on Neel to lead us into this week’s article!

Hello! I’m Neel, a rising senior at the University of Virginia, a diehard Celtics fan, and an avid reader of Day to Data. In this week’s edition of Day to Data, I’m excited to be a small part of this piece on recommendation systems and discuss its ability to create powerful consumer products, as well as how AI is continuing to innovate this process to better understand the next generation of users. 

Subscribe now

The evolution of the recommendation system

Recommendation systems have evolved in terms of how users interact with them:

1. Surfacing semi-static recommendations to a user through “picked for you” sections of content perhaps based on their preferences up to that day, but likely not in real-time, like Yelp or Netflix.

2. Recommendations powered by how we interact with previous visual or video content, with quick feedback loops iterating on the recommendations in real time like TikTok, Tinder, and Styl.

3. Hyper-aware and contextualized recommendations that exist within the native interface for an application.

Netflix and the Netflix Prize pioneered an early era of the “Recommended for You” algorithm and interface. Yelp followed suit, making a directory of local spots - from barbers to lawn care to restaurants - and trying to get community reviews to amplify what was best. But people craved better recommendations and more social features driven by people they knew.

I think Yelp walked so Beli could run in version 1.0 of recommendation system interfaces. When we last discussed Beli, the app had 3K app store reviews - today it’s got over 6K. Beli lets users list, rank, and rate restaurants to share with their community of followers. The app recommends your next dining experience by cross referencing your preferences with your friends that have similar preferences. Beli’s been able to see tremendous growth (and likely monetization) through their social media pages, sharing NYC restaurant recommendations to over 800K followers. As they’ve found compelling signs of product market fit, the next steps will likely involve in-app monetization, perhaps through charging restaurants to advertise on the platform, providing or matching customers to restaurants they’ll love through a Seated-esq approach.

In the version 2.0 of recommendations, we got Tinder and TikTok. They profited (tremendously) off our short attention spans and built algorithms that quickly ingested a user’s feedback and updated their desires with every swipe. These gestures of swiping and jam-packed feeds quickly moved out of dating and video creation and into new mediums, like clothing.

In 2022, as Dhruv shared in early 2024 on Twitter, H&M launched its own version of the Netflix Prize, paying $50K to anyone who could improve the fashion recommenders out there. What unfolded was Styl, which (as of March 2024) has 18k MAUs who have swiped 4.5M times on a piece of clothing. Styl is focused on women’s clothing and best described as “Tinder for Clothes”. The app tracks user’s preferences with every swipe left or right, building a powerful, quick-to-improve recommendation system. Styl’s even tracks trending fashion items at individual colleges across the nation, showing users what their fellow classmates are swiping on most. As for the future of this interface, perhaps they white-label this tech so that every retailer has “Recommendations powered by Styl”, much like what H&M was likely searching for with their 2022 call to action. But for now, Styl is balancing the inundation of fashion with an easily accessible interface that users know and love - the swipe-centric era of recommendation systems — version 2.0.

From natural language to your plans for the night

On to 3.0 and what’s to come. The future of recommendations will not look like those we know today.

Large language models provide an unbelievable unlock when it comes to understanding how to take strings of text data and truly understand what’s being conveyed in the natural language. “Date night for two in the West Village” can now lead to a booked reservation at Balaboosta for 7PM this Friday (a must try!!). The step-change improvement is in using an LLM to break down topics and context from a given piece of text, versus previous methods like Named Entity Recognition (NER). If you break it down, it can be represented like this for a given prompt: “date night in the west village for 8PM with vegan friendly options”

The unlock of LLMs means that recommendations no longer need to live in silos, based off a handful of datasets, and be unilateral. Recommendations can take a variety of styles of inputs in natural language and give on-demand recommendations.

I’d even venture to say that we’re actually no longer discussing recommendation systems, but rather the future of a hyper-aware, context-rich environment that can help us make decisions. If you power a system with not just user data, but contextual data from the surrounding environment, the model that powers that user’s end application becomes all the more valuable. We can inject contextual data - apps, sentiment, emotions, etc. - into the infra stack of the future for recommendations.

There’s tremendous opportunity. Here’s what we’re excited for:

• Recommendations will be hyper-contextualized by data from apps in the surrounding environment.

• Recommendation systems becoming conversational and more like the way humans navigate decision making, like Where Should I Eat NYC.

• Improved technology around translating input into preferences that better reflect the user’s intentions and goals, like Claro and Honcho

• Embedding recommendations into native, lower-level systems, living collaboratively amidst all your apps and becoming a real human-computer interface like Asta and what much of Apple Intelligence will likely feel like.

To close out with a “red-team” on this perspective, perhaps over-optimization will result in undesirable outcomes. In a tweet from Scott Belsky of Adobe:

Recommendation systems just seem to be the one topic I can’t get enough of! We’ll see how this one unfolds and report back with our learnings. One more big thanks to Neel for collaborating on this article, and thanks to you all for reading!

The evolution of language models

Language models have existed for decades. They’ve become significantly larger over time (hence the “Large” ones you’ve heard about lately). In an article I wrote in early 2023, I went over some of the historically significant predecessors, like ELIZA and IBM’s Watson, which even competed on Jeopardy.

Large language models, like GPT-4 from OpenAI, have exploded in popularity and early use cases are really promising to drive value. Enterprises are hungry to implement the technology. However, many implementations are never making it to production because of concerns around how a customer-facing LLM will perform “in the wild”. These concerns have spawned an entire industry building LLM guardrails (like Guardrails AI), firewalls (like Arthur Shield), and evaluation tools (like Patronus AI). All of these techniques serve different purposes.

In terms of evaluation, early benchmarks for LLMs measured things like accuracy and perplexity, like SWE-Bench for resolving Github issues. Many early benchmarks struggled to capture how much an output from an LLM aligned with a user’s preferences. With this issue at the core, researchers were off to the races, and one of the promising releases targeting this problem was LLM-as-a-judge. Let’s dive in.

Subscribe now

TLDR; alignment of LLMs with human preferences

(for my non technical audience, TLDR means “too long, don’t read”, and is typically followed by a short blurb describing what’s to come if you don’t want to read further!)

There is a need for a scalable, automated, and explainable method to evaluate LLM alignment based on human preferences. There is a gap with existing benchmarks on LLM performance metrics - they evaluate LLMs on standard performance metrics, but do not evaluate the LLM on human preference.

Take a look at the question below in red and the two responses of different LLMs. While both responses provide the right “answer”, Assistant B responds in a method that’s aligned stylistically to the question, more succinct, and follows the directions of the prompt more clearly, which asks for the answer only. LLM-as-a-judge is aiming to encourage LLMs to return optimized, preferred responses like Assistant B.

To do so, researchers propose that we can actually use LLMs to judge other LLMs and evaluate models. In fact, instead of using just one LLM to judge, they propose using several “LLM judges” by introducing two new benchmarks for performance: MT-Bench and Chatbot Arena.

MT-bench is a series of 80 high quality, open-ended questions that evaluate a chatbot’s multi-turn conversational and instruction-following ability. Chatbot Arena is a platform where users engage in conversations with two anonymous models at the same time, posing the same question to both, and rate their responses based on personal preferences. From their research, it suggests that LLMs and a human can actually agree on the quality of an output more frequently than two humans! These methods are a promising step in the direction of more “human-like” LLM evals.

Subscribe now

Now on to the more technical stuff….

How do we make LLMs smarter?

LLM-based chatbots saw a major performance improvement with the use of RLHF and fine-tuning. Once humans add their input through these processes, humans tend to like the outputs of the chatbot more. However, this increased alignment and preference does not correspond to improved scores on existing LLM benchmarks like HELM and MMLU. Researchers believe this suggests there is “a fundamental discrepancy between user perceptions of the usefulness of chatbots and the criteria adopted by conventional benchmarks”.

In response, this paper announced two new benchmarks - MT-Bench and Chatbot Arena.

MT-Bench

MT-Bench is a series of 80 high quality, open-ended questions that evaluate a chatbot’s multi-turn conversational and instruction-following ability – two critical elements for human preference. There are 8 categories — writing, roleplay, extraction, reasoning, math, coding, knowledge I (STEM), and knowledge II (humanities/social science). The responses to the 80 questions in MT-Bench are reviewed to measure the performance of the model in alignment with human preferences.

Chatbot Arena

This benchmark, Chatbot Arena, is a crowdsourced platform featuring anonymous battles between chatbots in real-world scenarios. Users engage in conversations with two anonymous models at the same time, posing the same question to both, and rate their responses based on personal preferences. They vote for which model provides the preferred response, with the identities of the models disclosed post-voting. Take a look below to get an idea of what Chatbot Arena can look like.

How do we get the judges to agree?

As you may have suspected, there’s some questions around what it means for all these LLM judges to “agree” on the best responses. For two judges (can be LLM or human), agreement is defined as the probability of randomly selected individuals of each type agreeing on a randomly selected question. GPT-4 and humans actually agree more often than just humans. Results reveal that strong LLMs can achieve an agreement rate of over 80%, on par with the level of agreement among human experts, establishing a foundation for an LLM-based evaluation framework.

What are the limitations?

1. Position bias — sometimes LLM-as-a-judge just likes the first response it reads more. This is actually not unique to LLMs - humans do this too.

2. Verbosity bias — an LLM judge occasionally prefers longer, verbose answers, even if they aren’t as clear

3. Self-enhancement bias — GPT-4 favors itself with a 10% higher win rate, Claude-v1 with 25% higher

How can I use this?

Want to use this tool for your own LLMs? Platforms like OpenPipe enable the use of LLM-as-a-judge to gauge performance of your LLMs. I haven’t used the tool myself, but engineers I’ve spoken to have spoken really highly of what they’re building here. OpenPipe completed the S23 batch of YC and raised a $6.7M seed round in March.

For enterprises, Databricks’ LLM-as-a-judge metrics were launched in MLflow in October 2023. They recommend using LLM as a judge as a companion to human evaluators.

LLM-as-a-judge is one promising tool in the suite of evaluation techniques necessary to measure the efficacy of LLM-based applications. In many situations, we think it represents a sweet-spot: it can evaluate unstructured outputs (like a response from a chat-bot) automatically, rapidly, and at low-cost. In this sense, we consider it a worthy companion to human evaluation, which is slower and more expensive but represents the gold standard of model evaluation. - Databricks

That’s a wrap!

Thank you all for reading this week’s Day to Data. As I shared earlier, I’ll be slowing things down here at Day to Data to a monthly cadence until September. I’ve got some long term projects I’m excited to focus on and will be back and better than ever going full speed this Fall! In the meantime - follow me on X at @gabyllorenzi.

This week was a busy one in the world of tech - most notably featuring OpenAI’s Spring update and Google I/O. While OpenAI’s announcements were nothing short of jaw-dropping, Google’s got me thinking a bit more. Today’s post likely will yield more questions than answers, but that’s representative of where I’m at with a lot of the aftermath from the announcement as only time will tell how this plays out.

Bringing AI to the front-page

Google I/O 2024 was a splashy AI-centric event that showcased plenty of new features coming to users this year. For our own sanity, we’re going to focus on one today - Google’s announcement of adding AI-powered overviews to its search results. Though subtle in its appearance, this update to Google’s search platform will most definitely affect what much of Google’s business is built upon - SEO and ad revenue. As I said, knowing the effects of this will take time and are up for debate, but today we’ll discuss potential implications of this update to their platform.

What does this mean for digital media, bloggers, and other businesses that rely on SEO to push traffic to their websites?

It’s crazy that a year ago, Bing AI announced their AI-powered search functionality. This Tuesday made it feel more real than ever that it’s not an if but a when will AI-powered functionality have an impact on traffic to web pages. A lot of us are scratching our heads and wondering - is an AI search engine actually the outcome we all want?

Google claims that the AI overviews resulted in users conducting more searches than without the technology “because they suddenly can ask questions that were too hard before,” said Liz Reid, who oversees the company’s search operations, while also declining to provide specific numbers to back the claims. I don’t buy that more searches means more clicks to web pages. More searches means more opportunities for AI to summarize the content you’re trying to see.

In order to maintain defensibility from the potential implications of AI dramatically reducing your web traffic, many bloggers are now moving to modes like Substack (how you’re reading this right now!), Beehiv, and other newsletter platforms as well as starting podcasts to start owning their audience and how they are able to engage with them. As always, the Hard Fork team had some good response to Google’s announcement on this week’s episode. It’s important to get a perspective from the journalists and reporters whose business will directly be impacted by these changes to Google.

What does this mean for Google’s ad business?

In Q1 2024, Google’s advertising business grew to $61.66B YOY, an increase from $54.55B YOY in Q1 2023. Google benefits more than anyone by having a robust advertising business. This means that Google should be incentivized to have traffic heading to websites where there ads are sitting, driving money back into their ads behemoth. So why would they release functionality that deters people from using those websites? It’s an open question in my mind, and one that points to the future of a consolidated internet, where even fewer are in the driver’s seat than there are today.

As Casey Newton said on this week’s Hard Fork, Google is ultimately able to pull the levers, swaying the traffic in the way that will yield the most positive outcome for them. The trickle down effects of this change to their site may be bigger than any of us are expecting.

How does this effect LLMs in the future, that have previously depended on text from the web to be trained on?

I’ve discussed Common Crawl before, but for a quick refresher, this free, open dataset contains over 250 billion pages spanning 17 years of web data. This data was the basis of many LLMs, including GPT-3. While a small percentage of this data may have been artificially generated, much of this data was written by humans, which is what made the LLMs trained upon this dataset feel more human in their responses.

It’s still unclear what the long term effects will be when more AI-generated content is fed into the training data of LLMs. Early research tells us things don’t look good. Researchers have discovered that when LLMs are trained on their own outputs, the model goes “MAD”, which stands for Model Autophagy Disorder. This basically means that a model “collapses” aka the model “loses information on the tails (the extremes) of the original data distribution, and starts outputting results that are more aligned with the mean representation of data”. In normal speak, models are becoming more generalized, and their responses are less varied across prompts. For this reason alone, we should be incentivized for the web to contain content primarily written majority by humans. So what does this mean for LLMs for the future? Yet another open ended question :)

Is SEO dead?

Every day when I swipe on Instagram, I get at least one ad or reel of someone pushing some SEO (search engine optimization) course that helps people improve the search rankings of their digital content. I’ve been seeing less and less of these, perhaps because my algorithm has learned that I’m not the target audience, but perhaps because SEO as we know it is changing dramatically.

Subscribe now

Anyone creating content and publishing it on the web was once concerned with how highly their page would rank in a Google search and how SEO-friendly their content was. Now, they are in a frenzy, trying to optimize their content to be picked up by new AI features on sites like Google and Perplexity. Or they’re using AI to spin up hundreds of websites, making content for bespoke topics in order to drive more engagement to their site.

SEO will exist as long as the indexed web exists. Some food for thought for you readers is how long will the indexed web exist? Well what happens when we stop engaging with the web in the way we do now? In an agentic* future, in order to book a trip, you’ll ask your virtual travel agent and they’ll do all the browsing. You can ask a personal shopper to flag relevant purchases for your next vacation. When you need a question answered, Perplexity and Google will answer it in one shot. I believe we’re farther away from this future than the technology may lead us to believe, but I wouldn’t doubt if more people are using agents to navigate the web and get information than ever before in the next 5 years.

*An “agentic” future refers to a world in which software is able to take a task, break it down into steps, and complete them on behalf of a user, without the user ever participating in the intermediary steps. If you want more on that, Andrew Ng’s talk to Sequoia is a good watch. I’ve got a whole series on agents coming soon!

That’s it for existential questions… for now!

If you’ve made it this far through probably one of the most open-ended articles I’ve ever written, I appreciate you! Every now and then you’ve got to muse on the big, impossible to answer questions. I’d love to hear folks thoughts on what this all means for journalists, website owners, and you and me. Find me @gabyllorenzi on X. Thanks for reading!

GDPR: the UK’s data protection regulation

You’ve potentially heard of something called General Data Protection Regulation, better known as GDPR. It is regulation, implemented in May 2018, that applies to companies based in the EU that process personal data and to companies established outside the EU that offer their services and monitor user behavior in the EU. If the regulation isn’t complied with, the fines are no joke. As of February 2024, almost €4.5 billion worth of GDPR fines have been paid by ~2000 organizations. The largest fine was a whopping €1.2 billion paid by Meta in 2023 for mishandling user data that was being transferred between the US and EU.

If you want to read GDPR in its entirety, find it here. I’m a few chapters in!

GDPR set a new precedent in protecting users and enforcing businesses to comply with strictly outlined data privacy regulation. There are important tenants to GDPR that made it a major advancement in data regulation.

Some highlights of the regulation —

• GDPR requires data protection by design and default.

• Data minimization — When implementing a new data application, regulation requires that “only personal data which are necessary for each specific purpose of the processing are processed”.

• Accuracy — Personal data must be kept accurate and up to date.

• Storage limitation — Personally identifying data can only be “stored for as long as necessary for the specified purpose”.

• The right to be forgotten: a user has the right to request the erasure of personal data concerning them “without undue delay and the controller shall have the obligation to erase personal data without undue delay”.

There’s been mixed reviews on how impactful GDPR has been. Consumers just navigate a jungle of “do you accept these cookies?” buttons on every website they enter while in Europe. There’s a new terms & conditions document to read with every new application. There’s been studies on how long it would actually take a consumer to read all the policies they are in agreement with — in the UK, Microsoft Teams would take 2 hours and 27 minutes to read. Even Candy Crush’s policies would take 1 hour and 53 minutes to read. And for businesses, small to medium businesses have been having to increase spend on data privacy teams, cutting into margins, whereas large enterprises like Meta & Microsoft have adequate talent resources and can pay fines when they have to.

Subscribe now

The US might get its own version of GDPR

On April 5th, American lawmakers brought forth a proposal for bringing sweeping data policy to the United States through new regulation. This proposal was bipartisan, proposed by Senate Commerce Committee Chair Maria Cantwell (D-Wash.) and House Energy and Commerce Committee Chair Cathy McMorris Rodgers (R-Wash.). The proposed act has been dubbed the American Privacy Rights Act (APRA). US states have been more successful in passing privacy legislation. If APRA is passed, it will rival the GDPR to become a leading global privacy standard.

It was a bit of a surprise to much of the public that APRA was brought forward — an American data privacy regulation has been in the works for years with no success. APRA’s predecessor, the American Data Privacy Protection Act (ADPPA), got the farthest along in any approval process by passing committee but failed to pass in the House. There are so many consumer privacy regulations in the works - most recent list here. Last April marked the 36th hearing the U.S. Congress has held on privacy and data security over the last five years.

A few highlights of ARPA include the following —

• APRA prohibits the transferring of sensitive covered data to a third party without the affirmative express consent of the individual to whom such data pertains.

• APRA would prohibit the use of covered data to discriminate against consumers and provide consumers with the right to opt out of the use of algorithms for consequential decisions.

• Small businesses (those making <$40 million over three years or processing the data of fewer than 200,000 individuals annually), and a small subset of other entities, are excluded from abiding by APRA.

Flipping the script: letting users define privacy preferences

I was listening to a podcast with a British lawyer where she said something that stuck with me about how the future of privacy standards could try shift to.

What if, instead of users accepting cookies on a website, websites had to accept the user, based on the privacy guidelines they’ve established for themselves? This got me thinking - what if we flipped the script? Perhaps when you got a new computer, or when you logged onto the internet in a new country, the user could define their preferences in a “privacy passport” of sorts. We could define what we wanted our data to be used for, or not be used for, how we wanted our data to be handled, and how long we wanted our data to be remembered for. Then, we’d navigate the internet and see where our preferences met those of the web. I’d presume two problems with this approach are that 1) users want way less to be done with their data then companies would be willing to adapt to and 2) there’d be a learning curve for users to even define their own personal privacy standards.

What’s next for privacy?

On March 13, 2024, the European Parliament adopted the AI Act, which is “considered to be the world’s first comprehensive horizontal legal framework for AI”. For violators, fines equate to either 7% of global revenue or €35M, whichever is larger (GDPR fines are the higher of 4% of global revenue or €20M). The AI Act will cover a very broad set of companies, including those developing AI systems, deploying AI systems, and those distributing AI systems in the EU. It’s worth dedicating an entire post to the future of AI regulation, in the EU, US, and globally, so I’ll leave this as a cliff hanger for the next data privacy article!

We’ll have to see what unfolds with APRA and the future of data privacy, particularly in the United States and with the rise of generative AI. It’s taken years of legislation to for the US to get to this point, so we’re definitely not at the finish line quite yet. And it will take decades, as it has with GDPR, to study the true effects of such monumental legislation. More to come. Thanks for reading!

Subscribe now

Hello from Copenhagen! If I’m keeping myself honest, I’m a week late on getting out a new article, but for good reason - last weekend, I raced my first half marathon! Turns out half marathons are a whole different beast then the marathon, so still lots of work to do for the next one, but a fun day nonetheless.

Today, I’m writing in from across the globe in a new city for me - Copenhagen! This city is quaint, full of colorful architecture, incredibly walkable and very hygge. But perhaps the most notable thing about Copenhagen is just how well designed the city is. It’s easy and safe to bike, thanks to dedicated bike lanes on every street. The metro, which opened in 2002, is easy to navigate and incredibly clean. The metro cars in the City ring are even driverless! And an estimated 62% of the Copenhagen population commutes via bike every day.

Being in Copenhagen got me thinking - where can we use data to better understand and optimize our cities? How do our cities affect how connected we are? How does a city like my home of New York compare to others? For today’s Day to Data, we’re exploring one research team’s mission to use data to better answer these questions.

Defining the connectivity of our streets

In 2020, Adam Millard-Ball and Chris Barrington-Leigh wanted to come up with a metric to measure how “disconnected” the streets of various cities were. They focused on sprawl, which was a measure of this connectivity. Their resulting metric was SNDi — street network disconnectedness index. A mouthful, I know!

We measure sprawl through the connectivity of the streets. Sprawl is characterized by poorly connected streets – a maze of cul-de-sacs and loops, where people are dependent on private cars. Well-connected streets – like New York City’s grid – are more walkable and can be served by public transit.

Streets and streets of data

As most research does, they needed data to start. So what data was required?

• OpenStreetMap — you can find the open global map here

• Satellite-derived time series data of urbanization — researchers have used satellite imagery to measure the speed of urbanization

Using this data, they identified, down to the street level, how connected cities were. They classified an area into 1 of 8 degrees of “connectedness”, shown below, as defined by characteristics of being circuitous, having dead ends, or being grid like. They also did this classification over time, seeing how cities evolved from 1975 to 2020.

From Lagos to Los Angeles, the world continues to sprawl

And with these findings, the research team created an interactive map of the world colored with the SNDi index, which you can explore here. My first instinct was to see how Copenhagen and New York City compared - they’re actually much closer than I thought! The blue regions on the map are lower values of SNDi, which means more highly connected regions. The red regions, really only seen on the left map below, are highly disconnected regions.

Some of the interesting findings from the team included:

Streets in new developments in 90% of the 134 most populous countries have become less connected since 1975, while just 29% show an improving trend since 2000. The same period saw a near doubling in the relative frequency of a street-network type characterized by high circuity, typical of gated communities.

Grids (type A) are the most self-evidently connected type. In the pre-1975 period, North America and Latin America were home to the highest proportions of type A (grid), at 9% and 13%, respectively. Since then, grids have declined in importance in Latin America and almost disappeared entirely in recent North American development, with that decline only partially offset by a marked upswing in the East Asia and Pacific region.

In terms of which cities had the most sprawl? Sprawl has “increased in many of the “usual suspects” such as São Paulo, Los Angeles, and Lagos”, but there are sprawling cities that receive much less attention, mostly found in southeast Asia.

And for a last bit of learnings from the researchers, they shared the diversity of ways in which street networks can grow in a connected or less connected pattern:

Gridded street networks are a primary means through which Latin American countries such as Bolivia, Argentina, and Peru have maintained their high levels of connectivity—and the move away from the grid characterizes sprawl in large Brazilian cities such as São Paulo. However, Japan has maintained low levels of sprawl via a more organic, irregular grid pattern. Northern European countries such as Germany, Denmark, and the United Kingdom, meanwhile, have maintained moderate levels of street connectivity through dedicated pedestrian and bicycle pathways, meaning that the network for nonmotorized travel is far more connected than that for cars and trucks.

This all said, connectedness alone doesn’t define how easy our cities are to get around and how accessible they are to everyone in a given population. New York and Copenhagen, both highly connected cities, couldn’t be more different! I’ll have to bring an expert in the world of urban planning and data science into a future article to truly dive into the intricacies of what make these special, highly connected cities hum!

For more on where data science meets urban planning

There’s an entire online course created by one of the researchers on this work, Adam Millard-Ball, who is currently a professor of urban planning at none other than my alma mater, UCLA! Go Bruins!

This course covers a variety of areas at the crossroads of urban planning and data science, from building ML models using scraped apartment data on Craigslist, to performing spatial relations analysis using geopandas. The swath of resources this course provides provides a substantial (and free!) foundation to more learnings on data science and urban planning. Open-sourcing this education and much of the data Millard-Ball has used in his work hopefully accelerates the findings of ways to improve our world - from city streets to beyond. Happy learning!

All block quotes are from the research team’s article here.

Thanks for tuning in! See you back here in two weeks. If you’ve been enjoying Day to Data, let a friend know! If you have any thoughts on today’s article, you can find me on Twitter/X at @gabyllorenzi.

Subscribe now

Last week, I devoured the latest episode of Acquired (which is my favorite podcast these days) on Renaissance Technologies. Ever heard of them? Well, if you haven’t, it’s not surprising, but you’re going to be intrigued. Per their website, Renaissance Technologies (also known as RenTech) is “an investment management firm that employs mathematical and statistical methods in the design and execution of its investment programs”. RenTech is known best for returning more than 66% annualized before fees over 1988-2018 in their Medallion fund, which is closed to outside investors. In 2020, per Acquired, they had gross returns of 149%.

Today, we’re talking about the academic brain power at RenTech and the algorithms that can understand markets better than humans.

This article is for informational purposes only. This is not investment advice.

A finance team that’s never worked in finance

If RenTech was looking to hire a new employee, they would not go looking to the big banks in Midtown Manhattan. RenTech has made a name for themselves by being an academic-focused financial institution. Employees have PhD’s in mathematics and computer science, and are ready to work on massive algorithms versus complex Excel models. This focus on academics comes from the background of founder Jim Simons. After getting a math degree from MIT and a PhD at Berkeley, he led the math department at State University of New York at Stony Brook - an important detail when wondering why RenTech is based in the seemingly obscure town of East Setauket, NY.

Simons also spent the 1960s at the NSA as a codebreaker during the Vietnam War. During this time at the NSA, the group Simons was a part of had a simple mandate - spend 50% of your time on codebreaking and you are free to use the other 50% of your time on academics, publishing research, or other initiatives. Per Acquired, Simons, alongside some other brilliant colleagues he had recruited, used his remaining time to apply technical learnings from codebreaking to a new medium - trading stocks.

Subscribe now

Using algorithms to understand the hidden patterns of stock markets

Simons and other researchers were using the Baum-Welch algorithm to determine the parameters of Hidden Markov Models (HMMs) which could be applied to historical financial data. HMMs are a statistical model that attempts to describe a sequence of events (or states) which depend on underlying sequences of “hidden” events. Essentially, HMMs are trying to make sense of patterns that are not understandable to the eye - much like what happens in our financial markets every day.

Sounds like magic! In finance, HMMs are used to capture “hidden” signals or patterns in financial data. This work eventually inspired the starting of his fund, Monemetrics, which was later renamed to Renaissance Technologies. RenTech focuses on building models like HMMs that remove the human elements of making financial decisions. Simons once told the WSJ that “I don’t want to have to worry about the market every minute. I want models that will make money while I sleep.”

Want to nerd out? This lecture from the University of Luxembourg was a good read to get up to speed on HMMs and stock trading.

A culture where research drives innovation

This culture of research has been a pinnacle of the firm, and shows no signs of slowing down. In 2014, a patent was quietly filed by Robert Mercer and Peter Brown, once co-CEOs of RenTech, for a system and method for executing synchronized traders in multiple exchanges. The abstract reads:

A financial trading system that includes a trading server, multiple financial exchanges, and multiple servers with each server associated with and co-located at an exchange and comprising a high accuracy clock. The trading server divides a large transaction order into multiple smaller transaction orders and combines each smaller transaction order with a transaction execution time. The trading server sends a financial trade instruction based on each combined smaller transaction order and transaction execution time to each co-located server. When the high accuracy clock on each server reaches the transaction execution time, all the servers submit their smaller transaction orders to the respective financial exchanges substantially simultaneously.

I haven’t read a patent as in-depth in a while as I have read this one. Prior technology required the use of clunky fiber optic cables and other complex circuitry to communicate and control the execution of trades. In order for trades to execute across several exchanges at the same time, the cables would try to delay the data moving to faster exchanges so that all trades executed at the same time regardless of speed to transfer information. The system described in the patent is a computer-based method that takes a large transaction order, divides it into a smaller set of transactions, and then determines the times at which all trades must be executed at separate institutions. While much of what RenTech does appears to be different than traditional high frequency trading (HFT) funds, this patent shows they are continuing to innovate the ways that they can execute trades faster and more efficiently than their counterparts.

Their focus on innovation shows in their performance. As mentioned, their flagship fund - the Medallion Fund - returned more than 66% annualized before fees over 1988-2018, and is closed to outside investors. Their model has performed best in the most volatile of times for the rest of the market (see late 1990s and 2008 below).

As more and more trading strategies get commoditized, there will continue to be innovation that moves further down the stack - meaning closer to the data, the circuits, and the hardware that are enabling our financial infrastructure.

Beyond the fund

Jim Simons is continuing to innovate even after stepping down from the CEO role at RenTech in 2010. In 2012, he established the Simons Institute for the Theory of Computing at his alma mater UC Berkeley. In 2016, he established the Flatiron Institute in Manhattan to house over 60 researchers working towards innovations in computational science. In a rare interview with the New Yorker, Simons spoke about the Flatiron Institute, where he seemingly hopes to replicate the academic culture of RenTech to be applied to areas outside of just finance.

Lastly, for your (and my own!) future reading, The Man Who Solved the Market by Gregory Zuckerman of the WSJ, published in 2019 alongside this article, details the secretive and fascinating life of Simons — or apparently as close as one has managed to get. The firm continues to be an elusive place, far out of the city, where few will ever know exactly what’s happening behind closed doors. One lingering thought after all this is…. those must be some complex taxes.

Thanks for tuning in! See you back here in two weeks. If you’ve been enjoying Day to Data, let a friend know! You can find me on Twitter/X at @gabyllorenzi.

Share

Subscribe now

Today’s Day to Data is about Kona, the AI-powered leadership coach for remote managers, which I learned about through an awesome conversation that I had with the CEO, Sid Pandiya.

While it doesn’t quite feel like it, remote work has been an operating model long before the pandemic. Some well known have been remote-first for decades, like Github since its founding in 2011. But it wasn’t until March 2020 that much of the world felt the weight of what it meant to be on a remote team and, for managers, the reality of leading a disparate workforce working across time zones and vastly different living situations. Managers were pressed to learn a new method of how to be effective. This is where Kona comes in.

This conversation taught me a lot about shipping early product, being a technical founder, and being creative to get closer to product-market fit. In a world where remote work is widely here to stay, there remains to be tremendous opportunity for tools like Kona to level up managers to be their best. Let’s dive in!

The power of communication skills

Since his early teens, Sid has loved solving problems that he had personal connections to. He was involved in the debate community growing up, which inspired his first project - a platform for kids to debate online. The idea of helping people communicate more effectively using technology and debate skills resonated - there was magic in understanding a problem and then solving it with technology. He pivoted to build a tool that taught communication skills in the workplace through the fundamentals of debate. These early builds and his interest in communication would play a role in guiding the products that were yet to come.

How the world showed up to work changed dramatically

“Each failure led me to my next company”, said Sid. And in October 2019, his past learnings led him to Kona. The founding team focused explicitly on managers of remote teams - they hypothesized that there were a lot of pain points in managing these disparate workforces.

“We interviewed 100 remote managers before writing a single line of code. We were college students just DM’ing people on LinkedIn. We’d ask people what they found difficult about managing a remote team. And then they started complaining - and it became obvious that managing a remote team was pretty hard.” - Sid

Then, in March 2020, something happened that no one could have ever expected. A majority of teams previously working in offices, stores, and other in-person means shifted to an entirely remote operating model. This global shift was the catalyst to the team going full time on Kona.

So what is Kona? Kona is an AI-powered leadership coach for remote managers. Kona understands your company, knows remote manager best practices, connects to all your internal tools, knows your strengths and weaknesses, and creates a personalized approach to being a better manager. Kona gathers contextual data from the manager and direct reports, as well as the context of the company and relationships, to be the executive coach for remote managers everywhere.

Building an MVP that excites your early customers

Sid built much of the initial Kona product while in school at UCLA. Now, a few years into being CEO, he is “very closely involved in all the architectural decisions we make. I think at the early stages so much of what we do is technical exploration - see what’s the limit of the technology to see what’s possible. So I’m spending a lot of time testing, playing around, and seeing what I can hack together”. And much of the building of an initial MVP is about this approach testing, learning, and hacking together solutions.

Kona is built on top of Slack. This was a huge unlock to move fast in the early days. Being built on top of Slack made it easy to ship real prototypes quickly. Slack is SOC2 complaint, HIPAA complaint, and has a slew of its own security best practices. Slack enables certain scopes of data access for external bots, which made it easy for the Kona team to start providing for a customer at a smaller scale, then grow with time as they gained trust and use of their tool. Sid credits the Minimum Viable Testing (MVT) approach to playing a huge role in how they shipped early products — Gagan Biyani’s article on the topic can be found here.

“In an MVP, you try to simulate the entire car. In an MVT, you are just testing whether the drivetrain is more powerful with an electric engine or a gas one.” - Gagan Bavani

In order to begin testing demand, without writing any code, Kona started the Remote Manager Accelerator. Over 30 days, they planned to build personalized coaching plans for participants to build one specific management skill that’s most important to you. They were overwhelmed with signups, opening up more spots, and quickly realized the demand was there. Their second learning was that it all came back to feedback. They were working alongside managers, seeing their struggles, particularly around giving back feedback within the context of their organization. In order to understand context for providing feedback and coaching managers, they did it manually. They reviewed managers’ calendars, they discussed their strengths and weaknesses, and talked through the hardest parts of their job. They took this manual context-building step and began the roadmap to productizing it.

Do things that don’t scale

One of the most famous quotes regarding building a company from Paul Graham, co-founder of famed startup accelerator Y Combinator, is to “do things that don’t scale”. An example Graham spoke to in his original essay on the topic was the “Collison installation”, referring to the Collison brothers’ approach to recruiting early users to Stripe.

More diffident founders ask "Will you try our beta?" and if the answer is yes, they say "Great, we'll send you a link." But the Collison brothers weren't going to wait. When anyone agreed to try Stripe they'd say "Right then, give me your laptop" and set them up on the spot. - Paul Graham

It’s an entirely unscalable effort for founders to set up every interested customer on the spot, but makes for a personalized customer set up, and builds out the initial user pool in a tangible way.

Kona’s approach to understanding context from their customers through accelerators and ride-alongs was manual and unscalable, but led them down the path of what they needed to build to deliver value to customers. Doing things that don’t scale can be incredibly insightful for the early days of building a business.

The future of Kona

“Kona is at a point where it’s giving out contextual advice that is really good”, says Sid, meaning that the tool is able to understand the intricacies of dynamics between team members, workplace structures, and context on the employee. Thanks to partners like Evolution Coaching, who has coached execs at Slack, Netflix, Microsoft, Google and others, Kim Scott, author of Radical Candor, and Amy Edmonson, who wrote the Fearless Organization, Kona has built a robust platform that combines research and psychology with exciting developments in data and artificial intelligence to empower remote leaders. “We want every manager to have the tools they need to succeed. Kona should be like a little birdie on your shoulder - a proactive guide that can analyze large amounts of data to turn it into targeted actions, so managers can stick to the important human elements of people management”.

You can find Sid and Yen, co-founders of Kona, on LinkedIn, where they regularly share resources and tips for remote managers.

I took my first algorithms class during my junior year and learned of an algorithm that has since become one of my favorite pieces of knowledge acquired during college. This algorithm has a beautiful story, including a Nobel Prize, medical residents, and the dating app, Hinge. This algorithm is known by the name of its two creators, Gale-Shapley, and I just love it.

Today, we’ll explain the algorithm and walk through how it applies to a feature on one of the world’s most popular dating apps that you may have a love hate relationship with - Hinge’s Most Compatible.

Some background to get started

David Gale and Lloyd Shapley were mathematicians and economists. They both found success tackling problems in game theory, logic, and eventually received the Nobel Prize in Economic Sciences in 2012 for their work, citing the success of a 1962 paper titled “College Admissions and the Stability of Marriage”.

Their paper proposed a problem you’ve probably thought about if you’ve applied to college: “A college is considering a set of n applicants of which is can admit a quote of only q. Having evaluated their qualifications, the admissions office must decide which ones to admit.” There is a desire from both applicant and college that assigning applicants to the right college be satisfactory for both student and college - the student goes where they want and the college gets the students they’d like. This algorithm hopes to tackle finding those most desired matches for all participants.

Subscribe now

How does the algorithm work?

While they ultimately wanted to provide a solution to the college admissions problem, having far more applicants than colleges can make it harder to tackle than a problem in which there is equal numbers in each sets of the problem. We’ll simplify things by assuming these are equal.

• We have n colleges and n applicants.

• Each set ranks the members of the other set by their preference - ie. all colleges rank the applicants from most desired to least desired and the applicants rank the colleges as well.

• The goal is finding a stable set of matches, with stable as being not unstable. A match is unstable if there are two applicants X and Y who are assigned to colleges A and B, respectively, although Y prefers A to B and A prefers Y to X.

To explain this in real life, imagine there are two students, Gaby and Sally. They are deciding where they’d like to go to college. They both rank the only two colleges that exist, USC and UCLA, by their preference. The two colleges rank Gaby and Sally.

Let’s look at these lists.

Take a look at how Sally and Gaby have the same lists. While Sally preferred UCLA over USC, a match of Sally at UCLA would have been unstable, since UCLA preferred Gaby over Sally and Gaby preferred UCLA over USC, which is a stable match.

Gale and Shapley’s paper proves that there always exists a stable set of matches (marriages in the case of the stable marriage problem). These matches are revealed through iterations of the Gale Shapley algorithm. For a technical walkthrough of the algorithm, check out the matching python package docs on the problem.

Taking from Gale and Shapley’s paper, which defined the algorithm to begin with, let’s explain how it technically works in the scope of marriage, despite the antiquated view.

1. Each man proposes to his most preferred woman.

2. Each girl who receives more than one proposal rejects all but her most preferred. She does not accept this most preferred, but says “maybe” for now.

3. The second round begins. The rejected men each propose to their second most preferred woman. The women receiving a proposal choose their favorite man from the new proposals she has received and her “maybe” match from the first round, if she had one.

4. This proceeds again, with more proposals and the women rejecting all but the best proposal they have received so far.

5. The proposals continue until the last woman gets proposed to and all the women must accept whoever she has most recently preferred the most.

It’s import to make note that the execution of the stable marriage problem in this way is optimal for the men. One could conduct the same experiment with the women proposing, and if the match from that experiment was the same as the match from the male proposing experiment, that match would be considered optimal.

Other versions of the algorithm include:

• The stable roommates problem where there is only one set of individuals, not two

• The hospital-residents problem which uses a variation of the algorithm to match medical school graduates with residency programs, and a further variation for couples who are allowed to be considered when preferring to stay together for residency

Where is the algorithm used today?

You’re scrolling through Hinge, liking, messaging, and denying potential matches on your quest to find love. You’re getting closer and closer to that special someone. All those interactions are generating data for Hinge to establish your preferences. While you never explicitly draft up a list ranking every single person in your dating pool, your likes, messages, We’ve Mets, and more all indicate the type of individual you are looking for.

Hinge’s Most Compatible feature ”uses machine learning to figure out each user’s taste” according to CEO Justin McLeod. The app is aggregating data on your actions to develop a personalized view of what you’re looking for while also aggregating views for those in your dating pool. The models are then applying a variation of the stable marriage problem to promote those who you are Most Compatible with. Both users in a Most Compatible match receive the notification on the same day and it expires after 24 hours. The process continues to repeat and hopefully be more and more successful with continued use of the app. Users have said that the app’s recommendations improve the more time you spend on the app, and that is due to the app’s ability to better understand your preferences with an increased number of likes, dislikes, messages and more.

Data science and a variation of the Gale-Shapley algorithm is powering the recommendations you receive every day on Hinge. It’s iterating every day, with more training data and improved understanding of users, to help you get one step closer to finding your Most Compatible.

A fun tidbit to end today’s post: Did you know Hinge CEO Justin McLeod’s own love story has made it to TV? What started off as a conversation with a New York Times journalist evolved into Justin reaching back out to “the one that got away” after being inspired by his conversation. The interview and love saga was turned into an episode of Amazon Prime’s Modern Love which is inspired by the NYT article series.

So it turns out data science is helping out millions in their journey to find love, hoping to better understand our preferences, ideal partners, and eventually find the perfect match. Thanks for reading this week’s post.

Subscribe now

The IDE is a fundamental platform in a programmer’s toolbox. An IDE - integrated development environment - is a software tool that enables programmers to write and test code efficiently, to build scalable, collaborative projects. Prior to an IDE, programmers wrote a majority of code in text editors. I was one of those programmers for a shamefully long time.

Sublime Text was my tool of choice for a while when I was learning how to code. Using a text editor vs. an IDE is somewhat comparable to writing without a formatter or Grammarly. It’s a pretty good way to get started as a developer. To use Sublime or other text editors, I’d type up my code, save the file, then open the Terminal, and run the code to see what happens. There was no way to see my errors until the Terminal spit something out like “Line 2: no variable named ‘test’”. I’d find my way back to Sublime, make an edit, save the file, then run it again in the Terminal. It took a while to get into a flow.

I eventually transitioned from a text editor to an IDE - like Microsoft’s VSCode, Jetbrains’ Pycharm (personal favorite), and Apple’s Xcode. Using an IDE was a major productivity unlock. I could write my code, immediately get feedback on how to finish each line, and quickly debug complex problems.

There’s been the looming question of what AI will most tangibly change about our personal, professional, and social lives in the future. Some of the most immediate changes are happening on the developer level, yielding another step-level change in productivity, far beyond the IDE.

There’s a practice in software development called “pair programming” which typically happens when two programmers are collaborating on the same code at the same time - like two writers in a Google Doc but usually more literally two devs sitting in front of one machine - solving each others mistakes, quickly fixing bugs, and speeding up their ability to execute. OpenAI President and co-founder Greg Brockman is a big fan.

However, pair programming is hard to scale, can be challenging for remote teams, and is limited to the collective knowledge of the two developers working on a task.

This changed with the release of ChatGPT and Github CoPilot. All of the sudden, every engineer had a friend sitting next to them, with knowledge of the entire internet speeding up their ability to execute. Per McKinsey, it’s estimated that developers can generate code ~40% faster with generative AI. That’s compressing a 40 hour engineer work week into 24 hours. Currently there’s limited efficiency boosts for high-complexity tasks, but with efficiency elsewhere, developers can refocus on the complex tasks, versus spending time writing documentation or boilerplate code.

I believe complex coding tasks will continue to be optimized by generative AI through building models fine-tuned to specific types of development. An engineer will go from writing code for a single part of a program, say front-end web development, to using a model that’s tuned for each part of the stack or a certain program - a backend model, a Python model, etc.

Quick step back for some folks to level set: what does fine tuning mean? If you had to get heart surgery, you’d probably want a cardiologist to do the surgery. A cardiologist went through extra training and education beyond general surgery to learn about the intricacies of the heart. A cardiologist was “fine-tuned” to perform best on hearts, but in doing so, might not perform as well at math or history as someone skilled across general, all-purpose knowledge.

I believe that the IDE of the future will be a platform where engineers come to use various code generation, documentation and testing models that have been fine-tuned to certain parts of the programming process. The IDE of the future will be agent-based, giving a programmer the power to deploy specialized agents to build a platform together. By taking in information like a project spec, dependencies and security requirements, the IDE of the future will have one agent build the frontend using a model specific for frontend development, another securing code as its written, and others building database integrations and documenting the code.

The IDE of the future will not be a place where code is written directly by a software developer. It will be a playground where agents trained to write code will be managed. Specialized agents will write code, that can be reviewed by an engineer, or another agent. The files will be automatically committed to Github, showing progress the agents are making along the way. The role of a software developer will move to a higher level, more architecture focused role. The developer will have to shift from thinking about how to write code for a siloed portion of a project, to how to bring together portions of code into a production-ready project. Developers will pair program alongside a multitude of agents, no longer limited to the knowledge of their fellow engineer.

There are already AI-first code editors, like Cursor, that are gaining a lot of attention from developers. However, I’m eager to see an approach to an agent-first IDE. I’m eager to see the next generation of tools that support the software developers stack, as their role shifts and prioritizes higher level thinking and strategy. I’m eager to see what becomes possible as it becomes easier for anyone to build software. There is so much progress and innovation yet to come as the underlying tools used by engineers change.

If you’re building here, definitely reach out - gaby@primary.vc!

New to the newsletter? Subscribe to get a dose of musings by a data scientist turned venture investor, breaking down technical topics and themes in data science, artificial intelligence, and more. New post every other Sunday.

Subscribe now

Today, we’re talking about one of the most ubiquitous pieces of technology that laid the foundation of the Internet revolution. As of 2021, the internet economy grew seven times faster than the total U.S. economy. The digital economy makes up ~10% of the GDP. The Internet is amazing, but getting access wouldn’t be possible without the technology we’re talking about today - Wi-Fi.

If you’ve been a reader for a while, you may have picked up on the love I have for the stories that take us back to the earliest days of a certain technology — this one is perhaps my favorite of them all. Ffirst, we’ll get a little technical about how Wi-Fi works, then we’ll go back to how it came to be.

Mixed signals might be a good thing!

Wi-Fi uses radio frequencies, instead of wires like Ethernet, to send information between a router and other devices, enabling communication with the Internet. Yes, this means that Wi-Fi and the Internet are different things! Your device, like the phone you’re probably reading this on, connects to Wi-Fi by sending out a radio signal in search of a router. The router than is able to process incoming and outgoing data signals. The router is where signals are received and also sent to the right recipient.

Wi-Fi relies on frequency-hopping spread spectrum (FHSS) to send signals. FHSS is a method of transmitting radio signals by rapidly switching the signal between many frequency channels, using a sequence that appears random, but is known by the sender and receiver.

Though seemingly more complex, frequency-hopping is advantageous over fixed-frequency signal transmission (where the signal doesn’t hop frequencies) for several reasons. With frequency hopping, you reduce the likelihood of interference (noise, other data, stuff you don’t want) from other signals, which is common in fixed frequency.

If you were to send a packet of data along a single frequency, it is easy for a third party to intercept that data as it travels from sender to receiver. With frequency hopping, it’s much harder to intercept data as it hops between frequencies because only the sender and receiver know the expected sequence of frequencies to hop between. A major benefit of frequency hopping is security. Now, let’s go back to 1930s Austria, where this security was top of mind.

The woman behind it all

In 1914, Hedy Lamarr, born Hedwig Kiesler, was born in Vienna, Austria into a Jewish family. From a young age, she was claimed to have a brilliant mind. Lamarr was discovered at age 16 by a director, which started off a life of acting in Europe. She quickly found an adoring fan in Fritz (note: some sources use Fredrik) Mandl, who she married in 1933. Mandl, however, was an Austrian munitions dealer, selling to the Nazis in the mid-to-late 30s. Mandl would bring Hedy to dinners with the highest ranks of Nazi officers, where Hedy would overhear of the horrors ahead, and of the plans to use technology, like radio signals, to communicate military plans across Europe. These conversations on the future of wartime weaponry would stick in Hedy’s mind as she fled her marriage and landed in London in 1937.

Her time in London launched her to success, when she was fast-tracked to fame after meeting Louis B. Mayer, the co-founder of MGM Studios. She made her way to Hollywood and was called the “most beautiful woman in the world”. It was during her time in Hollywood that her innovative spirit was being nurtured again, especially after meeting George Antheil, a well known composer. They spoke of their worries about the impending war — Antheil says, “Hedy said that she did not feel very comfortable, sitting there in Hollywood and making lots of money when things were in such a state.” So, they got to work.

Hedy’s mind took her back to the dinners, hearing talks of using radio signals, which were vulnerable to attack, to target wartime enemies. With Antheil’s expertise in sound and frequencies, and Hedy’s creativity and ingenuity, the two built a new communication system. In this system, the sender and receiver of radio waves hopped to new frequencies at the same time and pattern, helping prevent interference. This technology guided torpedoes to their targets, safe from radio jamming, which is the blocking of radio communications for interference. The duo was awarded a U.S. Patent in 1942 for their frequency-hopping invention. She didn’t make a penny for this work during her lifetime. In 2014, Hedy Lamarr was inducted into the National Inventors Hall of Fame for the invention.

P.S: I first heard this story on the Acquired podcast about the history of Qualcomm. It is a must listen for Wi-Fi and network nerds in the making!!

Putting it all together

The same technology that Hedy patented back in 1942 inspired the backbone of much of the Wi-Fi technology we use today. Frequency hopping has shifted to more advanced signal transmission methods. Modern day Wi-Fi, which, by the way, is short for "Wireless Fidelity," is the wireless communication technology that allows devices to exchange data, without the use of wires, within a local area network (LAN or WLAN for wireless). The network relies on the IEEE 802.11 standard, which ensures uniform and standardized data transfer that a wireless network can handle.

Many more teams across the globe, armed with Hedy’s foundational technology, should be credited for accelerating the building and adoption of Wi-Fi:

• The University of Hawaii, who created the first wireless data packet transfer in 1971, enabling inter-island communication through ALOHAnet.

• Victor Heyes, nicknamed “Father of Wi-Fi” and the IEEE 802.11 team who set networking standards and enabled early access to the Internet

• The CSIRO Division of Radiophysics, who created chips that could easily transmit wireless connections, constructing the first wireless local area network (WLAN)

Where would we be without Wi-Fi

Wi-Fi has powered a macroeconomic shift of epic proportions. Enabling public access to information such as the Internet has had massive downstream effects. Wi-Fi is something I most definitely take for granted. Thank you to the Wi-Fi network that enabled me to access the Internet to learn all this history - how cool!? I’ll save the story of Bluetooth for another day, but talk about something else that has changed how we work! See you all next time. Have a great week.

Additional references

• Hedy Lamarr’s Forgotten, Frustrated Career as a Wartime Inventor — The New Yorker

• Women’s History - article

• Hedy Lamarr and Frequency Hopping Technology

Did a friend send you this newsletter? Subscribe to get a dose of musings by a data scientist turned venture investor, breaking down technical topics and themes in data science, artificial intelligence, and more. New post every other Sunday.

Subscribe now

Behind all the cool technical products in our every day are the people building them. Engineers, data scientists, designers, and founders leading the charge on shipping great products to the masses. It would be remiss to not share their journeys, so for this week on Day to Data, we’re talking to the founder of an exciting new consumer product about building a company, his tech stack of choice, and the people who got him to where he is today.

When I was a sophomore at UCLA, debating whether to go all in on the pre-med path or do a 180 to follow my growing interest in computer programming, I was looking for an outlet to learn more about what a future in engineering and tech could look like. I met Pierson and learned about the startup and VC club he was leading on campus, and figured I’d try it out. As college kids, we mused on futures seemingly decades away as founders or investors, working in industries we admired from afar. Having this conversation was quite the full circle moment.

A bit on text to speech

We’ve been trying to get our computers to talk to us for decades. At the 1939 New York World’s Fair, the VODER was unveiled. It was a piano-like system developed by Homer Dudley and his team at Bell Labs that electronically synthesized speech by using a keyboard and foot pedals to create a human-like voice. We’ve come a long way since then.

Now, text to speech tech is somewhat table stakes. In 2007, Siri started as a research project at Stanford to be a virtual personal assistant. Siri was acquired by Apple and launched to the masses in 2011. “Hey Siri!” became the phrase that opened the eyes of the everyday consumer who realized that our voice could now let us send emails, search the web, and text our friends.

The real differentiator in the past five years comes from two things - latency (speed) and how realistic the voice sounds (which is definitely impacted by improvements to latency). And maybe I’ll add one more - ease of access. Text to speech (TTS) is just an API away! Any language, any gender, any phrase, with a few lines of code.

To dig into the tech:

• From Hawking to Siri: the Evolution of Speech Synthesis by Tife Sanusi

• Demystifying the Technical Structure of Text-to-Speech Models by Abdulkader Helwan

• NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality by Microsoft Research

• The product at ElevenLabs

• Using OpenAI’s Text to Speech (TTS) model

The journey to becoming a founder

As a builder at heart, from Legos to software, Pierson studied computer science at UCLA. During undergrad, Pierson led the aforementioned VC club, Bruin Ventures, through years of learning about starting and growing incredible companies. These interests led him to Amazon, where he spent 3 years as a software engineer on the “out of box” experience team for Alexa. “Every person that purchased an Alexa was interacting with something we built”, Pierson says. The global scale of an Alexa customer’s experience exposed him to skills across the stack, from hardware, to product marketing.

Deciding to build Jellypod

“It was on my walks to the Amazon office, where I came up with the idea”. He’d listen to podcasts or try to catch up on reading emails and newsletters he subscribed to. It seemed like there should have been an option for him to combine the two — turning his emails into a podcast, getting him the personalized information he cared about in a manner that worked for his day to day. He scoured the web for a solution, and couldn’t find anything to meet his needs, so he began to think about building it himself. He validated his ideas by talking to family, friends and colleagues. He bought a domain and placed an email box to get updates on the website. “Pre-product, people from around the world signed up to learn more, and that was validation enough”.

Behind the scenes: the tech behind Jellypod

When you sign up for Jellypod, the service creates a unique email address for you to sign up for your favorite digital publications with. Using AWS Lambda schedulers, Jellypod checks your unique inbox for newsletters at a time of your choosing everyday. It ingests the emails found in your inbox and begins to transform them into your daily podcast.

Before bringing in LLMs, the email text needs to be pre-processed. This means cleaning up the text — removing promotional text or boiler-plate language — to be ready for a large language model. Advancements in generative AI in text and speech were a clear catalyst as to “why now” with Jellypod.

“LLMs are very good at summarization,” Pierson says. Using GPT-4, Jellypod is able to transform numerous emails into summarized text. While Jellypod currently uses GPT-4, Pierson’s currently integrating open source models into the platform to perform the summarization, using OpenPipe to host and fine-tune these models, bringing costs down ~80%.

From there, that text is transformed into a podcast, using a text-to-speech model from OpenAI. “We tried a variety of text-to-speech models, like ElevenLabs, but the realistic sound of the resulting voice and price point of OpenAI’s model was the right fit for Jellypod”, says Pierson. The entire product is build on a serverless infrastructure, that responds to the spikes in usage (typically in the morning) and is low-cost when the podcasts are done being generated for the day. Now, you’ve got a podcast, personalized to you, ready to listen to directly in the app.

Jellypod includes a variety of additional features, like generating multiple daily podcasts based on category, offline listening, and generate-on-demand podcasts. It’s truly incredible the speed at which Pierson has been able to ship features and updates as a solo technical founder — much thanks to his vast technical skills and access to incredible AI tools.

As for someone who helped shape where Pierson’s career has landed him today? Pierson thanks Paul Eggert, a brilliant professor known for challenging lower level computer science classes at UCLA. Likely unbeknownst to you, we’ve all got Eggert to thank. He’s a coordinator and editor of the Time Zone Database at the Internet Assigned Numbers Authority (IANA). The database keeps phones, computers, tablets, and applications on the right time zone, as well as enables timestamps for documents and photos signaling creation and modification.

In regards to Pierson’s learnings from Eggert — “His tests were hard for everyone. When imposter syndrome dissolved, I realized you can do anything you set your mind to”.

The future of Jellypod & generative media

“Hyper-personalized media is where the world is moving. Everything you see is going to be created for you, and you alone,” Pierson says. Jellypod aims to shape the future of generative, personalized media. It’s a future that is being shaped as we speak. The intersection of media and generative content is up for debate — it’s likely that the New York Times lawsuit against OpenAI will set a new precedent for what these now intertwined industries will look like. And Pierson’s excited to be a part of it - shaping the way we interact with ideas, opinions, and perspectives.

If you’re interested in trying out Jellypod, it’s available in the Apple App Store. You can connect with Pierson here. See you next week!

Did a friend send you this newsletter? Be sure to subscribe to get a dose of musings by a data scientist turned venture investor, breaking down technical topics and themes in data science, artificial intelligence, and more. New post every other Sunday.

Subscribe now

Welcome to 2024! Hope you got to ring in the new year in style and take some time to rest. For the first post of the year, I’m launching a new format I’m really excited about, in hopes of providing value on the past, present, and future of technical topics in an easy-to-digest format. We’ll talk about the history, where we are today, and look to the future for a given topic. We’re starting 2024 with one of the most fundamental pieces of consumer technology: the browser.

It’s our window to the Internet. It’s where we spend most of our time. It’s where we explore new ideas and center our digital life. It has to evolve as the internet and its users do. Let’s break down the story of the browser, starting at the beginning.

Where it started: the World Wide Web

To talk about the start of the browser, we have to touch on the start of the internet. UCLA (Go Bruins!) sent a message to the Stanford Research Institute in 1969, marking the beginning of ARPANET and becoming the first node of what would become the Internet. Universities and governments were forming the early Internet, but it was entirely inaccessible by the public. Then came the WorldWideWeb. Tim Berners-Lee built a graphical interface for the Internet for the NeXT computer (the product Steve Job’s built during his leave from Apple) that gave keys to the Internet to a whole new cohort of users.

If you want to surf a recreation of the OG WorldWideWeb, use the re-creation here!

In 1993, the first popular web browser Mosaic was built by a computer scientist at University of Illinois Urbana-Champaign named Marc Andreessen, who is now perhaps the most famous venture capital investor in the world leading a16z. Andreessen went on to found Netscape, a commercial web browser built for Windows and Mac, and enable dynamic Web pages, RSS feeds, and other online innovations. This was the beginning of the people’s Web.

To see 14 years of Netscape browsers, take a look at the history here!

Web pages were once static places. But now that everyday users were gaining access, not just researchers and academics, people wanted more from the internet - color, interaction, and a dynamic nature. This was a catalyst for the launch of JavaScript by Netscape, Cascading Style Sheets (CSS) by Microsoft, Ruby on Rails, and PHP — all tools that enabled a more dynamic, engaging web through interactive sites and revolutionary design.

With the development of these engaging new tools and the integration with their operating system, just 5 years after the founding of Netscape, Microsoft had 99% of browser market share with its product Internet Explorer. There was a brief moment of “the browser wars” with some comedic battles between Microsoft’s Internet Explorer team and Netscape, which included the IE team sending a massive “e” to the Netscape headquarters and putting it front and center on their lawn.

Netscape went on to open-source its project and launch Moxilla Firefox in 2002 after seeing Microsoft endure a legal battle over a monopoly on the browser market. Firefox began to chip away at Internet Explorer’s domain, rising to its peak of ~30% market share in 2010.

Where we’re at: It is Chrome’s world

In 2008, Google launched an open source browser product called Chrome. The browser, open-sourced through Chromium, was launched to be streamlined and simple. This launch, though almost 16 years ago, is the most recent massive disruption to the evolution of the browser.

In May 2012, Chrome dethroned Internet Explorer as the most widely used web browser. A new era was upon us - it was a world in Chrome. Their market share has grown impressively since launch, eating away at Internet Explorer and Firefox primarily, with over 60% of web browser share today.

Today, the browser and the idea of search are fully intertwined - one cannot be imagined without the other. To open Chrome is to search the internet. It’s a portal. As search evolves, browsers will evolve too.

Where we’re going: the searchable, scalable mind

The future of the internet as a whole is evolving. And the browser, which hasn’t seen a massive shift since the launch of Chrome, has to keep up. The browser is an exciting battleground in my opinion - you’re the portal to the Internet and everyone’s first eyes in the start to their web journey. You have to manage rapidly advancing web applications, always prioritizing speed, latency, and memory management. The browser now also represents your online personality, being a profile that represents your searches, where you spend your time, and the ideas you save. And with the meteoric rise of generative AI, will all browsers just be a chatbot interface, for which your bookmarks, search history, and computer contents can effortlessly be engaged with? There’s lots more to come.

Companies to watch

The Browser Company is leading the charge as a well-funded startup changing the way we interact with the internet. Their product Arc has rave reviews of being intuitive, beautifully designed, and effortless at organizing your online life. Led by Josh Miller, the company has amassed over 100K followers on Twitter, raised $75M+ from Slack Fund, BoxGroup, and notable angels like Figma founder Dylan Field and LinkedIn founder Reid Hoffman.

Bookmarks have long merited their own platform, outside of just your web browser. With an emphasis on surfacing your creative inspiration alongside everyday life, mymind aims to bring your bookmarks, notes, inspiration and ideas into one central hub. The tool is made for “designers, writers, researchers, developers & visual minds” of all kind. It uses artificial intelligence to automatically categorize and organize your thoughts, surfacing what’s relevant across a multitude of dimensions.

To learn more

• What’s really the future of the browser?

• One startup's quest to take on Chrome and reinvent the web browser

• The History of Web Browsers

Thanks for tuning in! I’d love to hear any feedback or thoughts on this format, as well as any ideas for topics you’d like for me to cover in the future. And if you’ve been enjoying Day to Data, let a friend know! Have a great week.

Share Day to Data

You can find me on Twitter/X at @gabyllorenzi.

Day to Data is closing out its first year! I remember nervously posting my first article in January and having only one goal - just keep writing. And looking back, I’m proud to say I’ve done just that. In a world inundated with information, it can be hard to feel like you’re not just contributing to the noise. I’m still figuring out what resonates best with readers and how I can add value, but I’m excited about the challenges along the way.

A few numbers on the year —

• Day to Data: 48 posts, with a 60%+ open rate!

• Spotify: 77k minutes of music listened to

• Strava: 1700+ miles run and a 16 min marathon PR

• And 1 new job!

We’ll celebrate the end of the year with some of my thoughts on what’s happened in the past year and what’s to come in 2024. And as for Day to Data - I’ll be pivoting to a bi-weekly release of articles. I’ll keep writing about basics of data science and exciting use cases, while exploring additional avenues of writing. It will be a journey! More fun facts guaranteed :)

Data is still the new oil.

In 2006, mathematician Clive Humby was credited with saying that data is the new oil. Like oil, data’s potential isn’t when it is in its raw form - it’s the limitless opportunities for data once it is cleaned, formatted, tidied up, and reshaped into something even more powerful. This past year, ushered in new era of what data can do. Academics, founders, professionals and lay people are seeing the power of data in their lives more today than ever before. While there is some pessimism about using data as a catalyst for acceleration, we’ve seen some incredible things be built, shared, launched and updated this year - much thanks to data.

The world is more online than ever.

Per Cloudflare, global Internet traffic grew 25%, in line with peak 2022 growth. While this number may seem like we’re spending more time online than ever, it also shows that more people are online than ever. Starlink traffic more than tripled, with 4,200 satellites powering their high speed internet access virtually anywhere. The most popular internet service of 2023 was Google.

The venture landscape is changing.

Seed investments have been ”remarkably resilient of late”, per Carta. The median deal size has been $3 million. Series A investments are on the rise, with a median round size of $9.6M in Q3 2023, the highest value since Q3 2022. Series B and C rounds saw slight upticks, but later stage (D & E) fell. IPOs are still quiet. Instacart and Klaviyo were buzzy, but ultimately calm IPOs. Many speculated they’d open the floodgates, but instead they left the IPO market looking milder than ever. Both stocks are now performing 15-20% below their initial share price. Investors will continue to look for new exit opportunities. I think private equity buyout and M&A will be on the rise in 2024.

Where I’m excited to look for investments in the new year? New software & data infrastructure to support fast and secure multimodal model training. Platforms using AI to increase efficiencies in the warehouse. And software to support new hardware being built to change the energy industry (with a personal interest in hydrogen energy!).

Consumers are spending big money.

Consumer spend on Black Friday was up 7.5% from 2022 to $9.8B, according to CNN. Expectations for consumer spend in the US overall was 0.5% for 2023. Instead, we saw a 2.4% increase in consumer expenditures in 2023, per the Bureau of Economic Analysis. USPS has shipped 11M packages this holiday season to date. That recession many folks expected never really came.

And technical innovation saw step-change growth

Taking a glance at the visual below, we saw an incredible growth in the size of large language models. GPT-4 has an estimated 1T+ parameters. Amazon’s LLM that has yet to be released, Olympus, is expected to have 2T parameters. Google’s Gemini has an unclear number of parameters, but is expected to be large and outperform GPT-4, perhaps by a lot.

AI aside, we’re seeing some really amazing technology in healthcare and energy. Take nuclear fusion for example. Just this month, there was reports that the team that created the first net-positive nuclear fusion reaction was able to repeat that reaction three more times this year. While the feat may seem small, it is a huge step for finding new energy options for our future.

We are in for an exciting 2024!

There’s a lot to look forward to in 2024. Something I’d love to see is a continuing education and transparency on the rapidly evolving technical landscape that’s happening. That could be more companies moving to open source projects, or more consumer-facing education around tech behind products. Continuing to empower the individual about their data and how it’s being used. Peeling back the curtains a bit.

Some of my more technical, higher level predictions for 2024 include…

1. Open source will consistently outperform GPT-4

2. AI avatars will still not catch on, like Meta’s (perhaps they’ll even shut this down, as predicted by Platformer)

3. Mixture of experts models will become commonplace, like Mistal’s new model

4. More models will be natively multimodal, like Google’s Gemini

5. The parameter size of the largest models will plateau, and the new victory will go to those who can build the smallest models, which will sit on our phones and tablets

6. Novel technology will be build to enable a dramatic reduction in hallucinations

I’ll stop there. One last bit for today’s article below…

My holiday reading list

For all the folks using the holidays to catch up on that growing list of bookmarks, here’s what I’ll be reading over the break. **Warning - this is all technical and nerdy!

1. The Big Short by Michael Lewis (is a link needed here…? I know I’m late to the game!)

2. Why Every Company Will be an Energy Company from Meera Clark at Redpoint

3. We Need More Women Founders on Offense by Ashley Mayer

4. Long Live High-Performance Computing by Justin Selig at Eclipse

5. Building RAG-based LLM Applications for Production by Goku Mojandas and Philipp Mortiz

6. Theory of Mind Might Have Spontaneously Emerged in Large Language Models by Michal Kosinski

And for a final read, the most viewed Day to Data post of the year! Unsurprisingly, my piece on Silicon Valley and compression algorithms came in at #1.

That’s a wrap!

I’ll be back January 6th with the first Day to Data of 2024. I’ll take some much needed rest to end out probably the busiest and most eventful year of my life. Thanks for being a part of it!