(9/8/20) pulling the work vs pushing the work
Hey all,
Hope you all enjoyed your Labor Day Weekend. We were able to hop over to the beach for a Sunday morning sizzle.
I’ve got some fun stuff coming up to share with you.
First…
Longtime amigo Mike Gelb just published his 79th episode of The Consumer VC, where he interviews investors from Collaborative Fund, Lerer Hippeau, Greylock, and many more. It’s been really awesome to see him start his show from scratch and build momentum around his interests.
I’ll be chopping it up with him in an upcoming newsletter. We think it’ll be a fun and interesting read for you all.
From October 13 to 15, Mike is running an online summit for brands, startups, and investors in the consumer packaged goods (CPG) world. Wayne Wu from VMG will be there, as will 110 (and counting) leaders from places like Mondelez, L Catterton, Piper Sandler, and Bullish.
Mike and I will dig more into all this in a few weeks, but it really is an interesting time to be in CPG and direct-to-consumer right now. My Instagram feed, my closet, and my wallet can attest to this. You can register here for early-bird rates.
Second…
We launched a new project last week. Once we get the okay to talk about it publicly, I’ve got a lot of thoughts to share. The project involves schools and returning to space in a safe and socially-distanced way. Stay tuned…
What I’m Thinking About
I love UI rendering libraries. These are tools designers and programmers use to build beautiful experiences for the browser, the smartphone, and most other devices we interact with.
One of the most popular of these libraries is React. It began as an internal project at Facebook. The company then open-sourced it in 2013, making it available for the world to use.
Since then, over a thousand developers have contributed to the React codebase and many more have gotten jobs to keep improving the library and teaching its core concepts. A huge ecosystem of supporting tools have emerged around React.
A “Push” Work Culture
A few years ago, React redesigned its rendering engine. The original architecture of React featured an engine that pushed chunks of work onto a work stack. When something needed to change on the screen, it would essentially say:
[Subject] URGENT
Hey,
I need a Sierpiński triangle. Make it grow and shrink in size.
Drop everything else you’re doing to take care of this. And don’t you dare ask me what a Sierpiński triangle is.
For smaller applications, this high-urgency stack-based system sufficed. But the more complex and interactive an app got, the more issues the system caused. Like a snake eating a large animal, React would essentially stop everything to try to render these huge, compute-heavy updates.
Notice the lag? The human eye expects 60 frames per second for something to register as “smooth”. The compute-heavy tasks prevent React from achieving 60 painted frames. The above hits roughly 15 fps.
Source: Claudio Procida’s React Stack Demo
A “Pull” Work Culture
The core React team realized this was a problem and they went back to the drawing board. They landed on an interesting redesign that borrowed from foundational concepts in operating systems design.
The new engine chopped up chunks of work into smaller units called “fibers”. These fibers waited in a queue. The worker responsible for doing the rendering pulled fibers when it was ready.
Sometimes the worker put a fiber back in the queue midway through working on it to ensure that the engine rendered 60 full frames. Then the worker would pick it back up and complete the task.
Look at the improvement:
Much smoother. 67 fps. Rendering the same exact visualization, React Fiber uses a technique that maximizes the performance of the entire process by avoiding maximizing the performance of any one task.
Source: Claudio Procida’s React Fiber demo
The key principles behind why the redesign works so well:
1/4 Respect of Throughput: The worker pulled the work when it was ready. There's a limit to how much it can do before performance degrades.
2/4 Batch Sizing: The assigner doled out smaller units of work. Time estimates become easier when you have smaller chunks to estimate.
3/4 Static Analysis: The assigner also detected potential work that didn’t need to be done at all. For example, why re-render something that won’t change and is already on the page? It's a lot easier to get things done when there's not so much of it to do.
4/4 Prioritization: These two systems agreed on a contract. They had a shared understanding of what made something performant. One, 60fps or bust. Two, some types of important work (like user input or animations) are high priority and thus should leap frog to the front of the queue.
First-Order Efficiency == Exhaustion
Turns out there's an art and science to workflow that generalizes beyond UI libraries and operating systems.
Doug Reinertsen wrote a book called Principles of Product Development Flow. In it, he brings in concepts from queuing theory — the mathematical study of waiting lines.
One of Reinertsen's claims is this:
“In pursuit of efficiency, product developers use specialized resources loaded to high levels of utilization.
Our current orthodoxy accepts inflexibility in return for efficiency. But what happens when this inflexibility encounters variability? We get delays.”
A perfectly tuned system works at full capacity, and we call that efficient.
That same system, when fed inputs that exceed or surprise its abilities to function, falls apart.
This system is perpetually pushed to its limits. It has no slack or wiggle room. It accomplishes a first-order efficiency that I like to call “fuuuuuuck I’m exhausted.”
We need to do whatever we can to repair systems like this.
If this system were comprised of humans, workers would start rebelling in small ways to gain some semblance of control. Padding the hours they bill. Putting in minimum effort on each task. Sleeping less. Drinking more, eating more. Sliding into a deep depression. Isolating themselves.
Sadly, a lot of institutions work their people this way, and a lot of us work ourselves this way in our personal lives.
We whip ourselves into frenzies of activity. Sustain this for long enough, we burn out or get jaded. We are first-order efficient, second-order tragic.
That’s heartbreaking. It’s also not effective. So what do we do?
In Life, Love (and Efficient Queueing) Triumphs
Reinertsen's solution to this problem of work is very similar to the solution the React team came up with.
RE: Throughput, don't shoot for full capacity utilization. You'll burn yourself and your team out. Focus on outcomes instead. The literature shows that beyond a certain amount of effort, adding more sustained hours of work doesn't necessarily lead to better results.
RE: Batch sizing, make your units of work smaller. Focus on only a couple things for a few weeks at a time. If the batch is too big and takes too long, there is no opportunity for you to learn and recalibrate. If there are too many batches crammed into a small time frame, that's another recipe for burnout. High intensity bursts require ample time to recover.
RE: Static analysis, do less. Are there things that you're doing just for the sake of doing them? What is the outcome of doing them other than busying yourself?
RE: Prioritization, the people doing the work tend to know what's most needed. Decision-making needs to happen on the shop floor, not by a manager wearing Rec Specs in a board room.
In this world… The worker becomes a craftsman and decision-maker, not a cog. The manager becomes a player-coach, not a task master.
Everyone wins!
So why don’t we do things this way already? Because this system requires a lot of humanity and, for lack of a more technical term, love. We are conditioned not to express these concepts in the workplace.
It requires good-faith negotiation. Authority figures cannot just wave their wand of power to get things done.
It requires trust and accountability. Workers cannot confuse this laxity for complacency.
It requires self-confidence. Everyone needs to be okay with down time. You don’t need to be working during every single minute of every single hour M-F 8-6pm.
It requires vision. Both workers and managers need to know why they are there to work. To provide for our families? To push ourselves and grow?
All of this unravels the same way love unravels … lack of vision, confidence, trust, and negotiation.
In its wake we are left with trying so damn hard to do more, when we’re not really doing much at all.
React renders complex designs by pulling the work, not pushing it. We should follow suit and do the same. A growing number of organizations have taken pretty strong positions on this. See the Bullet-Point Summaries below for more.
Bullet-Point Summaries
Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency
The legacy of the nineties has been a dangerous corporate delusion: the idea that organizations are effective only to the extent that all their workers are totally and eternally busy.
It’s possible to make an organization more efficient without making it better. That’s what happens when you drive out slack.
It’s also possible to make an organization a little less efficient and improve it enormously. In order to do that, you need to reintroduce enough slack to allow the organization to breathe, to reinvent itself, and to make necessary change.
Why Greatness Cannot Be Planned - Kenneth Stanley
The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture.
Objectives might sometimes provide meaning or direction (especially for modest pursuits), but they also limit our freedom and become straitjackets around our desire to explore.
Count Basie, who was a respected name in jazz during the birth of rock and roll, described how new musical styles really come about: “If you’re going to come up with a new direction or a really new way to do something, you’ll do it by just playing your stuff and letting it ride. The real innovators did their innovating by just being themselves.”
As always, thanks for reading. Got a favor to ask of you — are any of you working on anything you’re excited about? Would love to hear from you.
Ammar