It's little surprising that the author was doing perf work and not already comparing distributions. As the rest of the article outlines, you learn a lot more with more data!
It's complicated, but really worth learning how prometheus and grafana heatmaps combine if you want dashboards for real time service data. Multimodal distributions are basically the expected outcome given all the caching done in distributed systems.
The median developer doing performance work has no background in the subject, they just have a complaint from someone that something is slow, and no real idea how to solve that problem.
By that standard, this guy is doing quite well. It's a good post.
True, I guess I just presume someone blogging about it has more than average insights on the subject, and the bar further raised by posting to HN. They do get to somewhere interesting by the end, I just had to scroll past a lot of basics.
> And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.
Thats a pretty strong outcome. It implies that not only are GPUs that power AI too expensive long term, but they cost too much to operate even if they were free.
Seems to me the more likely outcome is a wave of dotCom style bankruptcies wiping out equity holders for companies who contracted to buy chips and datacenters at MSRP, and a second wave for the groups that step in after to operate whats left without the absurd financing charges and lower capex.
They are free. You can download glm-5.2 and run it on your own hardware. Even you got hardware for free, electricity would cost you more than the sub, in most places.
It's true that markets are more _adversarial_. But there's still a lot of trouble with distribution shifts even in server metrics. As an example, our SRE team got paged a few times in the past month for traffic drops due to the World Cup. This stresses the nowcasting alert in several dimensions:
- there's no seasonal pattern to the matches, they happen sorta randomly.
- they drive increased query traffic in the hour or so before the game
- then during the game usage drops, sometimes to below "normal" depending on time of day and who's playing
So... now the accuracy of your forecasting tool depends on correctly predicting when world cup matches happen, and also who wins them!
edit: and this is just one recent example. others involve severe weather, national gameshows, earthquakes, and when you celebrate christmas.
The UK power grid operators famously plan for massive demand surges at eg the end of major football matches. I cant imagine what their forecasters thought of the England-Mexico nail biter. Half the nation heading to put the kettle on, half glued to their seats. Would love to see the charts of demand now that the World Cup is done...
They used to plan not just for major live events but for the end of popular soaps like Eastenders. With live TV becoming less important culturally and renewables eating the world most of their forecasting these days goes into weather predictions I think. Interestingly even other forms of power generation are affected by the heatwaves we have seen recently - it can reduce gas turbine efficiency by 10% or so and can even reduce the safe output of nuclear if their cooling water comes from a source that gets too warm.
But I’m sure the World Cup is still pretty relevant for operations.
Is a Chinese robot vacuuming and mopping my floor every day? Yes it is. Could I do it better manually? Absolutely.
I still prefer the robot to do an ok job than to do it myself.
Most people in software misunderstand how physical goods work. When making machines you don't care about getting everything perfect all the time, you care about staying within tolerable limits all the time.
How bad do the folds in your laundry need to be for them to be no different then just being dumped in a pile? As long as we are above that point the robot is useful.
I think there is a stark contrast between the plus or minus micron level tolerances, and whatever is being churned out of LLM ('not getting it perfect all the time')
Eh, this still has value for me if it can do it reliably. Leaving a robot to badly fold my t-shirts overnight and then badly put them in my closet the next day is fine. I don't need my casual wear beautifully folded. But I do need it organised, and that's apparently a chore I'll just never learn to like doing.
Physics. Right now the clothes folding is computationally limited, but assuming we get faster computers to get them folding faster, gravity and air resistance are still going to take their time.
Exactly. I will pay money for a robot to take laundry out of the washing machine and hang it to dry and then fold it and put it away or just sort it by size. This is a chore that my wife and I have to do several times every week.
My solution: stuff the clothes from the washer into the dryer, then stuff them into my wardrobe. I buy clothes that don't get wrinkled much even if simply shoved in the wardrobe and if they do, I tolerate it and wear them anyway.
Ha! You remind me of something I used to tell my colleagues back when I held a regular job. They would compliment me on my highly patterned shirts (eg. A very small floral pattern repeated). I would explain that a shirt like that just requires low spin in the washer, and then drying on a hanger. Whatever wrinkles are left are almost invisible due to the pattern. With collars and cuffs I’d just literally manually stretch/tension them while they were still wet. Bada bing bada boom.
This is a mostly American perspective that I find endlessly amusing. I only have a handful of clothing items that say that tumble drying is fine on the tag, and most of my clothing is very casual wear (t-shirts, jeans, sports clothing). Dryers eat up clothing like it's nobody's business. Plus, electricity costs money while the sun is free. And while yes, the time to hang laundry has value, it takes 10 minutes to hang a load. If it rains, I hang the clothes indoors with a dehumidifier which is cheaper to run than a dryer and dries clothes in a matter of a couple of hours.
> Plus, electricity costs money while the sun is free.
Paradoxically, wholesale electric costs are negative during the day here in California[1], especially in the Salinas region. In theory you could get paid to tumble dry clothes.
I was referring to shirts specifically. If you put any kind of cotton shirt in a dryer it’s almost certainly going to need ironing. Waiting for a shirt to dry naturally shouldn’t be a big deal.
So what? it can take hours to do it and I can focus on something else. If these robots actually become available on affordable price I would buy one without blinking. It would totally free up my weekends, no more hours wasted doing laundry, cleaning the apartment etc.
Sure but that holds true for most new tech. Expensive and slow but it can only get better or stay the same. I'm thinking it's only going to get better.
There's a difference between how you describe using "honestly" and how claude seems to prefer tokens like "honest" and "load-bearing." An example from some coworkers attempting to replace product managers with Claude.
> Deliberately avoid a heavyweight "alert governance" process; the lightest recurring check that keeps FP-rate honest is the right dose.
And one for load bearing:
> Five open questions still stand; the load-bearing two are the runbook-AC contradiction (ratify "high-priority set only") and pinning the "high-priority set" definition + SLO source-of-truth before Milestone 3 (small-sample noise on a low-traffic fleet).
This style of prose sets my teeth on edge and practically gives me PTSD I see so much of it. I prefer code but I get paid to read this shit instead now.
I want to say "ok, and now say that in a way that doesn't sound totally bizarre" yet instead I sigh and continue.
I think the request here is not about sounding like Majel Barrett but in keeping the output extremely terse and unobtrusive.
There's been a few studys showing that novices love LLM output that's long, but experts hate it. As an example, I've been tasked with using some agentic PM tool to write specs, and it keeps generating these huge page long outputs with "HBR voice" bolded summaries of paragraph long bulletpoints. I.e.:
> Right-size hard, and watch the one open-ended edge. Endorse the DRI's simplifications wholesale: drop the runbook-per-alert mandate (keep 1–2 diagnostic-only runbooks for the high-priority set), and ride durability on the existing weekly incident + monthly operational reviews — no new governance. The single scope-creep risk is the coverage strand (gaps are defined by absence); bound it to gaps evidenced by real, already-missed customer-facing outages, not a proactive gap hunt. Curing ownership gaps (e.g. foo-bar, no clear owner) is finite in-scope work.
There's dozens of these every iteration. I can't imagine trying to deal with that via voice, I would just zone out after the second sentence.
I was just wondering if there was a postgresql specific version of https://use-the-index-luke.com i could send a coworker who seems oblivious to the perf penalties of missing indicies. Certainly a good start!
Very optimistic of you to assume that someone who doesn’t understand why DB indices in general are important would care enough to read a detailed explanation about them.
It's complicated, but really worth learning how prometheus and grafana heatmaps combine if you want dashboards for real time service data. Multimodal distributions are basically the expected outcome given all the caching done in distributed systems.
reply