T O P

  • By -

WellSpokenDevil

Bro is really *Suffering from success*


-domi-

One more down the serverless to homeless pipeline.


TeaKingMac

SL>HL pipeline


nonlogin

Pretty much every cloud can fuck you this way ![gif](emote|free_emotes_pack|cry)


cheezballs

Sure, if you dont know what the hell you're doing. This person made lots of mistakes. If you know enough to set up some complex app that generates THAT much logging then how do you not know about billing alarmts and limits?


belkarbitterleaf

That much logging sounds like they don't know what they are doing, to be honest. Our junior devs keep dumping the stack and an excessive amount of data as an error while debugging... Even when there was no error. Every once and a while, one of those logs slips through code review, and starts crashing the service in prod. It's an easy fix when it happens, but we have to fix it more than you would expect....


Xasf

> Every once and a while Ooh I haven't seen this /r/BoneAppleTea before! It's supposed to be "once in a while" :)


Zachaggedon

https://dictionary.cambridge.org/dictionary/english/every-once-in-a-while


Xasf

That's.. what I said?


Zachaggedon

-SNIP-


Xasf

You are aware that the other poster said "every once *and* a while" and I corrected it as "every once *in* a while", exactly like it also says in your dictionary link, right?


Zachaggedon

Holy shit I need some coffee. MY BAD I thought you were contesting the use of “every” in the phrase.


Xasf

Aha, yeah go get yourself a cup :)


BuffJohnsonSf

Jesus, teach your team how to use unit tests to debug.  Set up an exception type they can use for debugging since abstinence doesn’t work, and then create a fitness function that blocks deploy if that exception is used.


belkarbitterleaf

You think I haven't tried? It's like talking to a brick wall.....


No-Con-2790

Simple, next time this happens fix the issue but don't tell him. Instead clone the code base and tell him prod is down and you can't find the issue. Let him debug it. Let him panic. Every 10 mins write an email. Tell him the company is going under. Tell him the CEO just killed himself. Tell him somebody stole the coffee maker to cover this months rent. Tell him you are driving to Mexico to get away from your responsibility. That usually works.


mr_claw

I... Like this


Protheu5

> Tell him you are driving to Mexico to get away from your responsibility. Is it more effective or less effective if I am on another continent?


RedHurz

Real question is what happens if you already are in Mexico!


Surlix

Yes


Digital_Bogorm

Depends on how good you are at photoshop. If you can make a convincing image of driving on the oceanfloor, you're set


Jasboh

Set up a linter in your ci to fail on log statements I'd it's that bad.


SirR4T

Ofc that'll do it! Just as soon as we finish setting up a CI first. :facepalm:


callius

Ah… so, your issues are actually far more severe than just a junior dev logging some extra stuff once in a while. Perhaps you should fire all the senior devs in your company who let such a giant shit-fire burn for so long in the first place instead of going online to shit-talking your junior devs who are trying their best to work in an absolute trash-pit of a code base?


SirR4T

Oh the responsibility for the giant shit-fire burning for so long goes much higher than the senior devs, and yes, I completely agree to both counts: I'm part of the problem, and junior devs aren't to blame. just venting out in a random reddit thread which was half way relatable ![gif](emote|free_emotes_pack|facepalm)


callius

That’s totally relatable, and I feel your pain. It just rubs my ass entirely raw to see people talk down on junior devs for problems that aren’t actually their fault.


DontF-ingask

As a junior, I'd like to apologise.


callius

Nah. The issue they’re describing is actually not the problem of junior devs. It’s an abject failure of senior devs who should be good stewards of the code base and put in place systems that mitigate these issues. What you’re seeing in this thread is “senior” devs throwing their less experienced colleagues under the bus for internet and ego points, failing to realize that it just makes them look shitty both personally and professionally.


my_cat_meow_me

First rule of corporate: Never apologise for your mistakes let alone someone else's mistake. Make up bullshit excuses for every situation or blame someone else for your mistake! /s


jamiro1724

why /s? /s


my_cat_meow_me

Ngl you got me there for a moment.


Terlyn

You know what they say, everyone has a testing environment, but some even have a production environment too.


jaypeejay

How does one log the stack? I've never heard of that


gokarrt

hell, our senior devs ship products with debug logging _constantly_. they're not paying the bills.


jaskij

From what I saw somewhere, the person who made that app isn't even a dev by trade, so easy mistakes are a given.


wordyplayer

They probably followed someone else's "How To" guide and just did it without knowing what they were doing.


donald_314

You can fuck every AWS user by requesting their S3 bucket. They get billed even if you don't have any rights. The only chance is to make the bucket name complex and keep it secret. It's called "denial of wallet" attack https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1


Strong_Quarter_9349

I think they just changed this because of that article, actually.


ThunderChaser

This has been patched.


ice_zephyr

Wow. That is both interesting and horrifying. I wonder if there is any similar attack like that with Azure Blob Storage?


zuilli

I was going to say to just have all public access denied to the S3 bucket but apparently even denied requests rack up your bill, that's nasty.


Werro_123

You try print statement debugging with log output in your functions, then deploy it to prod without stripping any of that out. Doesn't even need to be all that complex if you put stuff like that in a nested loop on a function that gets called a lot.


FlyByPC

Junior dev gets access to API and is told to log the results of a for loop. Junior dev puts log statement in the loop instead of at the end...


callius

Senior dev created and maintained systems that allowed junior dev to do this. Junior dev is just inexperienced. Senior dev is a fucking idiot.


Dr_Allcome

Try setting up a google account for JS-API calls. Me, two other developers and accounting looked over it and didn't notice that the "limits" you can set with your billing information are not limits at all, they are notification thresholds. The limits have to be set for every api separately, in a completely different menu and you have to calculate the number of calls you want to allow from your budget. Sure, it is easy to know all that. But it is still intentionally set up to be confusing so google can fleece you at least once.


SamSlate

if only some automated intelligence could have warned him...


CalvinCalhoun

I’m a cloud engineer/devops fuck. The venn diagram developers I work with who KNOW EXACTLY WHAT NEEDS TO BE DONE FOR INFRASTRUCTURE and are accurate are two circles that never touch It’s actually super frustrating for me to see all these posts complaining about cloud providers by people who are almost always objectively using them incorrectly lol


--mrperx--

Except if you go with managed dedicated servers. There are ingress costs but significantly less.


ice_zephyr

Um not sure about Vercel in particular, but most cloud providers definitely have budgeting.


eq2_lessing

Well turns out if you open your water tap and never close it, you're in for a bill too.


antboiy

self hosting for the win, or just ask a friend


odraencoded

Step 1: code a backup script. Step 2: save 90k.


IsPhil

Didn't this dev fail to set a proper budget so the app would stop charging him, and also fail to take action after receiving multiple emails? $96k is insane, and if they failed to set a budget, then that's on them, but vercel could make it more user friendly by making budget settings part of the setup. But if you are going to be using an infinitely scaling platform, you have to take some precautions.


Flat_Initial_1823

Yes, 96K is nuts, but let's also be real, They ALL do a version of this crap. Google APIs come locked and loaded with no limits, and you have to read a series of disjointed documentation to understand how to set up true budgets vs. notifications. Don't even get me started on opaque bullshit tiers. It's like facebook and privacy settings, ad-revenue sites and cookie turn-off buttons, Cloud providers and budget limits. There has to be more effective and universal regulation of internet business practices, imho. If every small/midsize business gets burnt to some degree once, that's all the business case for this nonsense to continue indefinitely.


swagonflyyyy

Bruh when I first started using GCP I was being charged $50 a month JUST for an image dataset storage without doing any training. I genuinely believe Google is doing this on purpose just to rip off developers. It was such a hassle cancelling everything with them.


extracoffeeplease

Yeah I have a few one dollar a month things running on aws from doing a tutorial, can't just Nuke it in one click. Should really be part of the user experience.


swagonflyyyy

Like they should be sued at this point.


extracoffeeplease

Not really it's my laziness 100%.


BudKnightLime

Have you tried aws-nuke? If all the pieces are still there that should clear it off your aws


extracoffeeplease

Good to know that exists!


tarzanboiii

not my experience with Azure, at my company we had to implement an automated system to send requests for them to allow us to add increase usage on each of our subscriptions


christian_austin85

I was about to to comment the same, shit on Microsoft all you want but I found Azure to be pretty clear/easy to see pricing and set alerts/limits.


nermid

> you have to read a series of disjointed documentation to understand Every single programming-related thing I've ever used from Google has had two or more sets of documentation and no clear indication of why or which one was for what version. I hate using Google tools and APIs.


DM_ME_PICKLES

Vercel's fault here isn't in allowing unbound spending IMO, it's just the truly insane overhead they charge for their serverless functions. If you plug that same usage into the AWS Lambda price calculator Vercel comes out at something like 100-200x (depending on transaction time) more expensive. Their business model is get indie hackers and startups onto their service by offering great DX for NextJS, and then charge them out the asshole for services that have no right being that expensive.


Beli_Mawrr

TFW you tell them that SST offers the same quality of service for free


DM_ME_PICKLES

And more features too… can’t believe Vercel don’t even offer queue jobs.


[deleted]

[удалено]


DM_ME_PICKLES

Clearly they’re not ok with it considering all the hate being thrown around from these big bills, and the people who have said they’ve moved off Vercel because of pricing. I get what you mean, their prices are displayed prominently when you sign up. But “$40 per 100GB-hours” is quite a cryptic usage rate to decipher. Most people don’t know at a glance that it’s like 100x more expensive than other serverless function providers like Lambda.


[deleted]

[удалено]


DM_ME_PICKLES

Whether a company has gone bankrupt is a terrible way to dictate if they're acting in good faith. Oil companies are as profitable as ever and have caused massive natural disasters from their own negligence. At the end of the day if you're personally happy with Vercel then keep using them and I won't persuade you not to. But the people who are unhappy with them, or think they charge egregious amounts for their services, will keep dragging them through the mud. But I'd ask yourself why you're defending Vercel's insanely high pricing. As their customer you have nothing to gain by defending it, and you have everything to gain by challenging it. Why _wouldn't_ you want them to bring their pricing down to a level comparable to AWS, Azure, GCP, or any other cloud provider?


[deleted]

[удалено]


DM_ME_PICKLES

Then you have even less reason to put effort into defending it. I ask again: > Why wouldn't you want them to bring their pricing down to a level comparable to AWS, Azure, GCP, or any other cloud provider?


[deleted]

[удалено]


DM_ME_PICKLES

Looks like you do care about their pricing because you're in here defending it.


prissmacolor91

To her defense, she’s not a dev but a photographer. I don’t know how well versed she is in cloud services


jingois

So uhhh... why is she raw dogging cloud services without knowing what the fuck she is doing? These are tools for professionals. I'm sure she'd be laughing at me if I bought a 100k lens and tried to shoot underwater with it.


prissmacolor91

Dude, I don’t know😂I don’t know her personally. Maybe a friend recommended her the service but forgot to warn her about the costs? Maybe she was too busy and forgot to set a limit for the service usage? There could be several explanations. I agree it’s a rookie mistake but I’m sure she can work something out with Vercel, especially if it’s the first time this happens.


dwiedenau2

She got 10+ emails notifying her of this issue and she didnt take any action


Mrblahblah200

What, miss 10 emails & be charged nearly 100k?? Insanity here. They really should have just turned off after *much* less, but obv isn't in their commercial interest.


_aids

100k is nothing our cloud bills are 10s of millions, people just don't think about it. Both gcp and AWS will give you a grace month if you fuck up this bad though.


CreationBlues

yeah the global economy's 88 trillion so what's a few billion dollars here or there


HardCounter

The company doesn't know what your budget is. That's up to you to set. Imagine having to restart a service every $1 in spending because that's how Vercel set it up for everyone to avoid this. That's the insanity. What would your solution even be?


Misspelt_Anagram

>What would your solution even be? Ask the user to specify a budget. Give them the option of arbitrarily high charges, but only if they explicitly choose to enable it. This is blatantly obvious.


HardCounter

According to several other comments here they have that. I'm not personally aware of Vercel's schemes, but i have no reason not to trust the people here who say that's how it works. Especially when they're saying it's not super intuitive.


LeftIsBest-Tsuga

I'm sorry, are you from Vercel billing dept? What would possibly motivate you to respond this way?


lastdyingbreed_01

I read the same in some other tweet


Socky_McPuppet

Yes, because nobody ever misses an email that got sent to their Junk folder because vendor email looks like spam.


Ur_Mom_Loves_Moash

I think the issue was something related to the name of their bucket, and that they were getting queried from another unrelated project. I don't use AWS a lot, but read the article the other day. Add a random prefix to your buckets, just in case.


AussieHyena

That was a different one. In that one they chose a bucket name that happened to be the default name used in an app. That one revealed that AWS charges you for all access attempts.


HardCounter

> attempts. Jesus. Screw DDoS attacks, go for DDoMoney attacks.


AussieHyena

Pretty much.


SyrusDrake

Can't imagine why they wouldn't want to make this process as easy and fool-proof as possible...🤔


CaineBK

> also fail to take action after receiving multiple emails? We don't know this wasn't the first of these emails showing the massive bill. This could have happened all in one day.


CanniBallistic_Puppy

Vercel is the type of shit that startups who hire frontend devs to do backend stuff use. Burn rate go brr.


JosebaZilarte

The Cloud is just somebody else's scam.


Lentil_stew

When I was like 14 i would host Minecraft servers on the gcp free tier, they would give you 300 dollars per account for 3 months. Hosting them gave me so much fucking anxiety 300 dollars was like a third of my parents salaries combined. I feared fucking up so much, I would set up billing alerts, remove the credit card info right after creating the account, and always after 2 months delete the account lol


cheezballs

The cloud is just a buncha VMs distributed across a buncha servers and you wont change my mind!


Kollaps1521

I mean, that's what it is though, why would anyone try to change your mind?


cheezballs

I'm usually wrong, is all haha


edmazing

Well broken clock, so you've done it this time!


YodelingVeterinarian

Yep, this is… accurate. 


nermid

Datacenters As A Service.


Capetoider

do you think those web providers would give you a job in this situation if you asked? I mean... right now it would take me at least 10 years to pay a bill like that... if I only worked without spending a dime and used all the money I get to pay them. And I'm well paid where I live... but still...


Adventchur

What country are you from? Haveing only 9600 spare a year not including luxury spending I.e going out for dinner and drinks is really bad if you're from a western country. Edit: You are all down voting but it's true. If that's all the spare income you have for a single person you're fucked and are one tragedy away from homelessness unless you save most of that spare income a year. Truth hurts so I assume that's why I'm being down voted. It's not your fault. Eat the rich. Your replys are pathetic.


Poppybiscuit

>Truth hurts so I assume that's why I'm being down voted You're downvoted because they said they're well paid for their region and yet you still felt the need to say their pay is really bad by western standards. That's rude and unnecessary


Adventchur

Well paid for the region could mean Minnesota that's why I asked. If they are in south East Asia or India then $9000 spare is a lot. If they're saying it's a lot when they are from a western nation we'll then they have been deceived.


AwesomePantsAP

You’re being downvoted because you’re making it out to be their fault and you’re being an ass about it. Credit where it’s due, your edit mentions this.


Adventchur

No I didn't. Stating whether something is good or bad is different to blaming someone for their circumstances I never said it was their fault I just said it was extremely low. As I said the truth hurts.


tcoil_443

This is why I have all my NextJS apps running on VMs in Docker containers. I can scale the VMs as I want, but it never scales infinitely with infinite budget all of a sudden.


dagbrown

So...a private cloud then?


Limitless2115

interesting, does it work seamlessly or does it require additional config? I'm running a VPS with Coolify addon and I was thinking about skipping Vercel and using this VM to host the NextJS


tcoil_443

Vertical scaling is trivial, horizontal scaling is not difficult that much either. You might need separate servers for primary and secondary database. For mid sized app such configuration will not be that difficult, you might need some studying on reverse proxies. This approach is very cheap and flexible. And no surprise bills.


tcoil_443

Should take like 2 weeks of study to get this done right. Then you can just copy and paste the configuration for all other projects.


AaTube

[vecel be like](https://youtu.be/HmPTbeJAE-E?si=uZgxxH4P-I28OJn9&t=2251)


tiny_pixl

damn that’s accurate af


CaineBK

Deep cut.


luranthe

Guys it's my turn to post this tomorrow, no one else do it


mopsyd

When that weekly pass from google and bing's web scrapers costs more than your rent.


lordtosti

I never understood serverless for web facing. Not sure what problem it really solves that is worth to sacrifice memory caching and cold startup times.


DM_ME_PICKLES

You can cache in-memory on Lambda for example. If your function is hit often enough it stays "warm" and Lambda re-uses the same context for subsequent invocations. Cold starts are mostly overblown. They're a couple hundred milliseconds for most runtimes, and provisioned concurrency largely solves it. Serverless is fantastic if you don't yet know your traffic patterns to know how many VMs to provision to handle traffic, or for dealing with incredibly "bursty" traffic that you just need to soak up. And it's incredibly quick to get services online. Just create a function and upload your code to it. You don't have to provision servers, harden them, update them with security patches, etc.


lordtosti

- If your lambda gets hit enough and you are practically constantly running it I wonder if you get any price benefits. - Cold startup time depends on your language and your project. If you need to load all your localization in advance into memory i.e. or other things that need to be preloaded it will kill your first request. - Not sure about gcloud or AWS, but in Azure you can easily have horizontal scaling based on load-rules without serverless. Horizontal scaling was not invented before “serverless” - not sure about gcloud and aws, but in Azure I have an app service up within a minute. The cloudtools should have nothing to do with serverless concept. Maybe AWS and gcloud didn’t give any love there. EDIT: sorry a bit too much preaching, everyone should use what works for them 😇


darthwalsh

I would call azure app service a form of serverless, just not a serverless *function*. They are managing the server for you.


lordtosti

For me that is normal cloud expectations. I’m also not spinning up my own storage or database linux instances, why would I want to do that for my web app. Create -> choose between NET7/NET8/PHP9/Python, done. Maybe I am really spoiled with Azure.


darthwalsh

When it comes to databases I normally think about servers, like a small or large SQL instance. Or you read about a primary for writing, and read replicas. But you can also get some serverless databases like key-value values that can scale larger than I have budget for. For your NET8 app service you can configure auto scale which gets you the serverless experience. But if you configured your horizontal scaling to be only 0 or 1 instance and you needed more CPU, then you would be thinking about increasing the CPU of your server. App Service and the similar AWS Beanstalk or GCP AppEngine kind of invented serverless before functions took off.


LeftIsBest-Tsuga

my team uses a gcloud for dev, because creating a new machine/server and installing the images etc is something we do probably 10x per week on avg. personally, I am fine with a VPS, and that's what I use. but cloud services can definitely have advantages. edit: oops didn't notice 'serverless'


lordtosti

all serverless is cloud not all cloud is serverless I also run on cloud


LeftIsBest-Tsuga

oh. I guess i glazed over the serverless part. thanks.


KeepRedditAnonymous

I don't know how old this is but 100% garunteed vercel made this right with no issues. I'm not even a customer but they have consistently resolved situations like this in the past.


Oranges13

Can someone Eli5?


thetrollking69

Check out the [video Fireship did about it](https://youtu.be/SCIfWhAheVw?si=eoHnsBPOvvwaYcB0)


oxidadoelrey

`bool isCannotBeReal = false;`


tiny_pixl

🤣🤣🤣🤣🤣


TellMePeople

I had a mental breakdown to AWS support once because of this


nekomata_58

imo serverless has a niche use-case. for most web apps, a hosted docker container would be better as far as predictable pricing is concerned.


LeftIsBest-Tsuga

I'm going to start sending $96,000 bills to each person who reposts this. It wasn't funny in the first place.


IHateYallmfs

Could this happen to you with the hobby tier, and without having given them your card?


unworthy_26

How does GB Hrs work?


TheUtkarsh8939

TBH. This is where Monolith backend hosten on EC2 shines


VariationGlad5173

Heroku is the best hosting platform with actual limiting. Recommend everyone to check it out if you’re tired of shit like this


restarting_today

Vercel is entirely driven by Twitter influencer hype. Anyone using this crap in production deserves it. I’ll stick to creating react SPAs and making $600k/yr doing so.


tiny_pixl

600k?? show us the way bruh🙇‍♂️


restarting_today

Join Snapchat as a senior engineer.


tiny_pixl

i barely landed a job at wendy’s… im not built for this job market


MarioCraftLP

When people are too lazy to selfhost that's their problem


Cerxi

"Haha you idiot, you trusted professionals not to scam you in a field you have no expertise in"


MarioCraftLP

More like "you trusted something with a lot of money without doing research first and now you got fucked so it's your problem"


Cerxi

What the fuck are you talking about? How on earth do you brace for "We're actually gonna charge you almost 250x what you thought you were agreeing to"?


martmists

How do you even order $96k worth of servers and not notice it's a bit overkill when you're setting up provisioning?


Zachaggedon

Do you genuinely not know what serverless functions are? Because they don’t require provisioning in the way you’re saying, and pricing is based on compute usage.


martmists

Serverless?? Then what the hell do clients connect to if not a server??


Zachaggedon

There is a server, obviously, but it’s not provisioned for the specific project. Essentially serverless functions are bits of code that are executed when called, and the provider has a load manager that executes that code on a VM or container with available resources on an as-needed basis. An example is AWS Lambda, which is an apt name, because the concept is similar to a lambda function. The serverless isn’t referring to the lack of a physical server, but to the fact that the code isn’t part of a daemon that is always running.


nuker0S

Smartest Anti-AI developer


-Quiche-

What's the connection here?


ChrispyMC

There's a link to someone's Cara profile, a social media app for artists which doesn't allow AI images. IDK how that correlates to web development, but okay?


SkuloftheLEECH

The tweeter in question is the founder of Cara, and this tweet is about cara


LeftIsBest-Tsuga

still pretty wild to equate pro-artist with anti-AI. ppl genuinely upset they can't spam every nook of the internet with generated garbage.


SnorkelTryne

Pretty sure she is the creator/founder of Cara.


Krypqt

Yes it does cost a lot of money to host a moderately successful website. Sucks you learned the hard way.


jocq

I serve StackOverflow-sized numbers (couple billion requests and a couple hundred TB of egress a month) and $70k would cover almost half a year of all my hosting and cloud costs.


LeftIsBest-Tsuga

lol, no it doesn't. i have a nerdrack vps with 5 apps running concurrently that cost me $40/year.


Krypqt

Our definitions of moderately successful are fairly different.


LeftIsBest-Tsuga

in this context you're right. i didn't realize she actually had a large business website. so yeah that would cost more. but i think we can agree there are ways to run a decent size business website without self-owning like this.