|
I've done it more than once. The screams usually come pretty quickly.
|
|
|
|
|
I've had to do this as well since no one seems to know what their resources are.
|
|
|
|
|
One chap I was called to help around 25 years ago explained to me that every time he pressed a key on his keyboard his computer would restart.
So I went over to his desk and observed what was happening and yes the computer did shut down every time he pressed a key on the keyboard.
Now my Sherlock Holmes senses kicked in as I noticed that he also had a folder resting on his lap.
Every time he leaned forward to press a key, on the keyboard, the corner of the folder pressed the restart button on his computer...
“That which can be asserted without evidence, can be dismissed without evidence.”
― Christopher Hitchens
modified 26-Aug-22 3:43am.
|
|
|
|
|
I once deleted ALL of the car loans for FL because someone changed the "mechanical" terminal Switch,
and I was NO LONGER on the DEV box.
OUCH!
A month later, someone else did a different state. And I patched the operating systems for production to block the command (renamed it to another command we would NEVER type accidentally)...
|
|
|
|
|
|
|
I accidently tested a database shutdown script I was developing on the PROD system, but who hasn't done that?
One day I found out that a DEC AlphaServer 1000 has a switch which kills power if the case is opened while the system is running. I mean, WTF? Definitely over-engineered. All I was doing was determining whether or not there were any spare RAM slots. : sheesh : At least that was a DEV system.
|
|
|
|
|
Hmm,
Done both of those! 8)
Worst for me was - on a production system - deleting an 'old' copy of a database from the backup filestore during a very busy morning. You guessed it - not the 'old' copy at all! This was all long before SQL (or even Linux) was a thing on PCs.
Had to restore from the overnight tape backup (we did have backups, and they were regularly tested, fortunately) and then get the reservation staff to go through all the paper printouts of that day's bookings to recover the mornings transactions, whilst suspending new ones to guard against double bookings. (Fortunately I also produced a paper copy of every transaction as - back in the day - neither the power nor tape backups were 100% reliable).
I was not Mr popular that day, I can tell you!
I learnt my lesson though and added an hourly local backup to a .zip file of transaction data just in case. That saved my bacon a few times too!
|
|
|
|
|
|
What the !?
Was this guy fired?
Did he die in a freak "accident"?
This is closer to sabotage than to ignorance
|
|
|
|
|
Sander Rossel wrote: Was this guy fired? Nope. The guy who did it, just did what he was ordered to.
The explanation is: big OEM company.
It is not that the right hand doesn't know what the left hand is doing. There are many times when not even the fingers of the same hand know what the others are doing.
M.D.V.
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
|
Back in the mid-80’s at a former employer who did photo processing they did a major upgrade of the DEC PDO-11/70 systems located in their multiple film processing plants to Microvax. Each of the new Microvax systems was setup with two large hard drives with one being mirrored on the other. They were hot swappable so if something went wrong with one drive they could stop it and keep going with the other one while the bad drive was sent for repairs. So one morning at work we were alerted to a problem that a bunch of invoices were missing in one of the plants. It turned out that at this particular plant they had to get a hard drive repaired and when it came back a few days later they started mirroring the drives again only someone (not me, thankfully) did the mirroring in the wrong direction so the drive that had the old data overwrote the drive with the current data. I’m not sure what they ultimately did to get the data back since I don’t think there was a tape backup because of the disk mirroring (I think it was probably the same person who thought they no longer needed the redundant DECNet cross links between the plants) but I’m thinking one of the devs had to write something so some poor workers in the plant could do a bunch of data entry.
|
|
|
|
|
Classic!
Actually iTunes did that to my iPod once.
First (and last) time I used it.
Don't even own an iPod anymore now.
No Apple a day keeps the doctor away
|
|
|
|
|
Congrats on your immortality! And welcome to the club 
|
|
|
|
|
I was immortalized many years ago actually
You don't get to be in IT for twelve years without accidentally deleting a table or two in production (well, I never did that, but locking an entire database was pretty bad too)
|
|
|
|
|
About 25 years ago I learned an axiom that I live by.
"It only takes one "Oh Sh*t!" to wipe out 25 "Atta boy"s.
|
|
|
|
|
Good post. There is absolutely no point working when you're tired. You get next to nothing useful done and you make mistakes. It also leads to burnout, which can be devastating. It (burnout) happened to me and also to a friend of mine, and it put us both in hospital (at different times). It took me a long time to figure all this out, but I've got it now.
I work for myself nowadays so I can do what I like, but more employers need to understand this and to take proper care of the mental health of their staff.
Sorry Sander, maybe a bit more than you were expecting, but I feel very strongly about this.
PS: Take breaks. Take breaks, take breaks, take breaks.
Paul Sanders.
If I had more time, I would have written a shorter letter - Blaise Pascal.
Some of my best work is in the undo buffer.
|
|
|
|
|
My first job was 100% Assembler. In an effort to squeeze out a few more bytes of memory for variables, I was cleaning up a program and deleted a single space located before the variable used in the print command. The program worked fine but, printed a few boxes of pre-printed forms with garbage due to the preceding valid blank space being removed leaving another character to be mvcl'd across the print line before it was built. Almost a career limiting move.
|
|
|
|
|
Hard work, assembler. I'm too old now to have the focus you need (but there are compensations).
My favourite oh no! story is from my college days, programming on punch cards (ouch, that really dates me). Somehow, a rogue card sneaked its way into the deck and as a result my printout consisted of 2000 or so blank pages. I bet the ops loved that!
Paul Sanders.
If I had more time, I would have written a shorter letter - Blaise Pascal.
Some of my best work is in the undo buffer.
|
|
|
|
|
Paul Sanders (the other one) wrote: There is absolutely no point working when you're tired. I sometimes get into a work vibe when I stumble across a problem and I make it personal.
Kind of an adrenaline rush I guess
Paul Sanders (the other one) wrote: Sorry Sander, maybe a bit more than you were expecting, but I feel very strongly about this. I agree wholeheartedly though
Paul Sanders (the other one) wrote: PS: Take breaks. Take breaks, take breaks, take breaks. I take works from my break
|
|
|
|
|
Lovely thread!
I once had an office of around 25 people running on a Linux mail and file server, which ran inside a Hyper-V VM. This is for a non-profit and I am working for free... I was really proud of the architecture, a replicated server (for fault tolerance) and 24 hourly automatic checkpoints, plus external backups.
The tech guy who was in charge of keeping this running, a friend of mine, also joining in pro bono, learned to create a checkpoint (aka snapshot) of the VM before running OS updates, just in case they needed to be reverted. Once he was hesitant about some problem and left the checkpoint there.
Months go by. The checkpoint keeps growing on the disk, but that's not going to be our problem.
Now a person in the office is having trouble running some Webmin commands and the tech guy teaches her how to do it directly from Linux. So now we have a user playing with an admin command-line, but that's not going to be our problem.
So she wants to run those commands from the Hyper-V server for convenience, and there's no Putty there. But there's this nice Hyper-V console which even has some nice icons on the toolbar. Like that useful "back" button that she pressed to get back to a previous screen or something she regretted doing.
Only it's not a "back" button, despite having an arrow pointing to the left. It's "Revert" - a really efficient time machine taking you back months to the point of that forgotten checkpoint!
Replica server does what it is designed to do - replicates all those changes throwing the data away. That's the reason why I always tell people that replicas aren't backups...
Nobody called me in the initial 24h. Hour by hour, the valued data is replaced with the old stuff. Then I am called, and learn that backups had been long neglected.
Over 20 people lose many months' worth of work... on an architecture that has more layers of prevention than most people even care to do (at least locally, I know a professionally-run cloud service would be different).
She think it was her fault, but my friend knows it was his.
In reality it was probably mine, in some dark twisted way...
|
|
|
|
|
You either resign a hero, or code long enough to see yourself become the villain.
|
|
|
|
|
So you felt yourself slipping away...?
How's the new job?
|
|
|
|
|
Certain trendy editor was after a Swiss man's number at the bar. (10)
"It is easy to decipher extraterrestrial signals after deciphering Javascript and VB6 themselves.", ISanti[ ^]
modified 25-Aug-22 3:40am.
|
|
|
|
|
Ok we give up - answer please
Life should not be a journey to the grave with the intention of arriving safely in a pretty and well-preserved body, but rather to skid in broadside in a cloud of smoke, thoroughly used up, totally worn out, and loudly proclaiming “Wow! What a Ride!" - Hunter S Thompson - RIP
|
|
|
|