Current Events > I accidentally took down parts of the corporate network tonight

Topic List
Page List: 1
CableZL
07/02/19 12:20:33 AM
#1:


Oops, lol

Fortunately nobody was in the building working, so I'm the only one who knew it happened. I called my boss to let him know because he's on call and he probably got alerts because of it. Fortunately, I had it fixed after about 15 minutes of the network being down.

I was adding some new switches to give some newly added network ports connectivity. I logged into the core switch with the intention of copying config from two ports over to the two ports we were going to use to connect the new switches...

I guess it's a mix of a lack of sleep, trying to hurry, and overall being a bit burnt out. I copied the config, logged into those same two ports I copied the config from and changed the configuration. This took down everything on the 1st floor and the 3rd floor of building 2.

It's always extremely stressful when I do stuff like this. But at the end of the day, you just gotta keep a calm head and think through what you have to do to bring stuff back up. This is my second time doing something like this, but again, I was fortunate that I didn't try to rush through this during the day.
---
... Copied to Clipboard!
Nidhoggr
07/02/19 12:24:52 AM
#2:


What's your job title and how much does it pay and how much education do you have?
... Copied to Clipboard!
CableZL
07/02/19 12:25:47 AM
#3:


Nidhoggr posted...
What's your job title and how much does it pay and how much education do you have?


Network Engineer III
I make $89k right now, but a raise to $105k is in process
I'm a college dropout, but I have Cisco certifications:
- CCNP Routing and Switching
- CCNA Security
- Cisco Meraki Network Associate
---
... Copied to Clipboard!
Nidhoggr
07/02/19 12:28:51 AM
#4:


Impressive. I am about to graduate with a Bachelors in Computer Science but I feel I have learned absolutely nothing and am a bit worried for my future.
... Copied to Clipboard!
CableZL
07/02/19 12:36:19 AM
#5:


Nidhoggr posted...
Impressive. I am about to graduate with a Bachelors in Computer Science but I feel I have learned absolutely nothing and am a bit worried for my future.


I went to UTSA for a year and dropped out.
- I paid $13k for a bunch of classes I didn't care about
- UTSA was raising tuition the following year, so I would have been paying even more for more classes I didn't care about
- They had no band back then, so I was bored
- I was tired of school

It took me many years afterward to figure out what I wanted to do in life, but these Cisco certs have helped me get my career off the ground.
---
... Copied to Clipboard!
Nidhoggr
07/02/19 12:42:04 AM
#6:


Yeah I need to figure out some certifications to get for myself if I want to actually get a job. Just having this degree isn't going to do much for me.
... Copied to Clipboard!
BuckVanHammer
07/02/19 12:58:14 AM
#7:


ive learned the hard way to use strict change controls even when they arent required. device ip checks when logging in to ensure youre in the right device, closing shit out when done, check steps off a change design doc, etc...
---
Minimal, yet feature rich.
... Copied to Clipboard!
CableZL
07/02/19 12:58:54 AM
#8:


BuckVanHammer posted...
ive learned the hard way to use strict change controls even when they arent required. device ip checks when loging in to ensure your in the right device, closing shit out when done, check steps off a change design doc, etc...


Yeah, I should have done all that.
---
... Copied to Clipboard!
SomeLikeItHoth
07/02/19 1:01:25 AM
#9:


Did you get in any trouble?
---
FAM NOW. FAM FOREVER.
... Copied to Clipboard!
Darmik
07/02/19 1:14:11 AM
#10:


Does your job have good hours?
---
Kind Regards,
Darmik
... Copied to Clipboard!
CableZL
07/02/19 1:17:33 AM
#11:


SomeLikeItHoth posted...
Did you get in any trouble?


My boss didn't seem too upset about it. He may have more to say in the morning, though, but we'll see.

Darmik posted...
Does your job have good hours?

My standard hours are monday - friday 8am - 5pm, but I end up working crazy hours for one reason or another. Tonight it was needing to add new switches to the network starting at 9PM. Another night, we may lose an internet connection at 3AM.
---
... Copied to Clipboard!
Darmik
07/02/19 1:25:01 AM
#12:


Does that happen often?

I'm basically sticking with my current job because my work/life balance is awesome and I don't want to lose it lol. But I'm not really progressing much either.
---
Kind Regards,
Darmik
... Copied to Clipboard!
CableZL
07/02/19 1:32:38 AM
#13:


Darmik posted...
Does that happen often?

I'm basically sticking with my current job because my work/life balance is awesome and I don't want to lose it lol. But I'm not really progressing much either.

Nah, this is the second time in 3 years I've done this. It mostly comes from carelessness.
---
... Copied to Clipboard!
MacadamianNut3
07/02/19 1:50:52 AM
#14:


... Copied to Clipboard!
Topic List
Page List: 1