work

what a day :rolleyes:

I’m meant to start at 8:30am but as I’m a good boy I thought I’d get there at 8:00 this morning as network support (NSG) were upgrading a switch in my building this morning and thought it’d be good if I was on hand to act as comms between them and the 1st/2nd line guys who actually take the calls from users.

Anyway I get there to find that NSG hadn’t started yet and the office was filling up quickly. Not good me thinks. There were about 40 people logged on when they pulled the plug. The 1st line guy’s phoned went mad.

30mins later NSG finish the upgrade and all seems well. Just as I’m heading out the door to get some breakfast 2 of our netware 3 servers disappear off the network. As I’m madly trying to get hold of one of the guys who has access to the server room (which is in another building) the phones start ringing again which means I get all the 1st line guys come and ask what’s going on.

I’m explaining that it looks like these 2 servers need rebooting when my phone rings, it’s the server guy calling to tell me that 2 netware 3 servers have crashed. “No sh!t” is my reply, but he goes on to say “and it looks like the proxy server is a bit dodgy” I ask him if he can check out the 2 downed servers and put the phone down.

As soon as the phone is back on the hook the proxy dies, followed by 2 more netware 3 servers. I call the server guy again with about 20 1st line people stood around me to be told that all the servers seem to be up and working but that NSG have done a sly router “upgrade” overnight and not told anyone and it looks like that’s what’s causing the problems.

Fine, I go to call NSG but then, 1 by 1, 5 of our lotus notes servers drop off the network and the larger of the 2 print servers stops all comms.

The time is now 9:30am and all hell has broken loose. By now the company may be loosing money fast. I go to my meeting that I have at 10am every day to report on IT related problems and get a good roasting.

Over the next couple of hours I spend my time trying to stop the server and network guys blaiming each other and refusing to help sort the problems. Eventually by about 12:30pm some Notes services are restored but most of the connections are still up and down more than a wh0r3s pants and my phone is still going mad.

By 1pm NSG report that they’ve stabilised the token ring and IP connections are sorted. Shame that netware 3 works over IPX!! Still with notes working again things are less serious. What we are left with is no printing in 2 head office buildings and a large number of people unable to access shared/personal storage on the effected netware 3 servers.

I decide it’s time for me to leave for the day as I’m about to punch someone :mad:

still lots of new toys have arrived at home for me today :wink:

shudder

This is why you have to suffer the horrors of something called Change Control :eek:

Our change control has gone uber-draconian since a fubar last week left one team refusing to support a critical server since no-one had completed the documentation to officially put it on the network.
Now every time we want to make a configuration change to a network switch (ie change someones port speed, duplex or even name) we have to fill out a change request form, that goes to the change control team, who ignore it for two days, then ignore it for another couple of days after you chase them up, then contact you complaining you havent notified them if you completed the change they never bothered giving you the go-ahead for.
And they wonder why I take the p!ss whenever I fill out change request forms :slight_smile:

Sounds like i had a much more fun than you guys at work last night, 2 displays of funny behavior than i can’t talk about on the forums, however it did provide some mirth on IRC :):slight_smile:

SB

If your network team want to play it that way thats fine.
Just tell them you want this to have access to every network device.

http://www.tripwire.com/products/network_devices/index.cfm

Then you will have an audit trail of what changed and when to aid in backing out changes.