Volkswagen Mark IV Forum banner
1 - 20 of 23 Posts

·
Registered
Joined
·
1,320 Posts
Discussion Starter · #1 ·
First off I'm not an IT person, we have a company outsourcing our IT and I somehow got lumbered with overseeing it.

We have 3 servers - 1 webserver purely for an E-commerce project, 1 print/email server and a "general" server.

The print server "controls" the other 2 servers apparently - and today the hard drive decided to pack up. So we have no usable servers at all.

The IT company are coming out first thing tomorrow, but the chap on the phone said something about no fault tolerance being in place on the duff server, and thought that this could be a very bad thing indeed...........

Can anyone give me an idea of how bad this could be - I really have to have at least the general server running somehow straight away, no e-mails for a while wont be too bad.
 

·
Registered
Joined
·
148 Posts
Ok you havnt mentioned what operating the system the server are running but lets assume windows server....

I would assume that the print server controlling the others may mean it could be a domain controller.

If you have backups of this server including the system state, and full windows folders, it should be a case of just reinstalling the os and then restoring Active Directory roles etc etc

If you havnt got backups then bad news..... everything may well be lost.....

Very basic outline above but as mentioned alot depends on operating system and backup scenarios etc etc

Hope you get it sorted any questions frop us a pm

cheers Andrew
 

·
Registered
Joined
·
4,528 Posts
As said above it really depends on what OS it is and what email server,
heres what to ask, post replys here there is enough IT people to tell
you what you need to do but will need these answers

  • What is the OS (NT,2000, Server 2003)
  • is there any backups of anything?
  • Why is there no Backups? off site or otherwise
  • Have any of the servers got mirrored hard drives?, if not why not where are the original spec sheets for the project?
  • If it is an NT setup (I doubt it these days) ask them where is the BDC (Backup domain controller)
  • if the hard drive is a total loss consider how long it would take
    to rebuild the domain and accounts as to getting a proffesional
    recovery done on the hard drive, its expensive but usually worth it?
Lalstly sack the Monkey that made a single hard drived server a Domain Controller............
 

·
Registered
Joined
·
2,990 Posts
You should consider adding some basic redundancy to your servers to prevent a re-occurence, i.e:

  • 2 x network cards
  • 2 x power supplies
  • 2 x mirrored disks for your o/s
  • RAID 5 array for your data storage with a hot swap spare.

With a config like this you could lose a disk, network card or power supply and the server would still stay up and running.
 

·
Registered
Joined
·
4,528 Posts
You should consider adding some basic redundancy to your servers to prevent a re-occurence, i.e:

  • 2 x network cards
  • 2 x power supplies
  • 2 x mirrored disks for your o/s
  • RAID 5 array for your data storage with a hot swap spare.

With a config like this you could lose a disk, network card or power supply and the server would still stay up and running.
i would add to what Sem said though that monitoring software is needed
to alert you when a component fails, as I once saw a server with a
failed PSU it turned out the spare had failed months ago and
noone had noticed .....
 

·
Registered
Joined
·
1,320 Posts
Discussion Starter · #6 ·
Thanks for the info guys.

In answer to your questions, plus maybe some more useful info;

I think the duff server is on Windows 2000 server. The IT guy was asking about mirrored disks but I dont think that we have this on that server. However the HD that is dead is the C drive of that server, we also have a D drive that is called exchange, presumably this contains our Email stuff.

We back up all 3 servers from one of the other servers using Veritas - so I believe that lost data wont be an issue - its just when the IT guy said that there is no fault tolerance in place that he came across as "oh sht".

My MD is on holiday skiing - I've already had an earbashing yesterday, hence the desire to get at least some of it working within minutes of this IT chap turning up!!
 

·
Registered
Joined
·
1,320 Posts
Discussion Starter · #8 ·
LOL yes thats the first thing I tried - but it wont start up at all - blue screen - error message
 

·
Registered
Joined
·
586 Posts
I think the duff server is on Windows 2000 server. The IT guy was asking about mirrored disks but I dont think that we have this on that server. However the HD that is dead is the C drive of that server, we also have a D drive that is called exchange, presumably this contains our Email stuff.
It may still just be the one hard drive, just different partitions. You need to find out how many hdd's you have and how they are set up.

We back up all 3 servers from one of the other servers using Veritas - so I believe that lost data wont be an issue - its just when the IT guy said that there is no fault tolerance in place that he came across as "oh sht".
You need to get a replacement hdd and restore your data from backup.

The IT company are coming out first thing tomorrow, but the chap on the phone said something about no fault tolerance being in place on the duff server, and thought that this could be a very bad thing indeed...........
If you outsource your IT then that company should really be dealing with fault tolerance and issues like this. Depending on the contract you have with them tomorrow morning isn't really good enough if you have no useable servers !
 

·
Registered
Joined
·
377 Posts
What I do is always have a server in standby.

<o:p></o:p>The server is not running, but it is built and working all
it needs is the most recent image from
what ever server has gone down, all of our servers are ghosted every 24 hours,
and all of the data is backed up to DVD?s and 2 onsite NAS systems, and one of
site system at my house.

<o:p></o:p>I have never been a great lover of having raid5 etc on
systems as I have seen raid 5 fail on so many occasions.

<o:p></o:p>If you use raid, and we do use raid mirroring and always use
a separate physical disk for the windows partition. If a window becomes corrupt
you can format it with out risking your data.

<o:p></o:p>The amount of customers I have been out to that have 2
servers with 2 of everything in them and when they go belly up they all seam to
go.
 

·
Registered
Joined
·
472 Posts
accounts departments always get lumbered with IT it seems.

It seems a bit daft that the company you are outsourcing your IT to don't know what you've got? i suppose they will do after today!

As above with the recommendation from Sem and Andy about the spec of the server Raid 5 with SCSI Hard Drives is what you need.

Also make sure that you have an APC UPS protecting everything that is plugged into the server.

Good luck
 

·
Registered
Joined
·
4,528 Posts
What I do is always have a server in standby.

<o:p></o:p>The server is not running, but it is built and working all
it needs is the most recent image from
what ever server has gone down, all of our servers are ghosted every 24 hours,
and all of the data is backed up to DVD?s and 2 onsite NAS systems, and one of
site system at my house.

<o:p></o:p>I have never been a great lover of having raid5 etc on
systems as I have seen raid 5 fail on so many occasions.

<o:p></o:p>
This is good advice... I too am not a lover of Raid 5 and in this day and age of very high capacity disks it is almost unnescacry in small server setups, aswell with what is said above hardware is very very cheap these days so having a fully hot swappable server is very realistic.

Even at home on my desktop I am runnig a basic raid configuration with 2 400 gb drives and serial ATA mirror..... for fault tolerance I brake the mirror every so often and rebuild it with an 80gb spare disk... then send that to my dads.

If i was local I would look at it for you, I wouldnt be taking an earbashing for it ... if anything it is either the IT companys fault or it is your companies fault for being mean with the IT budget and not speccing in fault tolerance.

If you have Veritas backup you should be OK..... for data recovery but dont expect anything to work today I would expect a recovery of this nature to take a day ......
 

·
Registered
Joined
·
4,528 Posts
.

Also make sure that you have an APC UPS protecting everything that is plugged into the server.

Good luck
very good advice also... power is major player in things like this.

If you have branded servers (DELL HP COmpaq) they usually have utilities to tell you when components are failing, this should prevent things like this as it alows proactive maintenance rather than reactive... IE you could have had the drive replaced before the catastrophic failure.
 

·
Registered
Joined
·
6,881 Posts
From what I'm reading (all good info above people), you have just lost your domain and network setup.

If there is no domain, the network won't work. So you will have lost stuff like your personal windows profiles and documents within them (should always keep documents elsewhere anyway). The other 2 servers are fine, it's just they have no way of communicating with the network. Your e-mails should be fine.

All that needs to be done is the domain set up again and the network configured to how it was before.

This would however be a hell of a lot easier if you had a backup. Backup's are vital. You don't have to spend a fortune to get backups either, a simple XCopy to another machine on a seperate hard drive would suffice (not that I'm reccomending just this route, but better than nothing), as they could take that hard drive out of the machine, stick it in place of the broken hard drive and it should all boot up (someone correct me if I'm wrong).

So it's not a total loss, though annoying that you havent got any backup placs whatsoever, hard drives WILL fail, doesnt matter if they are in a server or a workstation.
 

·
Registered
Joined
·
4,528 Posts
From what I'm reading (all good info above people), you have just lost your domain and network setup.

If there is no domain, the network won't work. So you will have lost stuff like your personal windows profiles and documents within them (should always keep documents elsewhere anyway). The other 2 servers are fine, it's just they have no way of communicating with the network. Your e-mails should be fine.

All that needs to be done is the domain set up again and the network configured to how it was before.

This would however be a hell of a lot easier if you had a backup. Backup's are vital. You don't have to spend a fortune to get backups either, a simple XCopy to another machine on a seperate hard drive would suffice (not that I'm reccomending just this route, but better than nothing), as they could take that hard drive out of the machine, stick it in place of the broken hard drive and it should all boot up (someone correct me if I'm wrong).

So it's not a total loss, though annoying that you havent got any backup placs whatsoever, hard drives WILL fail, doesnt matter if they are in a server or a workstation.
True, but if it is a true Active directory domain with exchange as the messaging software its a bit more difficult than just rebuilding the domain, when you rebuild it exchange will throw a micky and also exchange does not like to be copied from on domain to another........ it is not as simple as copying user profiles and document foldeers over... exchange is a nasty product when it comes restores.... there are so many factors that can effect it.... and if the messgae store is big it can take hours fo it to check itself after a failure like this.

Xcopy is great for things like file stores, but cannot cope with an active diretcory schema or exchange message store or the associated security catalogues etc etc.... for this you need good backup software and preferably a tape drive (DLT 40/80's) and the routine should be every day with weekend and monthly and yearly stored off site somewhere, it would probably cost ?1000 for the hardware but doing it this way is fool proof and should be part of disater recovery paln your company should have..... what oif your building was hit like buncefield or manchester by the IRA .... you need full comprehensive off site backups.
 

·
Registered
Joined
·
6,881 Posts
I know it's not quite as simple as I'm saying, but have never worked with exchange in my life.

Andy, why are you doing the job you do? You seem very knowledgeable, more so than me when it comes to networks. I do my own job within the NHS that requires other skills, but just wondering why your not in IT?
 

·
Registered
Joined
·
4,528 Posts
I spent 9 years working in IT and worked with Solaris, NT and lastly building Outsourced Exchange sytems (Monster sytems with clusterd exchange stores with 5000 users and hitachi sans with 100 terabytes of disks).... So yes I know my stuff but I jacked it in 18 months ago as I couldnt get anywhere without formal qualifications..... I was often the best engineer a department had and was told so by many people and have very good abiliites but due to my lack of certifications I was always knocked back by HR departments.

I tried to get my certifications, but got sick of failing exams (for reasons I will not post here I struggle immensely with formal examinations, I even failed my driving test 5 times)

I could have gone self employed doing small business stuff, but configuring desktops and fixing printers is not my thing.... I get bored easily and like playing with expensive gear in the basement LOL

its a trade i still like and i try to keep myself upto date-ish however at the moment the market is full of well qualified graduates the know jack when it comes to recovering exchange after a total failure............... but yet they would still come and get paid 40% more than me
 

·
Registered
Joined
·
6,881 Posts
I spent 9 years working in IT and worked with Solaris, NT and lastly building Outsourced Exchange sytems (Monster sytems with clusterd exchange stores with 5000 users and hitachi sans with 100 terabytes of disks).... So yes I know my stuff but I jacked it in 18 months ago as I couldnt get anywhere without formal qualifications..... I was often the best engineer a department had and was told so by many people and have very good abiliites but due to my lack of certifications I was always knocked back by HR departments.

I tried to get my certifications, but got sick of failing exams (for reasons I will not post here I struggle immensely with formal examinations, I even failed my driving test 5 times)

I could have gone self employed doing small business stuff, but configuring desktops and fixing printers is not my thing.... I get bored easily and like playing with expensive gear in the basement LOL

its a trade i still like and i try to keep myself upto date-ish however at the moment the market is full of well qualified graduates the know jack when it comes to recovering exchange after a total failure............... but yet they would still come and get paid 40% more than me
Ahhh, same as me, no qualifications but a niche market. I do alright, nothing spectacular and keeps me living. I don't expect to make a fortune in IT but am really happy in my job which means a lot I think.

Lots of respect when you post though mate, you certainly know your stuff!!
 

·
Registered
Joined
·
1,320 Posts
Discussion Starter · #19 ·
UPDATE

Still not sorted [6]

The C drive of the server (domain controller) is dead - IT guy plugged
in a new HD and tried to put the OS onto it - but the server wouldn't
recognize the HD.

Several hours later he got it to recognize it, we now have a HD working
on that server, he is currently restoring the backup for that drive
onto the new drive so we appear to nearly be there.

What a f in nightmare
 

·
Registered
Joined
·
4,528 Posts
Now do you see why a good disaster recovery plan is needed, imagine if
there had been a fire and all servers and backups where gone....... how
many staff were unable to work to day 10-20-30 all at ?100 to ?300 a
day its a false economy not to backup and have a fault tolerant network.

This is something that should go straight to the top of directorship......... and explained in a simplistic manner.

many of my colleauges in my last place were the guys that cleaned up
after the manchester bombings so I learnt a hell of alot of them.

In it Anything is possible... but once it happens the consequences are massive
 
1 - 20 of 23 Posts
Top