Home

Disaster Recovery ITSS Specific



Mount Marty I.T.S.S. Disaster Recovery Plan Revision C
November 30, 2011
  • Introduction
  • Description
  • Overview
  • Objective
  • Risk and Prevention
  • Disaster Recovery Planning
  • Recover Facility Plan
  • Backup Procedures
  • Disaster Notification List
  • Disaster Recovery Team
  • Disaster Recovery Team Responsibilities
  • Activating the Disaster recovery Plan
  • Damage Assessment
  • Cold Site Preparation
  • Platform recovery
  • Applications recovery
  • Critical Applications
  • Maintaining the plan
  • Introduction
    This document is the disaster recovery plan for the Mount Marty Yankton Campus, Department of Information Technology Support Services. The information present in this plan guides administrative and technical staff in the recovery of computing and network facilities operated by the ITSS department in the event that a disaster destroys all or part of the facilities.

    Description
    The Recovery plan is composed of a number of sections that document resources and procedures to be used in the event that a disaster occurs at the Mount Marty ITSS Server Room located in the building of Bede Hall room 116B at 1105 west 8th Street Yankton South Dakota. Each supported computing platform has a section containing specific recovery procedures. There are also sections that document the personnel that will be needed to perform the recovery tasks and an organizational structure for the recovery process.

    This plan is available through the Mount Marty College web site http://www.mtmc.edu/Computing/Helpdesk.aspx?title=Disaster Recovery ITSS Specific in order to make it more generally available to the community and Mount Marty Staff. But more importantly, a web document format permits it to be published in an online form that can be stored on diskette or CD-ROM media for viewing with a browser in file browse mode. This plan will be updated on a regular basis as changes to the computing and networking systems are made.

    Overview
    Over the years, dependence upon the use of computers in the day-to-day business activities of many organizations has become the norm. Today you can find very powerful computers in every department on campus. These machines are linked together by a sophisticated network that provides communications with other machines across campus and around the world. Vital functions of Mount Marty depend on the availability of this network of computers.

    Consider for a moment the impact of a disaster that prevents the use of the system to process Student Registration, Payroll, Accounting, or any other vital application for weeks. Students and faculty rely upon our systems for instruction and research purposes, all of which are important to the well-being of Mount Marty. Without adequate planning and preparation to deal with such an event, the Mount Marty central computer systems could be unavailable for many weeks.

    Objectives This disaster recovery plan has the following primary objectives:
    1. Present an orderly course of action for restoring critical computing capability to the Mount Marty campus within 14 days of initiation of the plan.
    2. Set criteria for making the decision to recover at a cold site or repair the affected site.
    3. Describe an organizational structure for carrying out the plan.
    4. Provide information concerning personnel that will be required to carry out the plan and the computing expertise required.
    5. Identify the equipment, floor plan, procedures, and other items necessary for the recovery.

    Risks and Prevention
    As important as having a disaster recovery plan is, taking measures to prevent a disaster or to mitigate its effects beforehand is even more important. This portion of the plan reviews the various threats that can lead to a disaster, where our vulnerabilities are, and steps we should take to minimize our risk. The threats covered here are both natural and human-created.

    Fire
    The threat of fire in the administrative building especially in mechanical and or electrical distribution rooms are very real and poses the highest risk factor of all the causes of disaster mentioned here. The building is filled with electrical devices and connections that could overheat or short out and cause a fire.

    Preventive Measures or Recommendations
    Fire Alarms
    The Administrative building Bede hall is equipped with a fire alarm system, however the building is not with smoke detectors. A plan to install smoke detectors is recommended.
    Fire Extinguishers
    Hand-held fire extinguishers are required in visible locations throughout the building. Facilities Staff inspect fire extinguishers on a pm schedule to comply with state law.
    Building Construction
    The Mount Marty Administrative building Bede hall is built primarily of non-combustible materials. The risk to fire can be reduced when new construction is done, or when office furnishings are purchased, to acquire flame resistant products.

    Flood
    The Administrative Services Building is located at a high elevation compared to other structures on the campus and therefore flood is not a concern from external sources however can be a concern since there are restrooms and other plumbing facilities inside the building on upper floors. We do not have any water or moisture detectors in the building.

    Preventive Measures or Recommendations
    It is recommended that some type of water leakage detection be provide for critical areas of concern

    Tornados and High winds
    Since Mount Marty is situated along "Tornado Alley", damage due to high winds or an actual tornado is a very real possibility. A tornado has the potential for causing the most destructive disaster we face.

    Preventive Measures or Recommendations
    While a fire can be as destructive as a tornado, there are very few preventative measures that we can take for tornados. Building construction makes a big difference in the ability of a structure to withstand the forces of high winds. Bede hall is a strong building however it's largest weakness is the many windows in the building.

    Earthquake
    The threat of an earthquake for the Yankton South Dakota area is low, but should not be ignored.

    Preventive Measures or Recommendations
    The preventative measures for an earthquake can be similar to those of a tornado. Building construction makes all the difference in whether the facility will survive or not. Even if the building survives, earthquakes can interrupt power and other utilities for an extended period of time. Standby power generators could be purchased or leased to provide power while commercial utilities are restored.

    Computer Crime
    Computer crime is becoming more of a threat as systems become more complex and access is more highly distributed. With the new networking technologies, more potential for improper access is present than ever before.

    Computer crime usually does not affect hardware in a destructive manner. It may be more insidious, and may often come from within. A disgruntled employee can build viruses or time bombs into applications and systems code. A well-intentioned employee can make coding errors that affect data integrity (not considered a crime, of course, unless the employee deliberately sabotaged programs and data).

    Preventive Measures or Recommendations
    All systems should have security products installed to protect against unauthorized entry. All systems should be protected by passwords, especially those permitting updates to data. All users should be required to change their passwords on a regular basis. All security systems should log invalid attempts to access data, and security administrators should review these logs on a regular basis.

    All systems should be backed up on a periodic basis. Those backups should be stored in an area separate from the original data. Physical security of the data storage area for backups must be implemented. Standards should be established on the number of backup cycles to retain and the length of their retention.

    We must continue to improve security functions on all platforms. Strictly enforce policies and procedures when violations are detected. Regularly let users know the importance of keeping their passwords secret. Let users know how to choose strong passwords that are very difficult to guess.

    Improve network security. Shared wire media, such as Ethernet, are susceptible to sniffing activities, which unscrupulous users may use to capture passwords. Implement stronger security mechanisms over the network, such as one-time passwords, data encryption, and non-shared wire media.

    Disaster Recovery Planning
    The first and most obvious thing to do is to have a plan. The overall plan of which this document is a part is that which Computing Services will use in response to a disaster. The extent to which this plan can be effective, however, depends on disaster recovery plans by other departments and units within Mount Marty

    For instance, if the Administration Building were to be involved in the same disaster as the Administrative Services Building, the functions of the Business Office, or more in particular, Accounts Payable, could be severely affected. Without access to the appropriate procedures, documents, vendor lists, and approval processes, the ITSS Computing Services recovery process could be hampered by delays while Purchasing recovers.

    Every other business unit within Mount Marty should develop a plan on how they will conduct business, both in the event of a disaster in their own building or a disaster at Computing Services that removes their access to data for a period of time. Those business units need means to function while the computers and networks are down, plus they need a plan to synchronize the data that is restored on the central computers with the current state of affairs. For example, if the Human Resources Office is able to produce a payroll while the central computers are down, that payroll data will have to be re-entered into the central computers when they return to service. Having a means of tracking all expenditures such as payroll while the central computers are down is extremely important.

    Recovery Facility
    If a central facility operated by ITSS department is destroyed in a disaster, repair or rebuilding of that facility may take an extended period of time. In the interim it will be necessary to restore computer and network services at an alternate site.

    Hot Site
    This is probably the most expensive option for being prepared for a disaster, and is typically most appropriate for very large organizations. A separate computer facility, possibly even located in a different city, can be built, complete with computers and other facilities ready to cut in on a moment's notice in the event the primary facility goes offline. The two facilities must be joined by high speed communications lines so that users at the primary campus can continue to access the computers from their offices and classrooms.
    Cold Site
    A cold recovery site is an area physically separate from the primary site where space has been identified for use as the temporary home for the computer and network systems while the primary site is being repaired. There are varying degrees of "coldness", ranging from an unfinished basement all the way to space where the necessary raised flooring, electrical hookups, and cooling capacity have already been installed, just waiting for the computers to arrive.

    Mount Marty has chosen to use the cold site approach for this disaster recovery plan. The ITSS server infrastructure and Critical administrative offices will reside in a existing permanent or non permanent structure that is a non occupied 800 square foot of open space referred to here on as the (Off Site Space). This space will should have adequate resources to install hardware, sufficient power requirements and environmental cooling will be an issue because of the large amount of heat generated by computer, server, networking hardware.

    Backup Procedures
    The ITSS Department has been charged with the responsibility for ensuring all campus missions critical data, whether it is stored on the mainframe or on a server, is secure and properly backed up. It is imperative that backups for mission critical data are current, in case of hardware or software failures, virus attacks or successful hacking attempts. Failure to do this properly has the potential to be extremely costly. For this reason, Mount Marty College has implemented a well-planned server backup strategy. Backups are performed nightly to account for any data that changed during the day. Four weeks worth of backups are kept and appended to tape monthly.

    We are currently using a combination of Symantec Veritas Backup Exec 11d for our backup and recovery software needs.

    Backup Policy
    Differential backups are completed every night for files that have changed.
    Full backups are performed monthly
    All backups are verified
    If a file is deleted then only 1 version of the deleted file (most recent) is kept for 30 days.
    Archiving to tape archive capacity is to be determined.
    Open files are not backed up.

    ITSS Server Support Service

    The ITSS department will monitor verify backups are completed successfully
    Will notify the POC (point of contact) of any problems with their server’s backup
    Ensures there is a copy of a system’s data offsite at all times for disaster recovery
    Critical school systems will receive priority support
    Admin privileges on the servers for access of the backup system and troubleshooting.

    Disaster Notification List
    ITSS Department

    Edward Koster
    Cheif Information Office
    edward.kosterr@mtmc.edu
    Office Location: Bede Hall -119
    Phone Number: (605)-668-1367
    Jay Scoblic
    Information Technology Support Services Administrator
    jscoblic@mtmc.edu
    Office Location: Bede Hall -116C
    Phone Number: (605)-668-1359 or 605-661-9885
    Paul Lammers
    Director of ITSS
    plammers@mtmc.edu
    Office Location: Bede Hall -116D
    Phone Number: (605)-668-1544 or 605-660-0412
     

    Emergency Numbers
    General College Number 668-1011
    Emergency 9-911
    Fire 668-5228
    Police 668-5210
    Sheriff 668-3567
    Ambulance 668-9033
    Avera Sacred Heart Hospital 668-8000
    Security (8 p.m. to 6 a.m.) 661-9883
    Sacred Heart Monastery 668-6000

    Key Venders

    Anixter
    4630 South 132nd Street
    Omaha NE 68135
    Phone 800-228-0062
    john.chamberlain@anixter.com
    Foss Communications
    Rick Foss
    Phone 605-336-9550
    Cell 605-941-3266
    Jason Foss
    CELL 605-940-5918
    John Pinkleman
    59150 558 Ave
    Fordyce, Ne 68736
    Phone 402-357-3385
    Cell 605-661-9916
    Loren Mendel
    Account Executive
    High Plains Technology
    1500 S. Sycamore Ave  Suite 200
    Sioux Falls, SD 57110
    605.782.2180
    605.359.7051 Cell
    605.977.9040 Fax
    lmendel@hplains.com
    Ted Neugent
    Lead Technical Analyst (ST/CORE)
    Datatel, Inc.
    Voice: 703-259-2839
    Fax: 703-449-6900
    Email: mzm@datatel.com
    John L Bettini
    Customer Service Manager
    Soft Docs, Inc.
    920 Hemlock Dr.
    Columbia, SC 29201
    Phone (803) 695-6044 ext 111
    Fax (803) 727-1070

    Disaster Recovery Team
    The discussion that follows identifies those skills that are needed by members of the Recovery Management Team. If these positions are filled with qualified individuals, then the odds for a timely and successful recovery are very high.

    Recovery Manager
    This individual needs to be a skilled manager/administrator who is accustomed to dealing with pressure situations. He should have a broad knowledge of the hardware and software in use at the site. He should be a "problem solver" as there will be many problems arise that have not been anticipated in advance. He must be able to delegate responsibility to others. He must also have signature authority to expend funds as a part of the disaster recovery process. The current Director of Computing Services is the first choice for the Recovery Manager.
    Facilities Coordinator
    This individual needs some of the same skills as the Recovery Manager. However, he also needs to be familiar with the process of getting construction work scheduled and completed on time. He should be able to understand and oversee the setup of the electrical, environmental, and communications requirements of a data center.
    Technical Coordinator
    This individual needs to be highly skilled in a number of areas. He must have a strong background in the setup and interfacing of as many of the platforms in use as possible. He needs to be able to communicate easily with vendor technical represenatives and engineers concerning installation options, performance issues, problem resolution, and a myriad of other things. He must also be able to schedule and manage people.
    Administrative Coordinator
    This individual needs to be skilled in the business operations of Mount Marty. This person should be well acquainted with the day-to-day operations of the college departments. This person should also be a "people person" who can deal with employees and their families during hard times.
    Network Coordinator
    This individual needs to be skilled in the area of network design and maintenance. He should be trained in diagnosing and correcting network outages and in connecting and debugging new additions to an existing network.
    Applications Coordinator
    First choice for this individual would be someone from the existing application support group. The person should have exposure to a cross section of the currently used applications. The most critical areas are Payroll, Accounting, and Student Records. If no one from the current staff is available, the most important technical skills are: a knowledge of Datatel, RazersEdge and other support software utilities and storage techniques, The person will need to use available tools to ascertain the status of files and data base objects and be prepared to restore later versions from backups if required. He will also need to interface with users to verify that applications are functioning as expected or analyze and develop solutions to problems that arise.
    Computer Operations Coordinator
    This individual needs to be skilled in the day-to-day operations of workstations, operating systems and application software, as well as the knowledge and skills to recreate or export and import critical data to client applications. This person will also be responsible for setting up a limited help desk function that will provide information to callers on status and availability of systems, how to access systems that are in a temporary setting, or any new procedures that users need for submitting their production applications for processing.

    The following table contains a sample list of the people currently employed who could fill the positions on the Recovery Management Team. Alternates are listed, but there are other qualified individuals who could step in should any of these persons not be available.

    Sample Recovery Management Team Roster
    Position Primary Alternates



    Recovery Manager Edward Kosterr  

    Facilities Coordinator Steve Hermanson  

    Technical Coordinator Paul Lammers  

    Administrative Coordinator Jay Scoblic  

    Network Coordinator Paul Lammers  

    Applications Coordinator James Bacorn  

    Computer Operations Coordinator James bacorn  

    Disaster Recovery Team Responsibilities
    As the recovery process gets underway, it is imperative that each of the recovery teams remain in close communication and strive to work together to complete the recovery as expediently as possible. The following section provides a brief description of the responsibilities for each team.

    Recovery Management Team
    The Recovery Management Team is responsible for the coordination of the entire project. It is composed of seven skilled people:

    1. Recovery Manager
    2. Facilities Coordinator
    3. Technical Coordinator
    4. Administrative Coordinator
    5. Network Coordinator
    6. Applications Coordinator
    7. Computer Operations Coordinator

    The Recovery Manager is the leader of the Recovery Management Team and has the final authority regarding decisions during the recovery process. Each of the remaining individuals will be the leader of a specialized team that will address a portion of the recovery tasks. As the recovery process gets underway, there will likely be areas of overlap between teams and close communication will be required. The Recovery Management Team will have regular meetings scheduled to provide for communication between team coordinators.

    Each coordinator should schedule a meeting for members of his team well in advance of their first planned activities. A first-meeting agenda might include:

    1. Reviewing the current status of the recovery operation.
    2. Emphasizing what the team's responsibilities are
    3. Making sure that members are aware of any changes to the original recovery plan
    4. Assigning tasks to individual team members
    5. Setting up time and location for future team meetings

    Damage Assessment Team
    The Damage Assessment Team will be led by the Technical Coordinator. He will be responsible for selecting the other team members. Likely choices would be a member(s) from Physical Plant, Operations, Network Services, Campus Telephone Services, and Technical Services. This team will not be responsible for a detailed damage assessment for insurance purposes. The primary thrust for this team is to do two things:

    1. Provide information for the Recovery Management Team to be able to make the choice of the recovery site.
    2. Provide an assessment of the salvageability of major hardware components.

    Based on this assessment the Recovery Management Team can begin the process of acquiring replacement equipment for the recovery.

    Facility Recovery Team
    The Facility Recovery Team will be led by the Facilities Coordinator. He will be responsible for selecting the other team members. Likely choices would be member(s) from Operations, Network Services, Physical Plant, Cold Site Building Representative, and Technical Services.

    This team will be responsible for the details of preparing the recovery site to accomodate the hardware, supplies, and personnel necessary for recovery. Detailed layouts and instructions for the Cold Site preparation are included in the recovery plan.

    This team will also be responsible for oversight of the activities for the repair and/or rebuilding of the primary site (the Administrative Services Building). It is anticipated that the major responsibility for this will lie within Physical Plant and contractors. However, this team must oversee these operations to ensure that the facility is repaired to properly support the operation of mainframe and networking equipment per the original design of the primary site.

    Network Recovery Team
    The Network Recovery Team will be led by the Network Coordinator. He will be responsible for selecting the other team members. Likely choices would be member(s) from Network Services, Technical Services, User Services, and Physical Plant. It may also be helpful to have the building and/or network manager for the Cold Site building be a part of this team should it be necessary to use the Cold Site.

    This team will be responsible for overseeing the restoration of the campus network and all network connections necessary at the recovery site. It is entirely possible in certain disaster situations that the Network Recovery Team may be the only team convened as a result of a campus disaster. For instance, should a fire occur at the Whitby Building and destroy fiber optic connections and network equipment, this team will be charged with the recovery of operations out of that building or in another building on campus in the most expedient manner.

    Because there is such a high degree of reliance on the campus network, for instruction, research, and administrative purposes, very high emphasis must be placed on restoring the network as quickly as possible.

    Platform Recovery Team
    The Platform Recovery Team will be led by the Technical Coordinator. He will be responsible for selecting the other members of the team, each of which will be the leader in charge of restoring one or more of the computer platforms described in this plan.

    Each team member may recruit others to assist in the technical and detailed work of the recovery. They are responsible for communicating needs and status information to other recovery teams and to coordinate restoration operations between parties working on different computer platforms.

    Each platform recovery group will follow this general plan of action:

    1. Review damage assessment.
    2. Determine which hardware, software, and supplies will be needed to start the restoration of a particular system.
    3. Communicate list of components to be purchased and their specifications to the Administrative Support Team.
    4. Review the recovery steps documented in this plan and make any changes necessary to fit the situations present at the moment.
    5. When hardware begins to arrive, work with vendor representatives to install the equipment.
    6. When all components are assembled, begin the steps to restore the operating system(s) and other data from the off-site backup tapes.
    7. Attempt to recreate status of all systems up to the point of the disaster if possible. If not, the system is handed off to the Application Recovery Team.

    Application Recovery Team
    The Application Recovery Team will be led by the Application Coordinator. He will be responsible for selecting the other team members. This team will be responsible for conducting activities leading up to the approval and acceptance of application systems for production use. In general, this team's activities will begin after the Platform Recovery Team has completed work on the target platform. Some of the team members may in fact be from the platform recovery teams.

    Some of the anticipated tasks include:

    1. Analysis of need for additional recovery activities such as data base restores or individual file restores
    2. Developing programs/procedures to address specific problems
    3. Interfacing with application users to test applications

    Computer Operations Team
    The Computer Operations Team will be led by the Computer Operations Coordinator. He will be responsible for selecting the other team members. This team will provide three major functions:

    1. Man the Help Desk to provide phone assistance and status information to end-users.
    2. Provide operator staffing for the computer systems at the Cold Site.
    3. Provide Production/Control function for establishing production job schedules after systems and applications are restored.

    Administrative Support Team
    The Administrative Support Team will be led by the Administrative Coordinator. He will be responsible for selecting the other team members. This team will provide administrative support to the other recovery teams as well as support to employees and their families. One of the most important functions that this team can provide is to take the burden of administrative details so that the engineers and technicians who are responsible for systems recovery can concentrate on their recovery work.

    One member of this team should be designated as Family Contact. This person will be available throughout the recovery process to provide assistance to employee family members.

    One member of this team should be a designated representative of the Accounts Payable Office. This person will the liaison to the Business Office for the purpose of expediting all emergency purchases.

    Some of the anticipated team tasks include:

    1. Provide support for executing acquisition paperwork.
    2. Assist with the detailed damage assessment and insurance procedures.
    3. Determine the status of staff working at the time of the disaster.
    4. Provide counseling services for staff or family members having emotional problems resulting from the disaster.
    5. Assist the individual Team Coordinators in locating potential team members.
    6. Coordinate food and sleeping arrangements of recovery staff as necessary.
    7. Provide support to track time and expenses related to the disaster.
    8. Provide delivery and transportation services to the Cold Site or other locations as required.
    9. Provide public relations support (this function may be provided by University Relations).
    10. Assist in contracting with outside parties for work to be done in the recovery process (such as the installation of equipment, or consulting assistance for the installation or recovery of software systems).

    Activating the Disaster Recovery Plan
    The Recovery Manager sets the plan into motion. Early steps to take are as follows:

    The Recovery Manager should retrieve the Disaster Recovery Lock Box located in Scholastica Mechanical Room and open it to obtain an up-to-date copy of the Disaster Recovery Plan. Enclosed in the lock box will be the Following

    This Disaster Recovery Plan
    Network Drawings
    Equipment Drawings and Specifications
    Server Drawings and Configurations
    Backup Documentation
    Backup Media

    All of the elements listed above will be printed form in the box as well on computer media (diskette or CD-ROM). Copies of the plan should be made and handed out at the first meeting of the Recovery Management Team. The Recovery Manager is responsible for the remaining contents of the Lock Box, which should probably be relocked if possible.

    The Recovery Manager is to appoint the remaining members of the Recovery Management Team. This should be done in consultation with surviving members of the Computing Services staff and Physical Plant management, and with upper university administration approval. The Recovery Manager's decision about who sits on the Recovery Management Team is final, however.

    The Recovery Manager is to call a meeting of the Recovery Management Team at the Recovery Control Center or a designated alternate site. The following agenda is suggested for this meeting:

    1. Each member of the team is to review the status of their respective areas of responsibility.
    2. After this review, the Recovery Manager makes the final decision about where to do the recovery. Tthe Recovery Manager is to search for the appropriate (Off Site Space) ASAP and notify the City of Yankton that Mount Marty is in the state of emergency if not already notified. . Pre-arrangements for this facility or other should have been already made available.
    3. The Recovery Manager briefly reviews the Disaster Recovery Plan with the team.
    4. Any adjustments to the Disaster Recovery Plan to accommodate special circumstances are to be discussed and decided upon.
    5. Each member of the team is charged with fulfilling his/her respective role in the recovery and to begin work as scheduled in the Plan.
    6. Each member of the team is to review the makeup of their respective recovery teams. If individuals key to one of the recovery teams is unavailable, the Recovery Manager is to assist in locating others who have the skills and experience necessary, including locating outside help from other area computer centers or vendors.
    7. The next meeting of the Recovery Management Team is scheduled. It is suggested that the team meet at least once each day for the first week of the recovery process.

    The Recovery Management Team members are to immediately start the process of contacting the people who will sit on their respective recovery teams and call meetings to set in motion their part of the recovery.

    The Facilities coordinator is responsible for immediately setting up a place where the Recovery Management Team is to meet. This location will be determined at this first meeting. The area should be supplied with the following equipment.

    1. 3 Office desks and chairs
    2. 3 Telephones
    3. 3 IBM-compatible personal computers (Pentium, 1GB memory, 50GB hard drive, LCD Monitor ethernet card, and CDROM drive, keyboard, mouse
    4. 1 Hewlett-Packard LaserJet printer
    5. 1 Fax machine
    6. 1 Copier
    7. 1 Cisco Switch
    8. 3 Network cables

    Mobile communications will be important during the early phases of the recovery process. This need can be satisfied through the use of cellular telephones and/or two-way radios.

    Damage Assesment
    This damage assessment is a preliminary one intended to establish the extent of damage to critical hardware and the facility that houses it. The primary goal is to determine where the recovery should take place and what hardware must be ordered immediately.

    Team members should be liberal in their estimate of the time required to repair or replace a damaged resource. Take into consideration cases where one repair cannot begin until another step is completed. Estimates of repair time should include ordering, shipping , installation, and testing time.

    In considering the hardware items, consider first the equipment lists provided in the recovery sections for each platform. These lists were constructed primarily for recovery at the cold site so they consist of the critical components necessary to recovery. You will need to seperate items into two groups. One group will be composed of items that are missing or destroyed. The second will be those that are considered salvageable. These "salvageable" items will have to be evaluated by hardware engineers and repaired as necessary. Based on input from this process, the Recovery Management team can begin the process of acquiring replacements.

    With respect to the facility, evaluation of damage to the structure, electrical system, air conditioning, and building network should be conducted. If estimates from this process indicate that recovery at the original site will require more than 14 days, migration to the cold site is recommended.

    Cold Site Preperation
    This portion of the document focuses on the preparation of the designated Cold Site for the recovery of primary computing and network facilities after a disaster has occurred. If the Recovery Management Team accepts the use of the (Off Site Space) for the site recovery after the disaster, some work may be required to convert the space from its present use to house the computer systems, network equipment and disaster recovery team personnel.

    Quick Review of Site Preparation Work
    The Cold Site has only had minimal advanced preparations, so much work is to be done in the early stages of the recovery process to make the site ready. Here is a quick review of the facilities and work that must be done.

    • The (Off Site Space) site has Adequate power capacity is available within the building, We are curtain this facility is already equiped with three phase power which is more than adquate power however it more than likely is disconnected from the main power grid. The facilities director will need to insure this this power service is restored and a temporary 200 amp drop is supplies where it would be decide where the server racks or servers would be deplayed. These consist of breaker panels and conduit back to the main power source within the building two 110vac circuits with 100 amp breakers.
    • The site does not have power conditioning equipment, such as an uninterruptable power supply (UPS) or motor generator. The Plan does not call for either of these, which may put the equipment installed at risk of power interruptions, a risk we must accept due to the temporary conditions of the Cold Site.
    • The facility has only standard building environmentals. No additional envriromentals would be necessary
    • The site does not have a raised floor such as found in major computer rooms and is not required.
    • The Cold Site designated as (Off Site Space) may not have any electronic security and this may need some consideration.
    • Terminations to the fiber optic cabling for the campus backbone network is located within the Hot Site and is not available to the cold site. A VPN or Cable modem will be required to connect the (Off Site Space) to the internet and additional equipment will also needed to reconnect any critical facilities at the main campus. Network reconnection of enviromental controls and emergency services may need to be considered.

    Platform Recovery
    Platform recovery will be the most intense part of the whole recovery process. Included in the lock box will be detailed information in regards to what servers are required for basic operations. And what those configurations will be. This hardware will need to be purchase along with the LAN support equipment and WAN interfaces. Once the equipment has been purchased and recieved the build process can begin, system states, applications, and databases can then be restored.

    Application Recovery
    Once the platform system software and subsystems are operating correctly, the task of preparing the remaining end-user applications can begin. Each platform will have a unique recovery road to follow. In some cases, there may be very little to do except for general testing. In other cases, considerable analysis and data syncronization work will likely be required.

    The Applications Recovery Team will be responsible for carrying out this phase of the recovery. Each application area will require a review. This review should be conducted by an analyst familiar with the application while working closely with an application user represenative.

    Items to be considered should include:

    • Review of the user department Disaster Recovery Plan with special attention to any "interim" procedures that have been required in the time period since the disaster event occurred.
    • Review of the application documentation concerning file and database recovery.
    • Review the status of files and databases after the general platform recovery processing is complete.
    • Identify any changes to bring the application to a ready for production status.
    • Identify any areas where the application must be synchronized with other applications and coordinate with those application areas.
    • Identify and review application outputs to certify the application ready for production use.

    Critical Applications
    Mount Marty has identified the payroll application as a critical application. This means that delaying the processing of this application could cause much hardship on faculty, staff, students, and others that depend on it. Other applications that may be handled as critical or given very high priority in recovery are the Purchasing application and the Web server application since they will be needed during recovery

    Maintaining the Plan
    Having a disaster recovery plan is critical. But the plan will rapidly become obsolete if a workable procedure for maintaining the plan is not also developed and implemented. It is inevitable in the changing environment of the computer industry that this disaster recovery plan will become outdated and unusable unless someone keeps it up to date. Changes that will likely affect the plan fall into several categories:

    1. Hardware changes
    2. Software changes
    3. Facility changes
    4. Procedural changes
    5. Personnel changes

    As changes occur in any of the areas mentioned above, The ITSS department will determine if changes to the plan are necessary. This decision will require that the managers be familiar with the plan in some detail. A document referencing common changes that will require plan maintenance will be made available and updated when required.

    Changes that affect the platform recovery portions of the plan will be made by the staff in the affected area. After the changes have been made, Mount Marty administration will be advised that the updated documents are available. They will incorporate the changes into the body of the plan and distribute as required.