Core Services Cumulative Monthly Report, January 2008

Previous monthly reports

• Content/Web infrastructure
• Develop/support SDR hosting environment.
• Jan: Feeman, Korner, and Snavely continued software installation and configuration, including a new Apache build, numerous adaptations to GROOVE, MySQL (with replication), TSM, and usage statistics.
• Develop/support enhancements to Library Web Services hosting environment.
• Finish transition to CoSign authentication.
• Replace web load-balancing system.
• Roll out new web-based authorization management system.
• Begin migration of Library Web Services to networked storage.
• Begin migration of DLPS development services to networked storage.
• Continue work with ITCS on Shibboleth origin pilot.
• Migrate all web service to Apache 2.
• Repository/Storage
• Finish migration to new Pillar Axiom SAN in ALDC.
• Jan: Snavely continued to monitor performance, and contacted Pillar about approaches for adding redundancy to the current storage configuration.
• Implement snapshot-based database backups.
• Jan: Korner began testing snapshot features and mechanisms for creating and mounting snapshots from the operating system, and sent a query to Pillar for additional information.
• Install and configure new Isilon NAS systems and migrate SDR services.
• Jan: Korner upgraded the software on both Isilon clusters. Snavely finished the first pass of batch synchronization, and kept it running on a regular schedule to keep files in synchronization as items were added to the repository. Korner and Snavely investigated a problem of excessive space consumption which turned out to be related to the large number of very small files in the repository. Consulting with DLPS staff, an alternative file layout using a single ZIP file for each item was developed and planning began for using the new layout. Korner did initial experimentation with synchronizing between Isilon clusters, but postponed the work pending the change to the ZIP file layout.
• Develop load process for SDR materials not submitted via GRIN.
• System administration
• Transition remaining servers to TSM backups; phase out Arkeia backups; perform testing and training on GUI-based recovery.
• Jan: This work was mostly completed in December; the Arkeia backup server will be retired in March.
• Improve remote access capabilities to server resources using serial console, network KVM (for blades), and VPN device(s).
• Jan: Korner experimented with serial port redirection and remote management functionality and will use the results as a model for new servers as well as for reconfiguring existing servers as opportunities arise.
• Standardize, document, and build mechanisms to centralize system administration processes.
• Replace production and development servers as scheduled, consolidating services and/or using virtualization as appropriate.
• Jan: Snavely continued work to upgrade the web authorization (Oracle) servers, focusing on backups, load balancing, and performance scaling configuration.
• Jan: Korner installed and configured a new SearchTools server and connected it to the SAN.
• Feb: Korner received and began installing and configuring pocarisweat, the new Deep Blue server.
• Relocate servers as appropriate for availability, redundancy, and/or proximity to specific storage systems.
• Transition to ITCS-supported DHCP, and transition DNS service off plumwine using DHCP lease configuration.
• Explore centralized storage for sharing among all LIT units.
• Security
• Complete security planning and perform TBD high-priority tasks.
• Investigate virtual firewall service and request/implement as appropriate.
• Miscellaneous, routine, or unplanned activity
• Jan: Korner made changes to DNS server configuration to address a security vulnerability.
• Jan: Feeman generated some statistics for Kleinman and Santelli regarding SDR contents, customized GROOVE to intercept a specific set of volumes and generate QA samples differently (for the Parsons Rau project), and manually ingested several volumes into the SDR for demonstrations during the millionth book celebration.
• Service Availability
• Arbor Lakes Data Center (ALDC)
• ethel.umdl (ARC): No down time
• ezra.umdl (Verde): No down time
• gracie.umdl (Aleph): Down on Sunday, January 13 from 8:00am to 9:00am for scheduled maintenance.
• hiram.umdl (Zebra): No down time
• tequila.umdl (Metalib and SFX): No down time
• Duderstadt Center Machine Room (DCMR):
• dlps6.umdl (Oracle): No down time
• dlps7.umdl (Numeric and Geospatial Data): No down time
• dlps11.umdl (DLPS collections): No down time
• dlps12.umdl (DLPS collections): No down time
• soymilk.umdl (MySQL): No down time
• Hatcher Server Room
• belle.umdl (Library Web): No down time
• coffee.umdl (Oracle): No down time
• cowmilk.umdl (MySQL): No down time
• hefeweizen.umdl (DLPS collections and MBooks): MBooks service degraded (some or all books unavailable) on Saturday, January 5 from 1:00am to 2:45pm and from Tuesday, January 22 at 3:30pm to Wednesday, January 23 at 6:30am due to problems with a storage system and a storage system upgrade.
• merlot.umdl (DLPS collections and MBooks): See hefeweizen.umdl
• porter.umdl (DLPS collections and MBooks): See hefeweizen.umdl
• quik.umdl (Survivors of the Shoah cache): No down time
• ting.umdl (Library Proxy): No down time
• Michigan Advanced Computing Center (MACC):
• sambuca.umdl (Deep Blue): No down time
• Data Loading/Archival Statistics

  CD/DVDs loaded or reloaded (bitonal page images)	7
  CD/DVDs loaded or reloaded and processed (contone images)	71
  Removable drives loaded or reloaded	2
  CD/DVDs burnt or duplicated	49