Archive for the ‘Uncategorized’ Category

Filed Under (Uncategorized) by Dave Mast on December-29-2007

I realize that my posting on our domain rebuild ended suddenly in the middle of the night, so I’m going to attempt to fill in what’s happened between then and now, as well as recap from the start.

Wednesday, 3:00pm - Prepping for what’s going to be a very big evening.  I’ve got food ready for "lunch," I’ve got water and coffee ready…. I even put a couple live webcams up so that my fellow geeks could keep tabs on me throughout the night.  I’m still excited, and still nervous, and why shouldn’t I be?  It’s a big job.

5:30pm - Zero hour.  First thing to do, get snapshots of the old DC and Exchange boxes.  If something goes horribly wrong, I want to be able to fail back.  Next, I check and make sure backups are decent.  A quick look at CommVault tells me something isn’t right:  Our 9TB backup array is reporting FULL, and thus Exchange is unable to run backups.  A quick look at my settings shows that our video server is putting WAY too many full backups on the server, and therefore clogging things up royally.  I make the adjustments, run a data aging process, and about 2 hours later, things are finally freed up enough for me to start running ExMerge against the Exchange server.

9:20pm - I’ve finally got ExMerge doing what it’s supposed to be doing.  There are 46 mailboxes that are being exported off the server, and they amount to about 16GB total.  I figured this was a small deal.  Little did I know it would be….

Thursday, 5:00am - …EIGHT hours later before ExMerge finishes.  CRAP!  That’s a 2GB/hour transfer rate.  There’s a couple reasons I think it took this long:

  • RAID 5 - My Exchange server is a VM, and I was writing the PST data to a share on the host machine.  This is all happening on a RAID 5 array, so I wouldn’t be surprised if it choked hard on all that data.  What I should have done was export the PSTs to an external HD or another machine.
  • Folder sizes - Yes, the PSTs only made up about 16GB, but there was MUCH room for improvement.  There were Deleted Items and Sent Items folders that were WAY too large.  I saw one Inbox folder (owner will remain anonymous) that contained over 1900 messages.  I should have asked our users to clean/export a majority of this data before the ExMerge operation.
  • ExMerge - By far the BIGGEST improvement I could have made.  I read through the whole ExMerge documentation, and somehow failed to remember that it can do incremental backups!  I could have run this a week ago and taken HOURS off the transition time.

    In any case, ExMerge is done now, and it’s time to shut down the old sugarcreekfmc.org DCs and Exchange server and get the new ones cranked up.  The new DC fired right up on its VM host, and at 5:12am (yes, I looked at the Event Log) I ceremoniously added the first member server to the newpointe.loc domain; our WSUS box.  The file server and my desktop PC would follow shortly after.

    Now that the new DC and Exchange server are cranked, it was time to bring mailbox data back in.  Still feeling the effects of an 8-hour ExMerge session, I decided to move all the PSTs onto an external HD before merging them into the new Exchange box.  For the record, this went MUCH faster.  However, I knew that I couldn’t get everyone in before 8:30 (when people usually come in).  I had requested that those who were coming in on the 27th email me so I could give their machines priority.  Thus, I had a list of about 8 people that I needed to get ready for.

    These 8 lucky staffers would be the FIRST people (besides me) to log in to the new domain, but more importantly, they had stuff to get done on Thursday, so I wanted to make sure EVERYTHING they needed was ready. 

    10:00am - Email, check.  Profiles moved over… uhm… almost check.  File permissions…those didn’t go very well.  I spent a LOT of time going back and forth between users’ desks and mine to change file ownership and permissions.  To make matters worse, not only were people coming in early, people were coming in that didn’t tell me they would be there.  (can I get a "DOH!").

    By lunch time, things were still progressing, albeit slower and slower.  I was wearing out.  The operation was coming up on 18 hours.  I had been up for about 26 hours at this point, and I was starting to lose steam quickly.

    1:00pm - The afternoon drug on … I was still going back and forth fixing file permissions, and now I’ve also discovered that my GPO for passwords is acting up as well, some users aren’t able to reset their passwords back to what they were.  This wasn’t the end of the world, but with my reduced ability to handle stress, it was wearing on me hard.

    2:00pm - I was about at my last straw at this point.  I’ve started on laptops, and for some reason, RPC-HTTP/S, which WORKED during the initial build and testing is no longer working.  I can’t even authenticate!  I’m way too tired by now to understand what the problem is, and nothing frustrates me quite like something that I can’t understand.  RPC-HTTP/S will have to wait….we’ll use OWA for now.

    I’m walking back to my desk feeling utterly defeated.  Did I set my expectations too high?  Did I not research enough?  I’ve experienced nothing but problems all day.  The users that are here barely have the functionality they need to do their job, and now I’ve got a broken Exchange server to deal with.

    I got back to my desk and saw this pinned to my chair…

    100_0531

    One of the staff had put it there, along with a bottle of flavored water, which I promptly drank.  The sign was strategically placed so that one of my webcams was aimed right at it.  This blessed me more than I can say, and it must’ve been something I needed pretty badly at the time.  Spirits lifted, I took care of a couple more issues before finally heading home at 3:30pm for some much needed sleep.

    Total work time so far:  22 hours.



  • Filed Under (Uncategorized, domain rebuild) by Dave Mast on December-26-2007

    Yeah, so this might by a little dorky. I don’t care. For your viewing enjoyment, the domain rebuild is going to be presented live on 2 different webcams. You can access them by following the links below.

    Camera 1 - My Desk

    Camera 2 - Mobile camera attached to my laptop.

    Enjoy!

    Update:  These cameras have long-since been taken offline, so don’t waste your time clicking on the links.  :-)



    Filed Under (Uncategorized) by Dave Mast on December-26-2007

    If you’re going to work good, you need to eat good.

    Just to quell any concerns that I may be blowing my diet to pull an all-nighter tonight (and yes, I do expect to be here all night), I thought I’d give a glimpse of what the menu will be like for the next 20-23 hours:

    domainrebuildbreakfast

    For starters, we’ve got some boneless skinless chicken breast.  That alone is good stuff.  I’ve also got some sweet onion, broccoli, zucchini, and mushrooms to throw into the mix for what stands to be a kickin’ stir fry combo, sans the rice.  You’ll also notice some eggs and OJ there as well.  That’s for breakfast.  Yes.  Breakfast.

    I’m a geek, but I love to cook, and I can be quite serious about it.  I might have also brought my wok and some olive oil with me to assist with the stir fry.

    wokkandoil

     

    It’s going to be a good night.



    Filed Under (Uncategorized) by Dave Mast on December-4-2007

    I’ve been in somewhat of a blogging drought lately.  Honestly, I’ve been trying to think of stuff to write on but those ideas have escaped me.  However, my mind has been far from blank … just some random miscellany

    -> I am psyched about Sharepoint training in January (read more about it here).

    -> I snagged a couple helpful apps for my Q last week:

    Power Tasks - An uber-enhanced version of the Task List in Windows Mobile, this software reformats your task list into something similar to the “To-Do List” view in Outlook.  Very handy for me, because I’m keeping more and more of my assignments here.

    Gentimer - A sweet clock/timer/stopwatch application that I’ve started using to keep track of ticket and task times.

    -> The domain rebuild that I’ve been working on is going well.  I’ve been taking some time to document every non-default configuration that’s happening on the new domain controller and exchange server.

    -> I’ve been using our helpdesk application to track my time spent in video versus IT.  I’ve been pretty surprised by the results.  (On the subject of ServiceDesk … I wish they would release an upgrade patch from version 6 to 7.)

    -> We’re going to be installing Leopard on our Final Cut rig this week or next.  I’ll be interested to see if the new Automator can add some automatic sweetness to our recording/encoding on Sunday mornings.

    That’s about it for now… hopefully I can think of something to post on soon.  If nothing else, then you’ll hear more about the domain rebuild as the time to switch over draws near.



    Filed Under (Uncategorized) by Dave Mast on September-29-2007

    Sorry for suddenly going silent since Wednesday morning.  There has been a LOT going on since then.  I’m just going to take a few moments and recap the week.

    The IT Roundtable at Granger was awesome.  I had a chance to sit and soak in some dialog from individuals that I consider to be much sharper than myself.  I’ll post more later about some stuff that I took away from the ITR.  Mega-props to the Granger IT guys for making this happen!

    Thursday was a myriad of meetings, phone calls to make, and orders to place.  We’re gearing up for budget season again, so I’ve been working on next year’s goals and what it’s going to take to accomplish them.

    On Friday I spent most of the day tinkering with a Miranda DVI-Ramp2 as we demo it for this weekend.  I’m pretty pleased with what it’s capable of.  After that, it was off to the Swiss Festival in quiet little Sugarcreek, Ohio for some walking around, running into people we know, and just basically enjoying the fall weather.  The best part of the festival?  Fried cheese.  No lie, man….best stuff ever. (4 out of 5 physicians disagree).



    Filed Under (Uncategorized) by Dave Mast on September-26-2007

    Well, unfortunately it looks like we won’t be doing any live streaming.  However, the audio from the Roundtable will be recorded and will be available in one-hour chunks at a location to be determined.



    Filed Under (Uncategorized) by Dave Mast on September-24-2007

    This morning I walked into the NewPointe offices after 2 weeks of being on vacation.  Don’t get me wrong, I would still rather fish, but it felt great to be back to work, also.  To add to that, the domain controllers that I was having problems with yesterday seem to be holding time just fine.  There were a few tickets that piled up over 2 weeks, but that’s about it.  All-in-all, it was a great first day back from vacation (or “holiday,” if you’re reading from across the Atlantic).

    I’m headed to bed here in a few minutes, and then I’ll be driving out to GCC bright and early tomorrow for the 2007 Church IT Roundtable.  Nothing like coming back from vacation and going straight into a road trip. ;-)



    Filed Under (Uncategorized) by Dave Mast on September-18-2007

    diy-gadget-mount.png

    I don’t know if I like having the weight of my iPod (plus the case) resting on its connector, but to each his own.  This method definitely wins the “budget prize.”

    Check out the article here.



    Filed Under (Uncategorized) by Dave Mast on September-11-2007

    I’ve been shying away from social networking sites for awhile now, just because it has always seemed like a chore to keep it all updated.  After some consideration though, I decided to open up an account on Facebook.  It’s actually been there for awhile now, I just decided to update it today while I’m waiting on other stuff.

    Check out my profile here and add me as a friend if you’re so inclined.  You know it’s the right thing to do.



    Filed Under (Uncategorized) by Dave Mast on August-22-2007

    …And I don’t mean ”decorative good-looking”….pretty stenciling on the server rack and trim on the IDFs that match the woodwork or carpet.  That’s unrealistic and quite frankly, a little fruity.

    I’m talking about “functional neatness,” if you will.  Wires tied down on racks, color-coded cables to show what is being linked up, proper labeling on every wire and piece of equipment, that sorta thing.

    Back in the day I was terrible about this.  I would just fling wires around and plug in what needed to be plugged in and as long as the little green lights were blinking on the switch, I didn’t give a rip what it looked like.  These days though, this is something I take pretty seriously.  Whenever we hook up a new piece of gear or patch in a new workstation, we make sure that things aren’t only functional, they look as good as they can.  So for me, can IT be too “good-looking?”  Definitely not.  Here’s why:

    If it’s hard to look at, it’s going to be hard to work on.  This is especially true in the IT world.  If you need to trace down a link, do you want to have to look through a rat’s nest of CAT/serial/power cables to do it?  Of course not.  A small time investment on the front end can save all kinds of diagnosis time down the road.   Note: This also applies to sound systems.  Our sound guys do an AWESOME job of keeping our wiring tidy during upgrades.

    Things need to be functional without you.  “Yeah, well I KNOW where every wire goes!”  Yeah, so?  You might get hit by a 1980 AMC Eagle tomorrow morning, too.  Especially in the church world, you need to make sure that the work can carry on WITHOUT you being there.  Don’t pass your wire mess onto someone else just because you know where everything plugs into! (This is also a strong case for…*cough*….thorough documentation.)

    Here’s a couple examples of functional neatness. (here and here).  Now I’m not there yet, but this is what I want our patch panels at NewPointe to end up looking like, along with color-coded cables for each connection, whether it be printer, server, LAG, whatever.  Is this something that’s going to happen immediately?  No, not likely.  Next year?  The chances are good.  Nonetheless, it’s been something that’s been on my mind lately.




    FireStats iconPowered by FireStats