Well, it would seem that our internet connection seems to be back in order, although I’m not too thrilled with the manner in which it was ultimately fixed.

Here’s a rundown of what’s been going on:

Friday PM-Sunday PM - Our internet connection starts dropping out for about a minute every 33 minutes, almost to the second.  In addition, outbound browsing is very slow and showing symptoms of packet loss.  I open a ticket with TW tech support, but they claim that my situation can’t be monitored until Monday morning.  I guess 24/7/365 support doesn’t apply to Tier 3.

Sunday PM-Monday AM - Our cable modem stops passing traffic completely.  It still syncs up, but I’m unable to ping any external addresses with it.  Another call to tech support ensues, and the tech finds that something may be wrong with whatever device that provisions are static IP addresses.  Apparently cable modems that use DHCP are still working though, because I — well, let’s just say I found a way to get around this issue for a short time.

Monday AM-PM - We get the pleasure of becoming part of a widespread outage.  No modem sync — at all.  "Everyone who’s got a pulse is working on this one," the tech tells me.  Well that’s good to know.

Monday PM - Wednesday - I finally see sync lights on our cable modem after popping my head in the datacenter for about the 15th time.  Our internet is back, and it’s no longer dropping out every 30 minutes, but it’s still very slow…almost like DNS isn’t resolving properly (the answer is always DNS).  Connections are fast ONCE their made, but it takes forever to get that far.

Wednesday - Slightly frustrated, I call Time-Warner again to see if they can look at the issue and stress-test our modem.  The stress test doesn’t show any issues of packet loss.  After looking at a few other things, this dialog ensues…

Tech:  "What DNS server are you using?"
Me:  "We’re using OpenDNS’s servers" (I rattle off the IP addresses)
Tech:  <paraphrase>"Well, I’m looking at a bulletin sent out by our network team in response to an issue that’s been going on for about 5 days. (hey that sounds like us!)  They’re recommending that anyone seeing speed issues make sure they’re using our DNS servers.
Me:  "Well that’s great, but we use OpenDNS for our content filtering, plus we’ve been using them for a year now with no issues."
Tech:  "Still, try changing your settings and see if that doesn’t help your speed issue." </paraphrase>

So I logged into our DNS servers and changed the forwarder addresses to match Time-Warner’s DNS servers.  BOOM — instant speed increase.  It would seem apparent that using TW’s DNS instead of OpenDNS fixed the issue, but now I’m left with just as many questions as answers.  What’s up with OpenDNS?  Do they have issues of their own?  Is TW doing something on their end to interfere?  If not, did a simple DNS query to a non-TW DNS server take forever?

Needless to say, I’m not too pleased with this situation.  If this is going to be a permanent issue, I now have to find another way to do content filtering.  I will most-likely be giving ScrubIT a try now to see if there are any similar issues.  We’ll see how that goes.

It doesn’t matter how many situations and problems you’ve dealt with as an IT guy, something new and different will eventually come along to make you scratch your head.

On Friday morning I got a notice that our internet connection had dropped out for a short period of time.  After just a minute of being down, the connection came back up.  No big deal, right?

Throughout Saturday morning, I’ve received another notice stating that our connection was down, and then up again.  This happened twice on Saturday morning.  Slightly concerned, I downloaded a little utility (Uptime Scout) to constantly ping our network and log any timeouts that occurred.  At the same time, I called Time Warner Tech Support and opened a ticket with them so they at least had it on record.

When I returned later in the evening, what I found in the logs was very interesting.  Our internet connection was dropping out every 33 minutes, almost to the second.  The outage didn’t last long … a minute at the very most.  However, we use F1, and if this happens on Sunday morning things could get a little messy.

Now I’m very intrigued about our problem, and so I decide to go on-site to look at our cable modem and firewall.  I’m figuring that SURELY our modem is resetting itself or that there’s something that I’ll be able to see.  However, after sitting in the cold datacenter for 50 minutes and watching 2 more outages happen, it’s quite apparent that there’s nothing wrong with the cable modem.  It’s not flinching, and the traffic indicators are still flashing despite my inability to ping anything on the internet.

Well, that leaves you with 2 possible causes … faulty equipment directly outside our building, or a faulty network card in our pfSense box.  As unlikely as I think it is that our pfSense box would have a bad nic (I just replaced it), it’s definitely not outside the realm of possibility. 

So, for the last 40 minutes, my laptop has been connected directly to the cable modem in an attempt to rule out any faulty component on our network, and just as I started typing this paragraph, my pings began to time out and Uptime Scout started making noises at me.  From what I can see, whatever is puking every 33 minutes is not connected to our network or even in the building.

I called Time Warner back with this extra tidbit of information, and I must say that their tech support has been very nice to work with, even at 1:00am.

We don’t support Macs officially at NewPointe as of yet, but I still want life to be as easy as possible for our Mac users.  One of the challenges I’ve been faced with over the past couple weeks is file syncing between iBooks and our file server.

While looking for a solution to this, I stumbled across ChronoSync.  It’s got a few nice options to it, like the ability to start a sync as soon as you log on…nice if you just need to sync before you head out of the office.  Another nice thing is that once you buy a license for ChronoSync, you’re entitled to upgrades for the life of the product.

Is this the best Mac file sync software out there?  I don’t know yet, but it seems to be working so far.  I’m going to be installing it on another iBook next week, and if it continues to work well, it will become standard issue for any new Mac user.  The biggest issue so far has been training the mac user to sync their files up…especially when said user has an iBook and a desktop PC.

I’ve seen this posted on a couple different forums and TUAW as well.  It looks like Microsoft has set up a site to show off the upcoming Office 2008 for Mac.  From a first look, it looks like MS is just showing off the new look of the software and talking about functionality from a very general standpoint.

Now if you’re an IT dude (or dudette) and you deal with Macs, I’ve got a pretty good idea what you’re thinking.  What about Entourage?  Well… the site doesn’t mention anything specific about Entourage functionality as of yet.  I myself am REALLY ANXIOUS to get a look at any improvements in that part of the package alone, especially with PC-to-Mac user conversions on the horizon.

Here’s hoping…

Over the past 3 weeks, we’ve been test-driving Service Desk Plus, a sweet helpdesk application that handles tickets, inventories, purchase orders, and just about any asset that could be combined with a PC.  We’ve also been using this program to take care of tickets for facility-based issues.  While I will say that SDP isn’t really built for facility management, it’s more than adequate to suit our needs.  The biggest win so far for our Facility Director is the ability to create preventive maintenance tickets, which will recur on whatever time period you specify.

During our trial, everyone that had the chance to use SDP had plenty of good things to say about it, and there were a lot of features that while we didn’t get to touch during the trial, we’re pretty pumped about.  That being said, this week we decided to ink a purchase order to buy a yearly subscription to Service Desk.

I’m very pumped about this step forward in our operations, because it’s going to benefit the staff as a whole.  We’ve needed a solid ticketing solution for some time now, and I feel like now we’ve got it (or at least we’re MUCH closer).

Last night around 6:00 we had a small power outage in our building.  This is the first time that we’ve lost power at all since the building was under construction, and it lasted for less than a second.  I wasn’t worried at all about our servers or phone system because everything network-related (servers, switches, both firewalls, cable modem, etc) is on a UPS.  I knew our desktops would receive a hard reboot, but just this once… shouldn’t hurt them, right?

Well, I came in this morning to find that 3 of those desktops weren’t doing too well.  All 3 of them were EXTREMELY sluggish once their respective users logged in.  I took a look at the processes running, and saw that svchost.exe was using +90% of the CPU.  Very odd.  I let the machines sit for a bit, and much to my relief they calmed down after awhile.  The event logs haven’t turned up anything definite

After looking through MS’s support site, I’ve got an few things to check out, although I do have reservations about trying to replicate the incident.  Was this incident related to the power loss?  It would seem so, given the order of events.  Intentionally cutting power on PCs is not my cup o’ tea though…perhaps we’ll try it on one of the cold spares. :-)  If the incident WAS power-related, it really raises the case for setting a “UPS for every PC (or Mac)” policy.  Under the right circumstances, dirty power could bring about a support nightmare for me and K.  I’ve been hoping to avoid the UPS issue until next year, as we’re trying to run lean in 2007.  Hopefully it can stay that way until 2008.

Have you ever had a period of time go by that you were so into your work that you can’t really remember what’s been going on?  That’s where I’ve been lately.  I’m going to attempt to recap some of that stuff here.

We had a training day for our Service Programming teams (video, worship, audio, etc.) this past Sunday.  I had the chance to sit down and talk to our control room volunteers in a non-production setting.  Our video volunteers are AWESOME.  I love being able to work with them and watch them get better at what they do.  Just a few months ago, we had no video people besides myself.  God has blessed us IMMENSELY with a complete control room staff, camera operators, editors for our offline stuff.  It’s been awesome to see His hand at work through everything. 

On Monday we started a trial run on Service Desk Plus.  I must say, I am really impressed with the software thus far.  It’s a huge step up from what we’ve been working on to this point.  We’re going to demo the software for 2 weeks and then make a decision on it.  So far, it looks like a win to me, but we’ll see..

Tuesday and Wednesday, along with messing with Service Desk, I’ve been tinkering with SSL Explorer.  This is something I tried out earlier this year when we were looking for a VPN solution.  It got moved to the back burner to make room for other things, and because at the time we really didn’t have the bandwidth to support it.  I installed it on one of our servers and got it connected to Active Directory.  Once everything is tuned in, K and I will start training on it.  We’ve had some users asking about it (we sent out a poll to guage how much VPN would be used if we installed it), so I’m pretty anxious to get it cranked up so those same users can take it for a test drive.  Much thanks to Andrew Mitry for talking me through what I really needed and helping us save some ca$h.

Tomorrow I was planning on simply getting ready for the simulcast and prepping for the weekend.  Late this afternoon though, Danny’s laptop began giving him fits, so it looks like I’ll be hitting that up first thing tomorrow morning.

Since I’ve come on staff at NewPointe, the only Macs that I’ve purchased have been for production use (Pro Presenter, Final Cut, etc.).  To this point we only have ONE staff member at NPCC that is a full-time Mac user, and he bought his iBook before I came on board.

I’ve wondered now and then how long it would be before I’d get an actual request to purchase a Mac.  A couple weeks ago, I got my answer.  One of our end users has requested a MacBook to run Final Cut on, as well as ProTools for both offline and live use.  This unit would also serve as their “working” system, i.e. they would be checking email, writing Word docs, and doing everything else they did on their previous laptop, on this MacBook.

Now I don’t know if this request is going to go through or not, but it did get me thinking about support issues for Macs.  When do you actually begin to support Mac desktops and notebooks.  Do you wait until you have a certain number of them deployed?  Do you just wait until your users are so frustrated that you no longer have a choice? 

And what about getting your Macs to play nice on a Windows network?  Do a lot or organizations bind them to the windows domain, or are they just free-roaming machines that wander around on the LAN with no external management?

Let me get this out of the way.  I’m not AGAINST Macs.  Those who know me know that I’ve been editing on Final Cut for a few years now, so I could actually sit on either side of the fence on all the Mac/PC issues (cost aside, of course).  What’s on my mind right now is this:  On a Windows network, how much support time do you dedicate to your Mac users, and where do you draw the line on Mac support (if you have a line)?

I know many of you have different thoughts on this.  Share them if you have time.