AC-EMWIN Server Down August 20th-21st...

The AC-EMWIN Server program somehow got locked up yesterday afternoon about 2pm and no bulletins went out for about a 24 hours period. This went unnoticed because I was not able to make my normal daily remote checks on the server due to the fact that I had suffered an on-the-job injury and had been busy with doctors appointments and related medical paperwork and dealing with insurance people.

While the server program might have been down the ingest program, however, continued to actually receive all bulletins downloaded through the EMWIN pipeline and on restart they were all eventually distributed, if but late, for some.

At this time, though, the server is again operating normally. We apologize for any inconvenience.

AC-EMWIN Server Hiccup Last Night (Now Resolved)

Looks like the AC-EMWIN server had a hiccup, last night...

The AC-EMWIN server program locked up at about 8:45pm. All the other SUB-programs kept operating and passing on files which were apparently stuck in the ingest directory (the place where the program temporarily places new bulletins just downlinked, which are USUALLY then immediately *deleted* after they've been processed). So ZFPs and HWOs that didn't get deleted from the ingest dir (because the program locked up at just the right time) and the other subprograms just kept pulling them so they got duped to some Facebook pages a couple times overnight. As well, the radar images from 8pm last night kept getting resent all night until I noticed it early this morning after getting up.

As to how I even noticed that things were locked up... On getting up I noticed that the EMWIN test clients - here at the house, and which are connected to the main server remotely - weren't offering up the normal early morning beeps and sounds that occur when they receive the usual morning ZFPs and HWOs and paints things on the map. After years of being used this this, you notice something's not right almost immediately. This caused me to take a look around and check for problems. Indeed, the AC-SKYWARN web page hadn't refreshed the watch/warning map since 8:30pm last night. All the text bulletins on the AC-EMWIN page were "old" from late yesterday afternoon/evening. As well, no (current) bulletins were being sent to the surrounding area Emergency Management Facebook pages which were subscribed to us. (EEK!)

We're in the middle of some current ongoing weather so I didn't even bother to deal with checking logs and figuring things out. I just rebooted the machine entirely right away and that seemed to take care of it.

...So things are again working properly. I'll keep an eye on the server to make sure it doesn't happen again. The program usually worked very efficiently and I almost never had any problems with it and it's pretty stable so I don't expect another problem like that.

Coincidentally, the server hiccup started the process of me checking ALL resources for the possible problem and this included checking Alachua FreeNet. While it's not part of our server problem, I discovered that Alachua FreeNet is also apparently down, too. Something hung on THEIR server, too, last night and some of you may notice that your web page dirs are empty. (EEK!) I've notified AFN admin about it and they're taking a look. So in a way, you can thank our own server hiccup for causing a causal exploration which ended up discovering the AFN outage. :)