Re: [Sbarc-list] Redundant systems
Hello Walt, I fully understand your frustration with our lack of constructive action. I have not given up, but will try this different approach. I appreciate your interest in the past and your efforts to clarify and highlight the need and try to motivate us to take action. That has been helpful and provides motivation to move forward. It is with sincere regret that my efforts have been ineffective to date. I know that you supported my efforts. I just fell short, though admittedly, this isn’t something that one person can implement. I wish you and Jayne all the best and hope that someday, your dream (our dream) of an effective emergency service system will be realized. Your friend, Rod WB9KMO From: Walt Sent: Thursday, March 12, 2015 10:43 PM To: Alain Michel ; mailto:rfritz22@cox.net ; William Talanian ; sbarc list Cc: 'WB6RDV' ; Jay Hennigan Subject: Re: [Sbarc-list] Redundant systems Greetings: Three years ago, Jayne and I rekindled our interest in SBARC (Jayne and our son were members back in 1978) because we hoped that he club would be useful in case of an emergency, especially an earthquake. WE have been trying unsuccessfully to interest club members and ARRL for almost three years about doing something about the matter. We have expressed our disappointment that neither ARRL or SBARC was interested in taking effective steps to establish some sort of effective emergency procedure. At this point Jayne has lost all interest in Amateur Radio, and I have developed other interests to which I intend to dedicate all my efforts, so, Rod, count us out. Walt From: Alain Michel Sent: Thursday, March 12, 2015 4:00 PM To: mailto:rfritz22@cox.net ; William Talanian ; sbarc list Cc: 'WB6RDV' ; Jay Hennigan Subject: Re: [Sbarc-list] Redundant systems " When I ask, "how do we get help when the big one hits?", the answer is always, "dial 911"." Hi Rod, I don't mean to sound impertinent but that's funny; "call 911". Both my wife and I are FEMA trained CERT instructors down here in San Diego County; we instruct our students that if they follow the "disaster preparedness" we outline, they may not need 911. First responders are going to be sooo overwhelmed that the fire, EMT's and law enforcement that Santa Barbara current enjoys, may very well be deployed elsewhere [if damage is just light to moderate] to cities [Ventura and elsewhere] where the damage may be much more severe. For at least the first 72 hours, folks are going to be on their own---prepared or not. We instruct our citizens to be prepared for 7 days...minimum. Late last year, I was listening to L.A.'s KNX 1070 AM radio station. A reporter was interviewing an L.A. fire chief who was retiring. She asked him how well LA County "first responders" were prepared to handle another Northridge or a Landers? His response? "There are about 10,000 first responders [emt's & firefighters] for 4 million people---you do the math." I must say that I am a bit surprised there there are not more SBARC folks chiming in... Vry 73 de Alan...N6HPO Valley Center C.E.R.T. WWW.VCFPDCERT.ORG On Thursday, March 12, 2015 8:42 AM, "Rod Fritz (rod@sbatv.org)" <rfritz22@cox.net> wrote: Thank you Andy and Bill. I was delighted to see your article. It clearly states some concerns that I have and have likewise tried to work around, as you have. One important contention of mine comes to mind. That's the vulnerability and limitations of the 911 system. When I ask, "how do we get help when the big one hits?", the answer is always, "dial 911". As you have just illustrated, that may not work. Furthermore, even if it does, I contend that 911 access and access to resources may quickly be saturated and ineffective. This rings of "best effort" again. Legislators might admit we can't afford a perfect system (or maybe even an excellent or very good one?). As the currently common (I hate it) reply goes, "it is what it is." Worse yet, when access to resources is limited, who gets them? I leave it to your imagination how this would play out but I contend that individual citizens are on the bottom of the list. I recognize and applaud CERT for what they are doing to prepare citizens. They are an important part of the solution but they don't address the communication that is necessary. I know they're working to improve in this area and we ought to assist them in this regard. Consider ham radio as a partial solution... I want to preface this by saying that I believe that ARES is a valuable resource. ARES is well established but it relies solely on 911 for community (citizen) input. It is my understanding that there is no plan in place for citizens to contact emergency services through ARES. ARES is subservient to the government and the 911 system. I believe that ARES could be structured to better serve the citizens of the community, but it is what it is. I welcome any changes that would make the ARES structure serve citizens better. And then there's SBARC... SBARC has incredible potential to serve the citizens of the community and we're already part way there. We need to stop relying solely on ARES for amateur radio emergency services. We need to help more citizens become "active" radio amateurs. How about at least one per city block? We need to establish an end to end plan with real redundancy to connect citizens to services they need, even when conventional means don't work. This would use a diversity of communication types and paths, not only ham radio. I think several of us have good ideas how we can make this work well and I know it is needed and would be oh so valuable when the big one hits. Details are beyond the scope of this email but I propose an SBARC Emergency Service committee to make it happen. As an SBARC Board member, I will attempt to establish this committee. Which of you would like to participate? 73, Rod Fritz, WB9KMO -------- Original message -------- From: William Talanian <w1uuq@cox.net> Date: 03/12/2015 7:22 AM (GMT-06:00) To: sbarc list <sbarc-list@lists.netlojix.com> Subject: [Sbarc-list] Redundant systems Think it can't happen? <http://andrewseybold.com/3539-it-cant-happen-to-me> _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list -------------------------------------------------------------------------------- _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list -------------------------------------------------------------------------------- No virus found in this message. Checked by AVG - www.avg.com Version: 2015.0.5646 / Virus Database: 4306/9283 - Release Date: 03/12/15
I don't want to turn this thread into a long polemic, but after a couple of years of participation with ARES I gave up because it became apparent that local authorities were not involving us pretty much regardless of what we could do. That was at the time of all the fires in the area and ARES was not mobilized a single time (unless I missed something). I didn't have the sense that it was SBARCs or ARES' fault but I'm also not really interested in those politics anyway... My conclusion was that I was wasting my time and I'm not trying to blame anyone here for this. Thorsten On 3/12/2015 9:06 PM, Rod - Traveling wrote:
Hello Walt, I fully understand your frustration with our lack of constructive action. I have not given up, but will try this different approach. I appreciate your interest in the past and your efforts to clarify and highlight the need and try to motivate us to take action. That has been helpful and provides motivation to move forward. It is with sincere regret that my efforts have been ineffective to date. I know that you supported my efforts. I just fell short, though admittedly, this isn’t something that one person can implement. I wish you and Jayne all the best and hope that someday, your dream (our dream) of an effective emergency service system will be realized. Your friend, Rod WB9KMO *From:* Walt <mailto:zharasty@verizon.net> *Sent:* Thursday, March 12, 2015 10:43 PM *To:* Alain Michel <mailto:opalockamishabob@yahoo.com> ; mailto:rfritz22@cox.net ; William Talanian <mailto:w1uuq@cox.net> ; sbarc list <mailto:sbarc-list@lists.netlojix.com> *Cc:* 'WB6RDV' <mailto:jay@west.net> ; Jay Hennigan <mailto:jay@impulse.net> *Subject:* Re: [Sbarc-list] Redundant systems Greetings: Three years ago, Jayne and I rekindled our interest in SBARC (Jayne and our son were members back in 1978) because we hoped that he club would be useful in case of an emergency, especially an earthquake. WE have been trying unsuccessfully to interest club members and ARRL for almost three years about doing something about the matter. We have expressed our disappointment that neither ARRL or SBARC was interested in taking effective steps to establish some sort of effective emergency procedure. At this point Jayne has lost all interest in Amateur Radio, and I have developed other interests to which I intend to dedicate all my efforts, so, Rod, count us out. Walt *From:* Alain Michel <mailto:opalockamishabob@yahoo.com> *Sent:* Thursday, March 12, 2015 4:00 PM *To:* mailto:rfritz22@cox.net ; William Talanian <mailto:w1uuq@cox.net> ; sbarc list <mailto:sbarc-list@lists.netlojix.com> *Cc:* 'WB6RDV' <mailto:jay@west.net> ; Jay Hennigan <mailto:jay@impulse.net> *Subject:* Re: [Sbarc-list] Redundant systems " When I ask, "how do we get help when the big one hits?", the answer is always, "dial 911"." Hi Rod, I don't mean to sound impertinent but that's funny; "call 911". Both my wife and I are FEMA trained CERT instructors down here in San Diego County; we instruct our students that if they follow the "disaster preparedness" we outline, they may not need 911. First responders are going to be sooo overwhelmed that the fire, EMT's and law enforcement that Santa Barbara current enjoys, may very well be deployed elsewhere [if damage is just light to moderate] to cities [Ventura and elsewhere] where the damage may be much more severe. For at least the first 72 hours, folks are going to be /*on their own*/---prepared or not. We instruct our citizens to be prepared for 7 days...minimum. Late last year, I was listening to L.A.'s KNX 1070 AM radio station. A reporter was interviewing an L.A. fire chief who was retiring. She asked him how well LA County "first responders" were prepared to handle another Northridge or a Landers? His response? "There are about 10,000 first responders [emt's & firefighters] for 4 million people---you do the math." I must say that I am a bit surprised there there are not more SBARC folks chiming in... Vry 73 de Alan...N6HPO Valley Center C.E.R.T. WWW.VCFPDCERT.ORG
On Thursday, March 12, 2015 8:42 AM, "Rod Fritz (rod@sbatv.org)" <rfritz22@cox.net> wrote:
Thank you Andy and Bill. I was delighted to see your article. It clearly states some concerns that I have and have likewise tried to work around, as you have. One important contention of mine comes to mind. That's the vulnerability and limitations of the 911 system. When I ask, "how do we get help when the big one hits?", the answer is always, "dial 911". As you have just illustrated, that may not work. Furthermore, even if it does, I contend that 911 access and access to resources may quickly be saturated and ineffective. This rings of "best effort" again. Legislators might admit we can't afford a perfect system (or maybe even an excellent or very good one?). As the currently common (I hate it) reply goes, "it is what it is." Worse yet, when access to resources is limited, who gets them? I leave it to your imagination how this would play out but I contend that individual citizens are on the bottom of the list. I recognize and applaud CERT for what they are doing to prepare citizens. They are an important part of the solution but they don't address the communication that is necessary. I know they're working to improve in this area and we ought to assist them in this regard. Consider ham radio as a partial solution... I want to preface this by saying that I believe that ARES is a valuable resource. ARES is well established but it relies solely on 911 for community (citizen) input. It is my understanding that there is no plan in place for citizens to contact emergency services through ARES. ARES is subservient to the government and the 911 system. I believe that ARES could be structured to better serve the citizens of the community, but it is what it is. I welcome any changes that would make the ARES structure serve citizens better. And then there's SBARC... SBARC has incredible potential to serve the citizens of the community and we're already part way there. We need to stop relying solely on ARES for amateur radio emergency services. We need to help more citizens become "active" radio amateurs. How about at least one per city block? We need to establish an end to end plan with real redundancy to connect citizens to services they need, even when conventional means don't work. This would use a diversity of communication types and paths, not only ham radio. I think several of us have good ideas how we can make this work well and I know it is needed and would be oh so valuable when the big one hits. Details are beyond the scope of this email but I propose an SBARC Emergency Service committee to make it happen. As an SBARC Board member, I will attempt to establish this committee. Which of you would like to participate? 73, Rod Fritz, WB9KMO
-------- Original message -------- From: William Talanian <w1uuq@cox.net> Date: 03/12/2015 7:22 AM (GMT-06:00) To: sbarc list <sbarc-list@lists.netlojix.com> Subject: [Sbarc-list] Redundant systems
Think it can't happen?
<http://andrewseybold.com/3539-it-cant-happen-to-me>
_______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
_______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com <mailto:SBARC-list@lists.netlojix.com> http://lists.netlojix.com/mailman/listinfo/sbarc-list
------------------------------------------------------------------------ _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
------------------------------------------------------------------------
No virus found in this message. Checked by AVG - www.avg.com <http://www.avg.com> Version: 2015.0.5646 / Virus Database: 4306/9283 - Release Date: 03/12/15
_______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
On 3/12/15 22:25, Thorsten von Eicken wrote:
I don't want to turn this thread into a long polemic, but after a couple of years of participation with ARES I gave up because it became apparent that local authorities were not involving us pretty much regardless of what we could do. That was at the time of all the fires in the area and ARES was not mobilized a single time (unless I missed something). I didn't have the sense that it was SBARCs or ARES' fault but I'm also not really interested in those politics anyway... My conclusion was that I was wasting my time and I'm not trying to blame anyone here for this. Thorsten
I can see both sides of the coin. ARES and SBARC have value in case of a *telecommunications* emergency. We aren't firefighters, we aren't paramedics, we aren't traffic cops, we aren't mountain rescue people. At least we aren't in the context of ARES and SBARC although some of us may be as individuals. If the landline, cellular and public safety voice and fixed and mobile data networks are operational during a fire, we probably shouldn't be mobilized. We kind of went down that road many years ago with close ties to Search and Rescue where ARES was activated very frequently in cases where there was no need for communicators. Eventually both groups realized that there wasn't a need. Andy's example of the Centurylink fiber cut in Arizona is a good example of a situation where amateur radio could have been quite useful. It would be interesting to talk to the area hams and any local ARES groups to see how, if at all, they contributed. And, as Bill and others have pointed out many times, our contribution to the community in terms of what we do on our own in terms of communication systems is itself an asset, even in the absence of any emergency. In my opinion we need to toot our own horn a bit more when it comes to the things the community takes for granted such as the vessel tracking systems, weather stations, ELT receivers, and remote camera systems that SBARC provides. These are real benefits to society that people not in-the-know assume are just there "in the cloud" or are provided by the taxpayers. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
There's a social media application called Nextdoor and we have used it to form some kind of neighborhood group. At a recent meeting, CERT was brought up so we're trying to get a lot of the local neighbors CERT'ed. I brought up the topic of amateur radio as an alternative means of reaching County OES. During the Northridge earthquake, the cell towers were jammed and the land lines were so jammed you couldn't even get a dial tone. Fortunately, one of the neighbors works in OES and he immediately thought it was a great idea. I volunteered another SBARC member so he and I seem to be emerging as the alternate way of communicating out (hoping the repeaters are working so we won't need to go to Simplex). We're looking at it more as augmenting CERT especially since I've never been able to get involved with ARES. Michael Kwan ----- Original Message ----- From: "Jay Hennigan" <jay@west.net> To: "Thorsten von Eicken" <tve@rightscale.com>, "Rod - Traveling" <rod@sbatv.org>, "Walt" <zharasty@verizon.net>, "Alain Michel" <opalockamishabob@yahoo.com>, "William Talanian" <w1uuq@cox.net>, "sbarc list" <sbarc-list@lists.netlojix.com> Sent: Thursday, March 12, 2015 11:25:28 PM Subject: Re: [Sbarc-list] ARES activation needs, was: Redundant systems On 3/12/15 22:25, Thorsten von Eicken wrote:
I don't want to turn this thread into a long polemic, but after a couple of years of participation with ARES I gave up because it became apparent that local authorities were not involving us pretty much regardless of what we could do. That was at the time of all the fires in the area and ARES was not mobilized a single time (unless I missed something). I didn't have the sense that it was SBARCs or ARES' fault but I'm also not really interested in those politics anyway... My conclusion was that I was wasting my time and I'm not trying to blame anyone here for this. Thorsten
I can see both sides of the coin. ARES and SBARC have value in case of a *telecommunications* emergency. We aren't firefighters, we aren't paramedics, we aren't traffic cops, we aren't mountain rescue people. At least we aren't in the context of ARES and SBARC although some of us may be as individuals. If the landline, cellular and public safety voice and fixed and mobile data networks are operational during a fire, we probably shouldn't be mobilized. We kind of went down that road many years ago with close ties to Search and Rescue where ARES was activated very frequently in cases where there was no need for communicators. Eventually both groups realized that there wasn't a need. Andy's example of the Centurylink fiber cut in Arizona is a good example of a situation where amateur radio could have been quite useful. It would be interesting to talk to the area hams and any local ARES groups to see how, if at all, they contributed. And, as Bill and others have pointed out many times, our contribution to the community in terms of what we do on our own in terms of communication systems is itself an asset, even in the absence of any emergency. In my opinion we need to toot our own horn a bit more when it comes to the things the community takes for granted such as the vessel tracking systems, weather stations, ELT receivers, and remote camera systems that SBARC provides. These are real benefits to society that people not in-the-know assume are just there "in the cloud" or are provided by the taxpayers. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
For what it's worth, I sent the following to the Scottsdale ARC and ARES, I will report their response: Subject: Amateur radio utilization during Centurylink outage Message: Greetings! I'm WB6RDV, President of the Santa Barbara Amateur Radio Club in California. There's an ongoing discussion among area hams about the use of ARES and amateur radio during emergencies. We conduct regular nets and drills, but are very rarely mobilized by local government during emergencies. One school of thought is that these events are rarely emergencies in terms of telecommunications infrastructure. As hams we aren't firefighters, paramedics, etc. As long as the regular communications networks are functional and not vastly overloaded, even if there is a major emergency, it isn't a communications emergency. The Centurylink fiber cut in Arizona was a different beast. It was a telecommunications emergency and only a telecommunications emergency. Both personally and as president of SBARC I would like to hear how, if at all, the local amateur community was utilized during that event and what lessons if any came out of it. Vy 73, WB6RDV -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Well written, Jay! I'd be interested in hearing the results too. Marina, KA6JWL On Fri, Mar 13, 2015 at 9:52 AM, Jay Hennigan <jay@west.net> wrote:
For what it's worth, I sent the following to the Scottsdale ARC and ARES, I will report their response:
Subject: Amateur radio utilization during Centurylink outage
Message: Greetings!
I'm WB6RDV, President of the Santa Barbara Amateur Radio Club in California.
There's an ongoing discussion among area hams about the use of ARES and amateur radio during emergencies. We conduct regular nets and drills, but are very rarely mobilized by local government during emergencies.
One school of thought is that these events are rarely emergencies in terms of telecommunications infrastructure. As hams we aren't firefighters, paramedics, etc. As long as the regular communications networks are functional and not vastly overloaded, even if there is a major emergency, it isn't a communications emergency.
The Centurylink fiber cut in Arizona was a different beast. It was a telecommunications emergency and only a telecommunications emergency.
Both personally and as president of SBARC I would like to hear how, if at all, the local amateur community was utilized during that event and what lessons if any came out of it.
Vy 73, WB6RDV
-- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
-- *Marina King* *President* *SB Area Council of PTAs(805) 696-3424* *www.sbpta.org <http://www.sbpta.org>* <https://www.facebook.com/santabarbarapta>
Jay--Scottsdale was not involved, it was only North Arizona, so not sure why Scottsdale ARES would have anything to do with it. BTW my comment about CenturyLink having to walk the route of the fiber came from the CTO of CenturyLink and the info about the outages from the 9-1-1 center, and also the press of that area. I also don't believe but I may be wrong that there is a phone company in the area served. Since my article was mostly about last mile connectivity in the Santa Barbara area when I discussed it, how many paths to you have serving each of your customers, not your connections to the Internet but the pipe between your facilities and your customers facilities? Best regards Andy -----Original Message----- From: sbarc-list-bounces@lists.netlojix.com [mailto:sbarc-list-bounces@lists.netlojix.com] On Behalf Of Jay Hennigan Sent: Friday, March 13, 2015 9:52 AM To: sbarc-list@lists.netlojix.com Subject: Re: [Sbarc-list] ARES activation needs, was: Redundant systems For what it's worth, I sent the following to the Scottsdale ARC and ARES, I will report their response: Subject: Amateur radio utilization during Centurylink outage Message: Greetings! I'm WB6RDV, President of the Santa Barbara Amateur Radio Club in California. There's an ongoing discussion among area hams about the use of ARES and amateur radio during emergencies. We conduct regular nets and drills, but are very rarely mobilized by local government during emergencies. One school of thought is that these events are rarely emergencies in terms of telecommunications infrastructure. As hams we aren't firefighters, paramedics, etc. As long as the regular communications networks are functional and not vastly overloaded, even if there is a major emergency, it isn't a communications emergency. The Centurylink fiber cut in Arizona was a different beast. It was a telecommunications emergency and only a telecommunications emergency. Both personally and as president of SBARC I would like to hear how, if at all, the local amateur community was utilized during that event and what lessons if any came out of it. Vy 73, WB6RDV -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
Dear SBARC Mailing List - As a person who in a previous career worked in the US Department of Commerce’s National Telecommunications and Information Administration (NTIA) on projects dealing with emergency communications, I am following this discussion with interest. My wife, Cathy Boggs (KD6LGG), is on the SBTV board of directors and they were very interested in partnering with the SBARC a few years ago in terms of merging emergency public access channel cable TV services with the amateur radio segment. However, as far as I know nothing concrete was undertaken. One possible avenue for the SBARC to promote greater public awareness would be to program segments on the SBTV public access channel. These could include programming segments that outline how the public might prepare a family and business communication plan for use in the aftermath of a large earthquake or other large-scale infrastructure shut-down. Larry WA6MVJ On Mar 13, 2015, at 10:01, Andrew Seybold <aseybold@andrewseybold.com<mailto:aseybold@andrewseybold.com>> wrote: Jay--Scottsdale was not involved, it was only North Arizona, so not sure why Scottsdale ARES would have anything to do with it. BTW my comment about CenturyLink having to walk the route of the fiber came from the CTO of CenturyLink and the info about the outages from the 9-1-1 center, and also the press of that area. I also don't believe but I may be wrong that there is a phone company in the area served. Since my article was mostly about last mile connectivity in the Santa Barbara area when I discussed it, how many paths to you have serving each of your customers, not your connections to the Internet but the pipe between your facilities and your customers facilities? Best regards Andy -----Original Message----- From: sbarc-list-bounces@lists.netlojix.com<mailto:sbarc-list-bounces@lists.netlojix.com> [mailto:sbarc-list-bounces@lists.netlojix.com] On Behalf Of Jay Hennigan Sent: Friday, March 13, 2015 9:52 AM To: sbarc-list@lists.netlojix.com<mailto:sbarc-list@lists.netlojix.com> Subject: Re: [Sbarc-list] ARES activation needs, was: Redundant systems For what it's worth, I sent the following to the Scottsdale ARC and ARES, I will report their response: Subject: Amateur radio utilization during Centurylink outage Message: Greetings! I'm WB6RDV, President of the Santa Barbara Amateur Radio Club in California. There's an ongoing discussion among area hams about the use of ARES and amateur radio during emergencies. We conduct regular nets and drills, but are very rarely mobilized by local government during emergencies. One school of thought is that these events are rarely emergencies in terms of telecommunications infrastructure. As hams we aren't firefighters, paramedics, etc. As long as the regular communications networks are functional and not vastly overloaded, even if there is a major emergency, it isn't a communications emergency. The Centurylink fiber cut in Arizona was a different beast. It was a telecommunications emergency and only a telecommunications emergency. Both personally and as president of SBARC I would like to hear how, if at all, the local amateur community was utilized during that event and what lessons if any came out of it. Vy 73, WB6RDV -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net<mailto:jay@impulse.net> Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com<mailto:SBARC-list@lists.netlojix.com> http://lists.netlojix.com/mailman/listinfo/sbarc-list _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list ———————————————————* From: Larry F. Martinez, Professor Department of Political Science California State University, Long Beach, CA 90840-4605 USA larry.martinez@csulb.edu<mailto:larry.martinez@csulb.edu>
On 3/13/15 10:01, Andrew Seybold wrote:
Jay--Scottsdale was not involved, it was only North Arizona, so not sure why Scottsdale ARES would have anything to do with it.
I got a reply from the Arizona SEC, my knowledge of Arizona geography isn't that great. We do have two Flagstaff customers who weren't affected.
BTW my comment about CenturyLink having to walk the route of the fiber came from the CTO of CenturyLink and the info about the outages from the 9-1-1 center, and also the press of that area. I also don't believe but I may be wrong that there is a phone company in the area served.
That's a scary thought that their CTO doesn't know that OTDRs exist and are often installed as RTU equipment that can be deployed remotely to get a measurement without even disconnecting anything. Here's another quote from the press that Centurylink first became aware of the problem due to complaints from customers and not due to their Network Operation Center lighting up like a Christmas tree with alarm indicators. If you believe the spin, not only do they not know how to shoot fiber with a TDR to find the break, they don't even bother to monitor their long-haul links and have to wait for customer complaints to realize they have a problem: The Arizona Republic reported, “Employees from Centurylink told police they stared receiving complaints of interrupted cable and internet service at about noon and, upon checking their system, determined the cause of the outage was coming from…”. The reaction on tech mailing lists: "Really? A major link is down and these employees aren’t getting calls from one of CL’s NOCs? I would hope that the paper got it wrong, and it’s just that some employees didn’t know but that other staff was already being mobilized." Our experience with Centurylink is very different. We have several Centurylink circuits around the country and get a proactive email from their NOC when even a T-1 goes down, so I really question where the press is getting their information or if they just make it up as they go along. It looks like in this case it was under six hours to get to a rather remote cut, set up, and begin splicing with total restoration in another six hours or so. This is definitely not too shabby, and really puts into question the "walking mile-by-mile" story.
Since my article was mostly about last mile connectivity in the Santa Barbara area when I discussed it, how many paths to you have serving each of your customers, not your connections to the Internet but the pipe between your facilities and your customers facilities?
It depends on the customer and what they're willing to pay for, how critical it is to their business. Larger customers with critical needs have two circuits on different last-mile carriers run to different Impulse points of presence. Typically these are active/active where under normal operation we will run voice and video down one pipe and data down the other, with cross-failover. If the primary data pipe fails then data rides the voice pipe. If the primary voice/video pipe fails then that fails over to the data pipe. Sometimes mixed-use active-active, sometimes active-standby. The diverse circuits converge at or very close to the customer premise. A backhoe in their driveway is a real concern, one a block away not so much. That being said, there are always single points of failure somewhere and failure modes not anticipated. In the big scheme, Earth is a single point of failure. Some customers are willing to take the risk of a single-homed situation. Some will have multiple circuits from the same carrier and POP which is good enough most of the time. An optics or electronics failure or bad copper loop will survive on the backup but a physical cable cut will break everything in that cable. Classic example of unexpected failure mode, local telecom carrier that I won't name but it rhymes with Tom King's callsign suffix. These guys do a good job of ensuring that their ring is pretty much uncollapsed. Picture a ring of fiber going from a data center to customers 1, 2, 3, 4, 5, 6, 7, 8 and back from 8 to the data center in a circle, with another pair of fibers going in the opposite direction - 8, 7, 6, 5, etc. Both directions are in the same cable but it's for the most part in a circle so that only one side will be cut in a specific incident. If there's a cut for example between 3 and 4, then equipment both at 3 and at 4 loops the return traffic back the direction where it originated. It's pretty robust and would require two separate cuts to isolate anyone. During the Gap Fire, there were several power outages lasting for up to a few hours each as the smoke caused the HV lines coming down the mountain to arc over. Imagine that you are customer 4. You have a backup generator and robust UPS. The carrier at their data center also has a generator and robust UPS. However, customers 3 and 7 are shops with just a small battery. When the power fails over a wide area, the batteries at both customers 3 and 7 die, their optical transceivers go dark, and this isolates 4, 5, and 6 from the data center. Further, 3 and 7 are businesses that have the gear in an interior closet and they have locked up and gone home because the power is out, hence no access to the gear. Whoops. The fix is to put an optical device with a mirror and electromagnet at all of your customer sites. When the equipment fails or the power dies, the magnet drops out and bridges the in and out fibers. But, this gear is expensive, it's mechanical and can break, the type of failure is rare, and most of your customers go home during an extended power outage so the business case for the mirror mechanism at every customer doesn't pencil out. After the fact and getting read the riot act from customer 4, you provision another pair through the ring just to customer 4 without drop-and-insert at the others. This too doesn't scale from a business standpoint but solves this particular problem for this customer this time (after the outage). Bottom line, each extra "9" adds ten times the reliability at typically ten times the cost and there will be gotchas out there to bite you. Murphy is alive and well. -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Hi Jay, You might be interested in talking to April Moell, WA6OPS since she is in charge of the Hospital Disaster Communications down there. While not directly related to large area communications disruptions, they do handle communcations between multiple hospitals when that type of communications get disrupted and outside help is needed. I talked to Joe last night, and she may or may not be at the Bonelli ARDF practice on the 21st. Thanks! Marvin On 3/13/2015 9:52 AM, Jay Hennigan wrote:
For what it's worth, I sent the following to the Scottsdale ARC and ARES, I will report their response:
Subject: Amateur radio utilization during Centurylink outage
Message: Greetings!
I'm WB6RDV, President of the Santa Barbara Amateur Radio Club in California.
There's an ongoing discussion among area hams about the use of ARES and amateur radio during emergencies. We conduct regular nets and drills, but are very rarely mobilized by local government during emergencies.
One school of thought is that these events are rarely emergencies in terms of telecommunications infrastructure. As hams we aren't firefighters, paramedics, etc. As long as the regular communications networks are functional and not vastly overloaded, even if there is a major emergency, it isn't a communications emergency.
The Centurylink fiber cut in Arizona was a different beast. It was a telecommunications emergency and only a telecommunications emergency.
Both personally and as president of SBARC I would like to hear how, if at all, the local amateur community was utilized during that event and what lessons if any came out of it.
Vy 73, WB6RDV
-- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV _______________________________________________ SBARC-list mailing list SBARC-list@lists.netlojix.com http://lists.netlojix.com/mailman/listinfo/sbarc-list
Here is the response from the Arizona ARES SEC: On 3/13/15 12:05, Dennis - KE7EJF wrote:
Jay,
Your e-mail has been sent to me as I am the ARES Section Emergency Coordinator for the Arizona Section.
There was no ARES or ham radio activation as this was not a "telecommunications emergency". The outage affected a portion of Northern Arizona and not the Phoenix Metro Area. It did not effect local phone service only long distance, most internet and cell phone services. Verizon cellular voice service were unaffected though data was disrupted in some areas. I'm sure the media made it much worse that it actually was.
I was in touch with my ARES Field Staff in the affected areas and they were ready to provide support had they been requested.
If you need any other information please don't hesitate to ask.
Dennis Bietry - KE7EJF ARES Section Emergency Coordinator / Arizona
-- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
I received another reply from an Arizona ARES official, copied verbatim below: Hi Jay, I am Bud Semon, N7CW, the DEC for Yavapai Co., which is just north of Maricopa Co. where Phoenix is located. The cut to the fiber optic cable occurred not too far from the border of the 2 counties and our county was impacted, along with some counties further north. This incident was not really a communications emergency. The local monopoly Internet provider, Cableone, did lose their Internet access. The smaller cell providers lost service - Verizon was not impacted. The local landline phone provider lost some long distance service. Personally, I have Verizon service and an app on my phone that allows me to use cellular data for my computer, so I lost Internet for 5 minutes or so. Unfortunately, many local businesses did not have an equivalent backup, nor did the local 911 service. We received reverse-911 calls notifying us of the 911 outage - in fact we received 2 calls on our landline (local service was still working), 2 on my wife's cell phone, 2 on my cellphone and an email. Of course, all the folks with VOIP phones didn't receive those calls until service was restored, which occurred 3 or 4 hours later. To my knowledge, there was no effect on radio communications for law enforcement, fire, EMS, etc. The local hospitals lost Internet service but maintained their Intranet between their campuses. We were not activated by the county OEM or by any other agency. Initially, there was a significant amount of incorrect information about the cut cable, from the media. As you might expect, I received a number of phone calls from local members, asking why we weren't in an emergency, particularly from those without a reasonable backup plan for phone and Internet. We did not activate a net, because I felt there was no reason - there was no one asking for backup communications. I did take advantage of the outage to test our local VHF Winlink Gateways - they should have been configured to accept and hold emails until the Internet was restored - they were not. That was my fault - I missed a setup parameter, which is now fixed. I also took advantage of this situation as a teaching moment for our members. The usual stuff - were your personally prepared? Do you have backups for what you consider important functions? Have you registered for the reverse-911 system? Did you test your ability to reach the Internet beyond the affected area (e.g. HF Winmor)? And, most importantly - we do not self-deploy. I have copied Joe, W7LUX, on this email. He is the ARES guru for Coconino Co. - our northern neighbor. Maybe he has something to add. I hope that helps. Let me know if you have any questions. 73, Bud N7CW Yavapai County ARES/RACES DEC/Radio Officer -- Jay Hennigan - CCIE #7880 - Network Engineering - jay@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
participants (8)
-
Andrew Seybold -
Jay Hennigan -
Larry Martinez -
Marina King -
Marvin Johnston -
Michael Kwan -
Rod - Traveling -
Thorsten von Eicken