June 5th, 2013 (01:00PM PDT)

Keyway : All services are fully operational.

All services are fully restored as of 06/5/2013 1:00PM.


June 5th, 2013 (12:30PM PDT)

Keyway : DSL Service Interruption - ATT Southern CA service areas

Some DSL customers in Southern California ATT areas are experiencing a service interruption as of 06/05/2013 10:00AM. We are working with ATT to restore service to these clients, but do not yet have an ETA for repair.



April 28th, 2013 (10:00AM PDT)

Keyway : UPDATE: Ontario Datacenter Service Impairment - DDOS

UPDATE: 10:00AM: All services are now fully operational, but the DDOS attack is ongoing. We have successfully placed firewall rules and reactive blocking which is mitigating the attack to the point where services are no longer impaired.


April 28th, 2013 (07:15AM PDT)

Keyway : UPDATE: Ontario Datacenter Service Impairment - DDOS

UPDATE: 7:15AM: The DDOS attack is affecting our BGP sessions causing complete routing loss to the Ontario datacenter for durations of 2 minutes spaced 10-15 minutes apart. We do not have an exact ETA at this time, but expect resolution within a few hours.


April 28th, 2013 (06:00AM PDT)

Keyway : Ontario Datacenter Service Impairment - DDOS

Our Ontario Datacenter is experiencing a massive DDOS attack that began around 5:30AM today, 4/28/2013. The attack appears to be targetting Wordpress and SMTP services. Customers may experience slow, or brief interruptions to services.



April 11th, 2013 (05:00PM PDT)

Keyway : All services are fully operational.

All services are fully restored as of 04/11/2013 5:00PM.



April 11th, 2013 (12:00PM PDT)

Keyway : UPDATE: DSL Service Interruption - Verizon Southern CA service areas

UPDATE 12:00PM: Some DSL customers in Southern California Verizon areas are experiencing a service interruption as of 04/11/2013 7:00AM. Verizon has identified the failed components in the datacenter causing the outage. They did not have spare stock on site and are transporting replacements to the datacenter. No official ETR has been provided by Verizon for this issue. Unofficially, a Verizon technician stated he thought the reapirs would be completed by 2PM today, 4/11/13.


April 11th, 2013 (07:00AM PDT)

Keyway : DSL Service Interruption - Verizon Southern CA service areas

Some DSL customers in Southern California Verizon areas are experiencing a service interruption as of 04/11/2013 7:00AM. No ETR has been provided by Verizon for this issue.

This outage appears to be relate to the outage from Tuesday, 4/9/13. The customers who are affected today were also affected Tuesday. No ATT or Covad areas appear to be affected and only a smaller portion of the Verizon clients appear to be impacted. Verizon has been notified of the issue but has not provided us with an estimated time to repair (ETR) yet. We will post updates here as we receive them.



April 10th, 2013 (02:00AM PDT)

Keyway : All services are fully operational.

All services are fully restored as of 04/10/2013 2:00AM.



April 9th, 2013 (02:15PM PDT)

Keyway : UPDATE: DSL Service Interruption - Southern CA

UPDATE 02:15PM: Multiple DSL outages are occuring in the Southern California Area as of 04/09/2013 8:30AM ETR is today, 4/9/2013.

Both telephone companies have advised us that they are rolling trucks to multiple outage locations in Southern California to repair damage to lines and poles caused by recent high winds. They have advised us that they expect most services to be restored no later than early evening today. We have been advised that some outages may be related to power outages occurring at the same time.


April 9th, 2013 (08:30AM PDT)

Keyway : DSL Service Interruption - Southern CA

Multiple DSL outages are occuring in the Southern California Area as of 04/09/2013 8:30AM No ETR.

We have identified multiple outages affecting customers in the Southern California area. We do not have an estimated time for repair as yet.



August 10th, 2012 (09:12AM PDT)

Keyway : All services are fully operational.

All services are fully restored as of 08/10/2012 9:12AM.



August 9th, 2012 (05:30AM PDT)

Keyway : Dial Up Service Interruption

Dial Up outage confirmed in Southern California Area as of 08/09/2012 5:30AM No ETR.

We have identified a major dialup outage affecting customers in the Southern California area. We do not have an estimated time for repair as yet.



August 2nd, 2012 (06:00PM PDT)

Keyway : All services are fully operational.

All services are fully restored as of 08/02/2012 6:00PM.



August 2nd, 2012 (02:30AM PDT)

Keyway : DSL Service Interruption

Verizon outage confirmed in "Southern California Area" as of 08/02/2012 2:30AM No ETR.

We have identified a major DSL outage affecting Verizon DSL customers in the Southern California area. We do not have an estimated time for repair as yet.



June 8th, 2012 (01:00 am PDT)

Keyway : All services are fully operational.

All services are fully restored as of 06/08/2012 1:00 am.



June 7th, 2012 (01:00 pm PDT)

Keyway : DSL Service Interruption

Verizon outage confirmed in "Southern California Area" as of 06/07/2012 1:00 pm. No ETR.

We have confirmed at least 75 DSL customers are being affected by this issue. At approximately 11:00 am on Monday, June 4th some Keyway DSL customers in the Verizon area began experiencing intermittent connectivity. The trouble began as brief 1-5 minute outages every few hours. Over the past three days the problem has worsened and now is occurring for 2-10 minutes as often as three to four times an hour.

We have confirmed with Verizon that the problem is in their internal network. Verizon's original ETR (Estimate Time to Repair) was 1AM 6/7/12. That ETR has passed and we have not yet received a revised ETR. As we receive updates, we will update this page and our greeting message on our support telephone line.



February 10th, 2012 (07:00 am PT)

Keyway : DSL Service Interruption

Verizon outage confirmed in "Southern California Area" as of 2/10/2012 7:00 am. No ETR given by Verizon

At approximately 07:00 am 2/10/2012 a large number of DSL customers experienced partial or complete loss of Internet service. We have confirmed with Verizon that the problem is in their internal network. Although Verizon has not provided an ETR (Estimate Repair Time), we expect the issue to be repair quickly due to the large number of affected customers. As we receive updates, we will update this page and our greeting message on our support telephone line.

UPDATE: 8:30 PM 2/10/2012 Verizon is reporting all issues have been resolved.



December 19th, 2011 (01:00 am PT)

Keyway : Service Interruption

At approximately 1:00AM 12/19/11 one of our smaller mail servers stopped functioning during a security patch installation. We are working with the vendor to repair the issue. This issue is affecting email service for about 20 clients. No websites are down.

12:00PM UPDATE: If we cannot resolve the issue within the next few hours, we will begin the process of migrating the affected clients to a redundant server.

1:30PM UPDATE: The repairs have been completed successfully and the server is now normal.



 

June 30, 2011 (9:00 am PT)

Keyway : Server maintenance

At approximately 5:00 am, one of our primary mail servers began experiencing difficulties. We immediately began performing maintenance and repair. This could be affecting the email of certain customers. We are working to resolve the issue and expect to have it fully resolved shortly. Thank you for your patience.

 

December 24, 2009

Office Hours : 09/07/09

Keyway's business office will be closed on Friday 12/25/2009 in observance of Christmas Day, and on Monday 01/01/2010 in observance of the New Years Day holiday.

Clients with premier support services should use their 24/7 contact numbers.

September 7, 2009

Office Hours : 09/07/09

Keyway's business office will be closed on Monday 9/07/2009 in observance of Labor Day. Clients with premier support services should use their 24/7 contact numbers.

 

July 2, 2009

Office Hours : 07/03/09

Keyway's business office will be closed on Friday 07/03/2009 in observance of Independance Day. Clients with premier support services should use their 24/7 contact numbers.

 

February 16, 2009

Office Hours : 02/16/09

Keyway's business office will be closed on Monday 2/16/2009 in observance of President's Day.  Clients with premier support services should use their 24/7 contact numbers.

 

February 12, 2009

Keyway : Server outage

At approximately 9:00 am, one of our primary servers began experiencing difficulties. This could be affecting the email and/or web sites of certain customers. We are working to resolve the issue and expect to have it repaired shortly. Thank you for your patience.

 

January 26, 2009

Level 3 : Network outage

At approximately 1:15 PM, one of our primary backbone T3 connections went offline. We have opened a trouble ticket with our backbone provider, who has projected a 30 minute window for repair. We will continue to work with them to troubleshoot this issue, and will provide updates here when available.

 

September 24, 2008

 Network Connectivity Issues : Possible DoS attack

Some of our clients may experience intermittent difficulty with their services. We are seeing random and widespread outages of very short durations. They appear to be lasting from one to fifteen minutes and are not limited to any specific service. We suspect the network is under a large scale "denial of service" attack and are working to identify and isolate it.

 

October 23 , 2007

cPanel Maintenance : Server Software Upgrade

We will be upgrading our cPanel web servers' software today after 5pm. Web hosting customers can expect approximately 15 minutes of downtime.  We thank you for your patience during this maintenance window.

 

October 23, 2007

SBC DSL : Network maintenance

Our SBC carrier will be swinging all of our DSL subscribers in the Los Angeles area to a new POP at 12:01 a.m. on Tuesday, October 23rd. We expect approximately one hour of downtime as circuits are migrated. This deployment brings with it decreased latency to the Internet for our LA customers.  We thank you for your patience during this maintenance window.

 

July 11, 2007

Level 3 : Network outage

At 2:49 PM, one of our primary backbone T3 connections experienced a two and a half minute outage. We have opened a trouble ticket with our backbone provider, and will schedule emergency testing after midnight on
07-12-07, pending the results of the monitor which is now in place.

 

June 23, 2007

Emergency Maintenance: Keyway Mail and Dialup Servers

1:30 pm -- Customers using mail.keyway.net to send or receive email might experience a temporary delay or an inability to connect while we perform unscheduled maintenance on that mail server.   Also, customers dialing in might experience login failures while we perform maintenance on our authentication server. We expect the maintenance to last no more than half an hour, and apologize for any inconvenience this might cause.

 

June 22, 2007

Emergency Maintenance: DSL Maintenance

12:01am on Friday, June 22nd -- Our SBC carrier will be performing emergency maintenance on a portion of their DSL aggregation infrastructure in San Francisco. Customers in Los Angeles as well as a large portion of the LATA 1 customers will be affected by this operation. While expected downtime is approximately 15 minutes, they are reserving a one hour maintenance window in case of further trouble.   We thank you for your patience and apologize for any inconvenience this might cause.

 

March 23, 2007

Level 3 : Network outage resolved

All services have been restored as of 3:45PM today. The total outage duration was one hour and forty-five minutes. The outage was caused by a failed Verizon fiber-optic high speed line. Keyway's engineers will be researching ways to avoid future similar outages and add more redundancy to our network.

 

March 23, 2007

Level 3 : Network outage

Keyway Internet Services is currently experiencing an outage with our connection to the rest of the Internet. The outage occured at 2PM today. The outage is somewhere in the fiber-optic high-speed circuit connections that connect our network to the Internet. Keyway Engineers are currently working with Engineers to locate and repair the issue. We do not yet have an official Estimated Repair Time but the issue appears to be minor and will likely be repaired within an hour or so.

 

February 19, 2007

Office Hours : 02/19/07

Keyway's business office will be closed on Monday 2/19/2007 in observance of President's Day.  Clients with premier support services should use their 24/7 contact numbers.

 

October 27, 2006

Scheduled Maintenance: Los Angeles SBC Area DSL

On Friday, October 27th at 12:01 a.m., our SBC carrier will be performing routine maintenance on the equipment that aggregates our Los Angeles SBC area DSL customers. This operation is intended to increase performance and reliability for all our LA area customers. Downtime will be approximately 15 minutes. We apologize for any inconvenience this might cause.

 

October 5 , 2006

Emergency Maintenance: Power interruption

9:15 PM -- We have been notified by the building management at our Los Angeles Facility that they need to perform Emergency Maintenance on a power circuit there. This maintenance will interrupt service to some of our
clients in that facility. If you you believe that you are experiencing a
service interruption due to this maintenance, please call our support
department at 909.933.3699. We apologize for the short notice.

 

August 18 , 2006

Hardware failure: Power outage

In the morning hours of 8/18/06 (approx. 4:30AM), a Southern California Edison power surge tripped three of the breakers in the Keyway Ontario
Datacenter. The Los Angeles Datacenter was completely unaffected. The
power surge permanently damaged one UPS and one DNS server and temporarily disabled a backbone switch and two more UPS'. All Internet services were restored at 5AM with the exception of DSL and some T1 clients who were restored at 6:30AM.

 

August 5 , 2006

Hardware failure: Spam Filtering Outage

This morning at approximately 8am , the server which performs our spam-filtering went offline due to hardware problems.  This issue is being worked on, and we expect to have it resolved by the end of today.  We apologize for any inconvenience this may cause.

 

July 17, 2006

Keyway office - phone outage

Today at approximately noon, our office lines went down.  We have opened a trouble ticket with our telephone line carrier and hope to have the issue resolved very soon. 

If you need sales or support assistance, please email us at either sales@keyway.net or support@keyway.net.    We apologize for any inconvenience this may cause.

 

June 30, 2006

Network Issues : DNS

Today at 10AM our primary DNS server ("name server"), ns1.keyway.net stopped responding to name lookup requests due to configuration file corruption. We are repairing and determining the cause of the corruption to effect preventative measures. We have three name servers online at this time and so this should not impact your performance at all. Our ETA for repair is 11AM this morning.

If you find your connection does not seem to be performing properly, please call our technical support at 909.933.3699 for assistance.

 

June 14, 2006

SBC DSL : Network Issues

We received a report from our SBC carrier that they were experiencing network instability issues and that they are working to find the cause and resolve the problem.   We apologize for any inconvenice that this may cause.  This issue is only impacting some clients who have SBC as their phone company and have Keyway DSL, no other services are affected at this time.  The issue started today, June 14th at 10AM and was corrected shortly thereafter.

 

 

June 12, 2006

Mail Issues : DNS caching

We are receiving reports of clients unable to send or receive email when
they are using non-Keyway DNS servers. The cause is due to bad information that was distributed to the root domain servers on Sunday. The problem was identified and corrected on Sunday, but some DNS servers are caching (carrying over) the bad information. There are two solutions, first to 'refresh' the non-Keyway DNS server so that it stores the current valid data, or second, to enter a Keyway DNS server into the settings for your computer.

 

 

May 27, 2006

Mail Issues : SORBS

Some of our client's who run their own mail servers located on the IP
blocks listed below may experience some difficulty sending email to recipients who choose to use SORBS to filter inbound email. The reason for this is that SORBS has erroneously listed the affected IP addresses as "dynamically issued" when in fact they are permanently assigned. We have opened a ticket with SORBS to correct the erroneous listing but they take several days to respond to each mailnote, so we do not expect this issue to be resolved until 6/9/06.

Here are some actions you can take in the meantime:

  1. Use Keyway's primary mailhost, mail.keyway.net, to send your email. This host accepts email from any Keyway issued IP address and will allow you to remain fully operational until the SORBS issue can be resolved.
  2. Contact the recipient and ask them to either 'whitelist' you, or
  3. Contact the recipient and ask them to temporarily stop using SORBS until SORBS corrects the erroneous listing.
  4. Request a SORBS 'exclusion' entry. To request an exclusion, please contact our technical support.

Keyway has three team members dedicated full time to correcting this issue and it is our current highest priority item. 

Affected IP addresses:
----------------------
216.117.192.0 to .255
216.117.193.0 to .255
216.117.194.0 to .255
216.117.195.0 to .255
216.117.196.0 to .255
216.117.197.0 to .255
216.117.198.0 to .255
216.117.200.0 to .255
216.117.202.0 to .255
216.117.203.0 to .255
216.117.204.0 to .255
216.117.205.0 to .255
216.117.207.0 to .255
216.117.211.0 to .255
216.117.212.0 to .255
216.117.213.0 to .255
216.117.214.0 to .255
216.117.215.0 to .255
216.117.216.0 to .255
216.117.217.0 to .255
216.117.218.0 to .255
216.117.219.0 to .255
216.117.220.0 to .255
216.117.221.0 to .255
216.117.222.0 to .255
216.117.223 0 to .255

 

May 27, 2006

Scheduled Maintenance: Router Maintenance

Keyway will be performing router maintenance on 5/27/06 between 7:30 - 9 pm.  Your DSL service may be briefly interrupted. The maintenance window should not exceed the timeframe mentioned.  Thank you for your patience.

 

May 16 , 2006

Wireless Network Maintenance

At 12 Midnight (May 16-17, 2006), some wireless customers will experience a brief period of downtime, as network maintenance is performed.   We anticipate it will be complete within 15 minutes.  Thank you for your patience during this maintenance window.  We sincerely apologize for any inconvenience this might cause you.   For technical support, please contact us by emailing support@keyway.net.  

 

March 7 , 2006

Network Outage

From approximately10:15AM to 10:30AM Keyway experienced a network outage. The duration was 15 minutes. The outage was caused by an Edison power loss. Normally no network outage would have occurred as a result of an Edison outage, as all of Keyway's systems are on battery powered Uninterruptible Power Supplies (UPS). Unfortunately, during this outage, one of the UPS' supplying power to a core router failed. The router was moved to a redundant UPS system and service was restored. The UPS will be replaced and no further outages are expected. We sincerely apologize for any inconvenience this might've caused.

 

March 3 , 2006

Keyway Outage -- Office Phone Lines

We are currently experiencing an office-wide business/support phone line outage.  We believe this to be weather related, and have reported it to our phone carrier.  We sincerely apologize for any inconvenience this might cause you, and hope to have the issue resolved very quickly.   For technical support, please contact us by emailing support@keyway.net.  

 

March 1 , 2006

cPanel Server Maintenance

We will be performing maintenance on the cPanel web and email server between 4pm-5pm on March 1, 2006.  During this time, customers might experiencing difficulty logging into their admin control panels.  We thank you for your patience during this window, and hope to finish the maintenance as soon as possible.  If you need additional assistance, please call our technical support team at 909-933-3699.

 

February 20, 2006

Office Hours : 02/20/06

Keyway's business office will be closed on Monday 2/20/06 in observance of President's Day.  Clients with premier support services should use their 24/7 contact numbers.