My Cari.net dedicated server is down..and so is cari.net!
Don’t want to be bitter but I am starting to get fed up with cari.net…The dedicated server I bought with Cari.net 2 weeks ago has been down on three occasions so far!!! that’s a lot… lucky I only use it as a dev environement at this point. Imagine if that was my production server.
What sucks the most is that I wasn’t contacted at any point regarding a scheduled/unforseen outage on all three occasions. The last one was just the worst of all, my servers has not been accessible for the pat 15 minutes?! so is cari.net anyways!
That without counting the fact that it takes me sometimes 10 seconds to open an ssh session (I have a 18Mbps connection at my home office)
Let’s hope these issues are addressed promptly and a valid explanation is provided! otherwise, my cari.net adventure will have to end right here.
Shawn said,
June 4, 2008 at 1:15 pm
This is posted under out “network status” page
http://www.cari.net/network-status.html?netstat_month=3&netstat_year=2008&netstat_day=false&netstat_start=0
Datacenters: C2, C3, C4, C5
Date & Time (PDT): Thursday, Mar 13 2008, 08:51 – Friday, Mar 14 2008, 10:10
Effect on network continuity: –A variety of connectivity problems ranging from DNS to direct Internet connectivity issues.
Details
At 8:51am Cari.net’s network began experiencing significant routing and BGP issues. The problems continued until 10:10am, during which time customers experienced a variety of connectivity problems ranging from DNS to direct Internet connectivity issues.
We apologize for the outage and any inconvenience it may have caused. A full scale inquiry is being conducted and further details will be provided within the next 2 hours.
If you are still experiencing connectivity issues, please call our 24/7/365 Support Center or open a trouble ticket.
——————————————————————————–
The following information is data that has been compiled and it being provided as our 2 hour post incident update. Cari.net management and networking teams are working to evaluate this mornings connectivity issues in an effort to adequately explain and prevent further occurrences. Below is our 2-hour post incident update. The information provided is based on “first response” review of the situation. Another update will be provided before close of business today with more a more in-depth and definitive review of the situation.
*****************************************
Broadcast storm of undetermined origin caused link flapping which in turn caused HSRP and spanning tree failures. The broadcast storm apparently began in C2 data center, disrupting traffic on key corporate vlans as well as hosted servers. The C2 core router’s CPU became overloaded and inter-data center links were non-responsive, causing STP recalculations and HSRP failures. Key corporate infrastructure became inaccessible as multiple routers attempted to take over (or relinquish) gateway IPs as spanning tree calculated switching paths appeared and disappeared. The C2 router shares switching infrastructure with the C3 core and the initial state of the data center interconnections had most traffic passing through the C5 data center. The broadcast storm cascaded through both the primary and backup C5 distribution networks, leaving access switches with no egress. The broadcast storm propagated through the shared switching infrastructure of the C3 data center facility. Both primary and redundant customer colocation access routers were affected and the storm propagated to the customer access switches. As a result, many customer access devices (in the colocation cabinets) were left in a non-functioning state and required a reboot to restore services. Cisco engineers are on site to determine the root cause of the issue. In the interim we have taken the steps to deploy additional equipment and to remove certain HSRP and redundant switch paths to reduce the severity of link flapping in 100% resolution is proven.
*****************************************
As stated above, all resources are focused on this critical situation and Cari.net is leveraging other business relationships to have third party visibility and oversight into the issue resolution process. We thank you for your continued patronage as we work to provide 100% resolution to this issue.