Global Crossing - DO NOT USE.

[hostdime]

After many calls, emails, rude techs, and other things that make me angry, I have decided Global Crossing is WORTHLESS.

I live 10 miles from NAC.net Data Center in Parsippany. HostDime has purchased around 70 servers there. I got constant 16MS pings, and this was a main factor in getting alot of servers there. Starting 3 days ago pings have constantly jumped to 200MS. This is pitiful. I called Global Crossing, and a rude woman told me 'Unless I was a direct customer, she will not talk to me.' This is pitiful, and the lag time is horrible, ever try to admin a server, when it takes 10 seconds to type in ps x? HostDime is thinking of pulling all 70 servers out of NAC, due to Global Crossing. I finally got an email today, after 5 emails:


Kris,

The issue you are seeing is due to a known bandwidth issue between our network and AT&T. We have additional bandwidth on order but cannot provide an ETR.

We apoligize for the inconvenience

Thank you,
Glen G. Walker
NOC Coordinator, GDSNOC
Global Crossing, Phoenix, AZ (USA


That is sickening. I emailed for a guestimate, Hours, Days, Weeks.. Nothing. Global Crossing is sickening, I'd rather use Cogentco, no problems ever with them. See some of the problems below. I have Comcast Pro, It hops from ATT->GBLX, from ATT->GBLX=The Problem.




Pinging nac.net [207.99.0.69] with 32 bytes of data:

Reply from 207.99.0.69: bytes=32 time=238ms TTL=47
Reply from 207.99.0.69: bytes=32 time=230ms TTL=47
Reply from 207.99.0.69: bytes=32 time=243ms TTL=47
Reply from 207.99.0.69: bytes=32 time=241ms TTL=47

[Chicken]

Hmmm well, why exactly would they talk to you unless you were a direct customer? I think that's pretty standard. I'm also not so sure that those ping times are anything to worry about (nor would I use a ping time to judge much of anything other than to tell me if the machine was up... or not). You might have problems connecting, but you should be tracing (not pinging), if that is the case.

[hostdime]

Have you checked the trace route attached? It's been 17MS until 3 days ago, they admit they have a problem, and will not respond after that. Sleazy I say, If your machines are 200MS+, I pity you.

[Chicken]

Yep, saw the traceroute posted. I still don't think 240ms +/- is completely out of line. I mean, 16ms is pretty dang low, and if that's what you're expecting, then I'm not sure you're going to have an easy time finding a provider. The other question is, what are traces like from other locations? One single trace... from one location... to another... doesn't tell you too much overall.

[hostdime]

Other traces are fine. If you live in the Northeast, and happen to route from ATT->GBLX, you're fine. By the way... I live 10 miles from NAC, and since last year have had around 12ms pings, so It's not what I ask for, it's what I get.

[Adam]

IF it not "what you ask for" then hoe come in above posts you seem to "expect" them sorts of ping times?

[Adam]

WHT is fast to me (I would have said HHO but pings are turned off here...) and from my ping on my 45Mbit DS3 i get 326ms average.

And it still loads very fast. I dont know anywhere i can get 12ms pings from. The only pings i get at that speeds are other workstations and servers and equipment interconnected onto my network throughout the UK. MY fastest is 2.8ms - and thats the workstation just next door...my slowest ping for things internally on my network is 128ms and thats a server which is 8 hours up north from me.

It all depends, your 12ms pings you got before...do you an ISP with its PoP in the data center your servers at or peering or anything like that?

If anyone knows someone i can test to see <50ms ping times please post it here in reply for me to check out as i've never personally seen this low across the internet.

[hostdime]

Pinging jose.farciert.net [64.191.10.22] with 32 bytes of data:

Reply from 64.191.10.22: bytes=32 time=24ms TTL=47
Reply from 64.191.10.22: bytes=32 time=25ms TTL=47
Reply from 64.191.10.22: bytes=32 time=24ms TTL=47
Reply from 64.191.10.22: bytes=32 time=24ms TTL=47


I get that to NOCSTER in Scranton, PA

[Chicken]

Here are 4 more from various places...


Ping 64.191.10.22
[jose.farciert.net]

Round trip time to 64.191.10.22: 94 ms
Round trip time to 64.191.10.22: 109 ms
Round trip time to 64.191.10.22: 125 ms
Round trip time to 64.191.10.22: 110 ms
Round trip time to 64.191.10.22: 94 ms
Round trip time to 64.191.10.22: 93 ms
Round trip time to 64.191.10.22: 94 ms
Round trip time to 64.191.10.22: 109 ms
Round trip time to 64.191.10.22: 94 ms
Round trip time to 64.191.10.22: 94 ms


Average time over 10 pings: 101.6 ms

----------------------------

1 49 50 64.191.10.22
2 49 50 64.191.10.22
3 49 40 64.191.10.22
4 49 40 64.191.10.22
5 49 50 64.191.10.22
times (ms)
min 40
avg 46
max 50

---------------------------

Reply from 64.191.10.22 bytes=128 time=111ms TTL=119
Reply from 64.191.10.22 bytes=128 time=120ms TTL=119
Reply from 64.191.10.22 bytes=128 time=120ms TTL=119
Reply from 64.191.10.22 bytes=128 time=110ms TTL=119
Reply from 64.191.10.22 bytes=128 time=120ms TTL=119

---------------------------

PING 64.191.10.22 (64.191.10.22): 56 data bytes
64 bytes from 64.191.10.22: icmp_seq=0 ttl=53 time=75.258 ms
64 bytes from 64.191.10.22: icmp_seq=1 ttl=53 time=75.078 ms
64 bytes from 64.191.10.22: icmp_seq=2 ttl=53 time=74.994 ms
64 bytes from 64.191.10.22: icmp_seq=3 ttl=53 time=75.106 ms

--- 64.191.10.22 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 74.994/75.109/75.258/0.095 ms

[hostdime]

Ouch, I guess I'm lucky. Still getting 250+MS. BOO :(

[Chicken]

Well, 250 millisecond = 0.25 second which is pretty fast for a roundtrip packet journey. not out of the norm. range.

120 millisecond = 0.12 second
25 millisecond = 0.025 second
16 millisecond = 0.016 second

[hostdime]

I get 1KBPS down from the server also ;)

[jasonl813]

Nocster has been having problems with their peering when data gets to AT&T and already de-preferenced that data path because we were also getting 200-700MS traces when the standard from where I am is 60MS. They also attribute it to be an AT&T routing issue.

Nocster's SLA also states a below 85MS a month average for network performance.

[hostdime]

It's not NOCster, I still get +/- 20MS from them, It's ATT->GBLX.

[jasonl813]

It's not NOCster, I still get +/- 20MS from them, It's ATT->GBLX.

Exactly, I'm just saying that NAC.net is not the only one having this problem with the AT&T peering.

On Sep 26, 2003, BurstNet Network Engineering de-pref'ed outbound BurstNET->GlobalCrossing->AT&T path in New York Metro area while attempting to bypass overrun of AT&T<->GBLX PNIs in New York metro.

Took that from Nocster's forum.

[tranz]

Just for arguements sake. What do you get from one of my servers?

69.28.64.130

[Adam]

Pinging 64.191.10.22 with 32 bytes of data:

Reply from 64.191.10.22: bytes=32 time=304ms TTL=47
Reply from 64.191.10.22: bytes=32 time=304ms TTL=47
Reply from 64.191.10.22: bytes=32 time=300ms TTL=47
Reply from 64.191.10.22: bytes=32 time=296ms TTL=47

Ping statistics for 64.191.10.22:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 296ms, Maximum = 304ms, Average = 301ms

[tranz]

Adam,

This is what I get for the IP you tested.


==================================================
=== VisualRoute report on 27-Sep-03 4:39:26 PM ===
==================================================

Report for 64.191.10.22 [jose.farciert.net]

Analysis: '64.191.10.22' [jose.farciert.net] was found in 17 hops (TTL=44).

-----------------------------------------------------------------------------------------------------------------------------
| Hop | %Loss | IP Address | Node Name | Location | Tzone | ms | Graph | Network |
-----------------------------------------------------------------------------------------------------------------------------
| 0 | | 192.168.1.100 | hppav.cfl.rr.com | - | | | | (private use) |
| 1 | | 10.110.160.1 | - | ... | | 11 | x- | (private use) |
| 2 | | 24.95.225.109 | pos0-1.orldflsnfd-rtr1.cfl.rr.com | - | | 8 | x | 24.95.225.0 |
| 3 | | 24.95.224.73 | srp2-0.orldflwrpk-rtr2.cfl.rr.com | - | | 9 | x | 24.95.224.0 |
| 4 | | 24.95.224.10 | srp4-0.orldfldvsn-rtr4.cfl.rr.com | - | | 8 | x | 24.95.224.0 |
| 5 | | 66.185.136.173 | pop1-tby-P0-2.atdn.net | | | 29 | -x-- | 66.185.136.0 |
| 6 | | 66.185.136.160 | bb1-tby-P0-0.atdn.net | | | 18 | x | 66.185.136.0 |
| 7 | | 66.185.152.245 | bb2-atm-P7-0.atdn.net | | | 35 | x | 66.185.152.0 |
| 8 | | 66.185.152.31 | bb2-cha-P6-0.atdn.net | | | 41 | x- | 66.185.152.0 |
| 9 | | 66.185.152.50 | bb2-ash-P13-0.atdn.net | | | 52 | x- | 66.185.152.0 |
| 10 | | 66.185.148.211 | pop3-ash-P1-0.atdn.net | | | 73 | -x------- | 66.185.148.0 |
| 11 | | 66.185.136.6 | XO.atdn.net | | | 52 | x | 66.185.136.0 |
| 12 | | 65.106.3.137 | p5-0-0.RAR2.Washington-DC.us.xo.net | - | | 58 | -x | 65.106.3.0 |
| 13 | | 65.106.3.154 | p4-0-0.MAR2.Philadelphia-PA.us.xo.net | - | | 57 | x- | 65.106.3.0 |
| 14 | | 207.88.87.46 | p15-0.CHR1.Philadelphia-PA.us.xo.net | - | | 58 | -x | 207.88.87.0 |
| 15 | | 66.236.204.10 | 66-236-204-10.hostnoc.net | | | 56 | x- | 66.236.204.0 |
| 16 | | 66.197.191.45 | - | | | 63 | x | 66.197.191.0 |
| 17 | | 64.191.10.22 | jose.farciert.net | | | 61 | x | 64.191.10.0 |
-----------------------------------------------------------------------------------------------------------------------------
Roundtrip time to 64.191.10.22, average = 61ms, min = 61ms, max = 66ms -- 27-Sep-03 4:39:26 PM

[inogenius]

To Tranz server...
----------------------------------------------------------------------------
Ping statistics for 69.28.64.130:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 64ms, Maximum = 117ms, Average = 68ms

To HostDime's server
----------------------------------------------------------------------------
Ping statistics for 64.191.10.22:
Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 54ms, Maximum = 58ms, Average = 55ms

[sigma]

Originally posted by hostdime:
That is sickening. I emailed for a guestimate, Hours, Days, Weeks.. Nothing. Global Crossing is sickening, I'd rather use Cogentco, no problems ever with them. See some of the problems below. I have Comcast Pro, It hops from ATT->GBLX, from ATT->GBLX=The Problem.


I'm coming in late on this thread, but I do have some relevant information. This issue was on NANOG around September 25th. Global Crossing was reporting that AT&T was refusing to upgrade the peering between their networks, citing GC's financial status (bankruptcy protection) as the reason.

When two large providers have a peering session reaching saturation, they usually upgrade well ahead of time. When that doesn't happen, it's usually because of politics on one side or the other. Each side will blame the other. Note that it doesn't matter in which "direction" it's saturated; the users on both networks have paid for end-to-end transit and both networks are responsible for providing that.

I don't know if this situation has been resolved, but over the years, these spats usually last a week or two and then politics gives way to angry customers, and the peering gets upgraded.

It's very unfair to assume that Global Crossing is solely at fault. The reports indicated quite the opposite, and I suspect that may be true, but at any given time, any provider might play politics. None of them are saints in this regard.

Kevin

[rusko]

att is notorious for being hardasses about this. a while ago they had peering capacity issues with level3 in the ny area, i believe. not much you can do.

paul