[olug] Linux networking weirdness

[Obi-Wan]

Hey folks,

My home Internet stopped working suddenly last Friday night, and I'm at 
a loss to explain what I'm seeing.  It was an instantaneous failure, not 
a slow degradation, and nobody was doing anything on my firewall at the 
time.  The kids were just web browsing on their tablets, which is how we 
first saw the problem.  If any of you have any suggestions after reading 
this entire treatise, I'd love to hear them.  Here's what I think I know:

Normal setup:  Internet comes wirelessly via a Future Tech radio dish on 
my roof.  An ethernet cable (with POE) connects the radio to my 
firewall, which is a dedicated Linux server.  Only the POE power 
injector sits between the two.  The firewall has a static public IP 
address on a /25 network that sends traffic to a gateway at my ISP's 
site.  The firewall runs IPtables and handles NATting / DNS / DHCP for 
my home LAN.

Problem symptoms:

My LAN (both wired & WiFi) can still reach the firewall from the inside 
just fine.  The firewall can no longer reach the ISP's gateway IP or 
hence the Internet at large.  TCPdump on the firewall's external NIC 
shows repeated unanswered ARP requests for the gateway from my 
firewall.  I tried turning off IPtables entirely, but that had no effect 
on my firewall's ability to see the outside world.  The firewall's 
external NIC still shows link lights and traffic flashing.  I've tried 
replacing all the short cables, and the visible portion of the long 
cable running from my roof to my basement shows no visible damage.  I've 
tried powering down & un/re-plugging all the related equipment, but to 
no effect.

The ISP can connect to the rooftop radio from the outside, so that link 
to my house seems to be good.

If I disconnect my firewall from the radio and plug my linux laptop 
directly into the radio (configuring it to have the firewall's static 
IP), then my laptop can get out to the Internet just fine. That seems to 
indicate that the POE injector, the long cable, and the gateway 
configuration are fine.  Physical distances forced me to use a different 
cable to connect my laptop to the POE injector than I use to connect the 
firewall to the POE injector.

If I connect my laptop directly to the external NIC on my firewall using 
a crossover cable (configuring my laptop to be a different IP on the 
external /25 subnet), then the laptop & the firewall can communicate 
with each other just fine.  That seems to indicate that the firewall is 
working just fine.

If I connect the rooftop radio directly into my LAN switch (bypassing 
the linux firewall) and let the radio handle NAT / DHCP on a 
non-routable subnet that it provides, then the rest of my LAN can get to 
the Internet at large, but at an unusably slow speed (240 Kbps 
download).  That's how I left things at the moment.  I didn't have to 
change any config on the radio to make this happen, so apparently it's 
able to do this and serve my normal static IP simultaneously.

If both the radio and the firewall both test fine, and the cable between 
them has already been replaced, why isn't this working? What else should 
I be looking at?

I tried calling Future Tech's phone support on Saturday, but I could 
hear the guy's eyes glazing over when I described my normal setup with a 
linux firewall.  He wasn't able to offer any suggestions.

As I type this, it occurs to me that it's *possible* (though highly 
unlikely) that *both* the cables I tried using to connect the POE 
injector to the firewall could be bad.  I'll have to verify that when I 
get home tonight.  In the mean time, I'm at a complete loss.

-- 
*Ben "Obi-Wan" Hollingsworth* obiwan at jedi.com <mailto:obiwan at jedi.com> 
www.Jedi.com <http://www.jedi.com>
The stuff of earth competes for the allegiance I owe only to the
Giver of all good things, so if I stand, let me stand on the
promise that You will pull me through. /-- Rich Mullins/