Your internet BB speed ?

[Jaded]

Where I is now

[Kim]

Just got the L2TP failover working properly with the FireBrick.  Had to drop the MTU to 1406 to prevent broken IPv6 for some reason... (?IPSEC in the cellular provider's network)

Anyway https://speedtest.aa.net.uk/ reports 112Mbps down / 19.4Mbps up, with a ping of 6.90/0.27ms running on the FTTP connection.  On the L2TP tunnel that drops to a stunning 3.12Mbs down / 6.02Mbps up, with a ping of 67.5/31.9ms.  The LTE modem reports that it's currently^[1] connected via O2, who are apparently a rubbish.  Still, it's better than dialup.


_{[1] It's got a groovy funky prepayed data roaming SIM in it, so can switch network as the whim arises.}

[robgul]

Just got the L2TP failover working properly with the FireBrick.  Had to drop the MTU to 1406 to prevent broken IPv6 for some reason... (?IPSEC in the cellular provider's network)

Anyway https://speedtest.aa.net.uk/ reports 112Mbps down / 19.4Mbps up, with a ping of 6.90/0.27ms running on the FTTP connection.  On the L2TP tunnel that drops to a stunning 3.12Mbs down / 6.02Mbps up, with a ping of 67.5/31.9ms.  The LTE modem reports that it's currently^[1] connected via O2, who are apparently a rubbish.  Still, it's better than dialup.


_{[1] It's got a groovy funky prepayed data roaming SIM in it, so can switch network as the whim arises.}

Is there an English transalation of all that? - I thought the trick for top performance was just to keep the string wet.

[Feanor]

Just got the L2TP failover working properly with the FireBrick.  Had to drop the MTU to 1406 to prevent broken IPv6 for some reason... (?IPSEC in the cellular provider's network)

Anyway https://speedtest.aa.net.uk/ reports 112Mbps down / 19.4Mbps up, with a ping of 6.90/0.27ms running on the FTTP connection.  On the L2TP tunnel that drops to a stunning 3.12Mbs down / 6.02Mbps up, with a ping of 67.5/31.9ms.  The LTE modem reports that it's currently^[1] connected via O2, who are apparently a rubbish.  Still, it's better than dialup.


_{[1] It's got a groovy funky prepayed data roaming SIM in it, so can switch network as the whim arises.}

Is there an English transalation of all that? - I thought the trick for top performance was just to keep the string wet.

The Firebrick firewall / router devices provide for a backup Internet connection if the primary one goes down.
The primary Internet connection comes in to an Ethernet port from a DSL Modem or FTTP ONT typically.
But there's a USB port where you can plug in a 3/4g dongle, to use mobile data as a backup if the line goes down.

So far so ordinary. But here's the clever bit:
AAISP let you connect to them *via* the 3rd party mobile ISP.
So under failure conditions, you are switched to the backup mobile ISP.
But you don't go out onto the Internet directly via the mobile ISP, using their IP addresses etc.
Instead, you use this mobile ISP as a conduit to set up a Tunnel into AAISP, and all your traffic then passes through AAISP, exactly the same as it did when the primary line was up.
This is called an L2TP tunnel.
The mobile ISP carries only the tunnel traffic.
The actual user traffic is inside the tunnel, and has no idea it is passing through the mobile ISP on it's way to AAISP.

Why would you do this, instead of just using the mobile ISP directly?
Because this way, you retain all your AAISP IP addresses, and routing.
So any servers you have on your end of the connection retain their addresses and routing, and remain contactable from the Internet.

I am doing something similar, using an L2TP tunnel into AAISP via Starlink.

Only thing I find surprising is the FTTP speeds of 100Mbps. I have never investigated FTTP beyond knowing I can't get it, but 100Mbps seems rather, er, pedestrian, no?
Is it a case of you get what you choose to pay for?

[Mr Larrington]

Only thing I find surprising is the FTTP speeds of 100Mbps. I have never investigated FTTP beyond knowing I can't get it, but 100Mbps seems rather, er, pedestrian, no?
Is it a case of you get what you choose to pay for?

Mine is ~80 Mbps with AAISP & FTTP.  My Yorkshire soul considers this adequate VFM

[Kim]

Why would you do this, instead of just using the mobile ISP directly?
Because this way, you retain all your AAISP IP addresses, and routing.
So any servers you have on your end of the connection retain their addresses and routing, and remain contactable from the Internet.

The other benefit is that if the changeover happens quickly enough^[1], it can happen with only a bit of momentary packet loss.  So any TCP connections that are in progress tend to stay up, rather than being dropped or timing out and having to be re-established on another route.  I haven't tried it during a video call, but I imagine you get a couple of seconds of potatovision, which is the sort of thing that people with Hearing People's Crap Internet™ are used to anyway.

In the event that the problem is due to the ISP having a really bad day^[2] and the L2TP doesn't work either, it will fall back to NATing everything down the backup connection.  Which is pants because it tends to mean that the IPv6 internet becomes unreachable, but it's better than having to get your phone out.

Only thing I find surprising is the FTTP speeds of 100Mbps. I have never investigated FTTP beyond knowing I can't get it, but 100Mbps seems rather, er, pedestrian, no?
Is it a case of you get what you choose to pay for?

Yep.  115/20^[3] is the cheapest our-favourite-telco option, and we see no reason to pay for more as 20Mbps upload is perfectly sufficient for a couple of high-bandwidth video calls, or an overnight backup job of a few gig.  It's not like we've got a house full of gamers.  (It's slightly cheaper and nearly twice as fast as the VDSL we had before, which was adequate for all but really big backup jobs.)



_{[1] Pfsense was bad at this, as it relies on ICMP pings to determine whether a gateway is reachable, and you can't be too aggressive with the timeouts lest it switch spuriously.  The Firebrick can monitor the state of the PPPoE connection directly.
[2] Many years ago, when broadband speeds were lower and the ISP's end was correspondingly a lot less complicated, AAISP had a massive outage due to a single point of failure (I forget the details, but vaguely recall some sort of layer-2 switch that wasn't normal Ethernet).  They learned from that one.
[3] We have some traffic shaping at both ends that sacrifices a few percent of the total in favour of making small packets (interactive stuff, VOIP, etc) get first dibs in the queueueueue, hence seeing 112/19 in the speed test.  AAISP can do this on the downstream for everyone from their end, but you need a router with some sort of shaping capability to do it on the upstream, which is where it's most useful.}

[Vernon]

Just had FTTP installed at the new house in the wilds of Auckland.
getting 600 Mbps down, 300 Mbps up.
I think that'll do for most purposes.

[Kim]

Just got the L2TP failover working properly with the FireBrick.

For small values of properly:

After application of some more advanced voodoo, it should now be able to handle the not-uncommon situation where some Openreach problem means the PPPoE session terminates at the BRAS, leaving you with a PPP link that's up and a default route to nowhere.   

(click to show/hide)

Testing for this is non-trivial if you've got a L2TP session that comes along and provides the same routes that the PPPoE is supposed to, leaving you with a race condition where the test fails, it brings the L2TP up, the test passes, so it tears the L2TP down, the test fails again, rinse, repeat.

In the absence of some sort of "ping via this interface" test option (which seems like rather a conspicuous omission), I worked out how to do it by putting the PPPoE in its own routing table^[1] where routing and ping tests can be performed without the L2TP complicating matters.  Then you just have to craft a ruleset that jumps packets between tables (in both directions, otherwise your servers are unreachable) when the connection is up...


_{[1] Fibrebrick supports more than one routing table, which is surprisingly useful when manipulating tunnels for fun and profit.}

[Feanor]

Firebrick supports more than one routing table, which is surprisingly useful when manipulating tunnels for fun and profit.

Which is pretty much essential to avoid routing the tunnel up it's own ass like a snake trying to eat itself...

[Kim]

Firebrick supports more than one routing table, which is surprisingly useful when manipulating tunnels for fun and profit.

Which is pretty much essential to avoid routing the tunnel up it's own ass like a snake trying to eat itself...

*nix manages without.  Of course, that allows you to specify an interface (rather than just a gateway) for a route...

[Feanor]

Firebrick supports more than one routing table, which is surprisingly useful when manipulating tunnels for fun and profit.

Which is pretty much essential to avoid routing the tunnel up it's own ass like a snake trying to eat itself...

*nix manages without.  Of course, that allows you to specify an interface (rather than just a gateway) for a route...

Yes, you can do that in windows too, and I last looked at that umpty years ago when I was tunnelling IPv6 before it was native.

For the FB, I've not needed to delve much deeper than the advice offered here:
https://support.aa.net.uk/L2TP_Client:_FireBrick

I'm not using L2TP as a fallback, but as a primary route, so that makes things a bit simpler.

One thing I'd love on the FB would be a mini-TFTP and mini-HTTP server to serve up config files to phones so I could shut down some more boxes.

[Kim]

So far, coming from pfsense, the conspicuous omissions of the FireBrick seem to be:

- Ability to specify an interface in routing tables and tunnels (but multiple routing tables can achieve the same thing with a bit of thought).
- Ability to specify IP ranges by DNS name.  Yes, it's not ideal to do this, but neither is eg. having to change your firewall rules because the octopus API moved to a different corner of AWS and now the fridge doesn't know what electricity costs.
- Ability to tell it to "only forward DNS requests for *this* domain to *this* specific server", which is useful for reverse-lookups of RFC1918 addresses and spoofing things for internet-of-shit devices.  Presumably the individual entries could be included in the XML with a bit of scriptage.
- Much cruder, but also *much* less buggy, traffic shaping.  You can allocate arbitrary flows to a queueueue with a rate limit, but you can't say "this traffic gets priority over that traffic".  It seems more designed for enforcing bandwidth quotas than ensuring quality of service.
- mDNS bridging (a bit niche, and easily solved with avahi in a container).

Meanwhile:
- Compact, energy-efficient hardware.
- Incredibly slick web UI, quick to apply config changes, boots in a couple of seconds.
- Profiles, which are a killer feature for doing failover.
- CQM graphs.
- Versatile approach to logging.
- MQTT support.
- SNMP counters that don't lie.


A lightweight TFTP/HTTP server is a good 'nice to have'.  Perhaps the USB port could be used to store the files if they were non-trivial...
I'd also like the config download to include a timestamp in the filename.  I'll write a script for archiving backups at some point.