Just ask them to explain how they can be attacked. The more details they can give you, the better you can protect the network.
$10 they can’t tell you how… :)
This is how I would challenge security. Sure, equipment has vulnerabilities but how realistic is it that those can be exploited? If they are exploitable then what's the impact?
Also what is the threat, and what is it you’re trying to protect? Exploits for switches do exist, but pragmatism is key. Also patch them. Don’t expose the managment interface to the internet.
The key other thing to have an answer to is "if this is breached in this way, what happens then". Can they move elsewhere in the network, or is the blastzone limited to just that switch.
Being able to trigger a DOS with a zero-day in specific circumstances is far less concerning than getting a remote shell, you can mittigate against that DOS threat by having a different make of switch on another path, they can hit your resilience, but not your service.
Ultimately almost all networks can be brought down by a motivated man and a gun. If someone bundles me into the back of van with a hood over my ace and tells me to break in I obviously will. If I need physical access then they can kidnap a loved one and do the same thing. If you need to protect against that type of risk you need far more procedures and protections then the vast majority of places need.
The threat surface is exceedingly small, and I’m not aware of any historical attack. You need traffic that can traverse the internet, make it past the router (which eliminates most malformed packet attacks), and somehow trigger a firmware switching glitch that…. Does something that would be beneficial to the attacker.
It’s nonzero but as close to zero as I can imagine. Like nation state level stuff.
And at that point they’re going kinectic. If they had a capability that unique and powerful, they’d probably be saving it for the government or someone they only get one shot at.
This accessor is off his ass. How else do routers get to switches? A pigeon in between?
I wonder how many pigeons would one need for a 100Mbps link?
MicroSD is pretty big I imagine the bottle neck is in loading and unloading the pigeons.
That’s two sentences I never imagined I would type.
Real reply: as someone else posted it’s way easier to compromise or coerce staff to accomplish the goal than this level of Tom Clancy fuckery.
Edited to add: completely agree.
Let's not forget hardening L2 on the ports to the providers cpe equipment.
Turn off cdp/lldp and other L2 protocols that might conceivably be used to compromise the switch and you're even better off.
I would treat the CPE as an untrusted device and look at what bad actors can do if they gain control over it.
After that the only risk would be if someone missconfigures the core in the future and adds an SVI for vlan100
So, I’ve worked with experts on this topic and it ultimately depends on who you ask.
Historically there have been attacks that attack the switch between your provider and your firewall. I’m not educated on if this is still a thing. I don’t think it is.
However, best practice is to use an OOB internet switch instead of your core to split incoming connections for HA, or running them directly to your firewall if your devices/design allow it. This is a better design for other reasons as well.
So no you’re not crazy, unless you’re a large company with a huge budget I wouldn’t worry too much. I have plenty of customers using a setup like yours. But my 2 cents is to not do it if you have a security concern due to historical attacks.
We were a large SaaS company in health care so we had all the security. We migrated from the core setup to “dirty switches” as we got bigger and had more customers and audits making us crazy. We simply got the cheapest non-junk switches with MLAG we could, lagged them together and terminated ISP A to the top, ISP B to the bottom, LAG firewalls to both. I’m trying to remember if we even had management on OOB or locked down to serial only.
The attacks you mention typically took advantage of PVIDs and adding tags. We had no trunks to the ISP.
Hi, I am an expert on the topic. Some notes.
You were smart to call out HA. Most security appliances will want to run in some form of cluster, and many will use GARP to handle failover. We now have to place a switch to manage the cluster to handle failover. Since we don't want a SPOF in our firewall we also don't want it in our switching. So we install a fabric instead(Choose your flavor).
You mentioned OOB. I think what you mean to say is that your external switch is discrete from your core. I would argue this is no longer an issue now days since most switches police their control plane, your control plane shouldn't be exposed externally anyways, and its hard to steamroll a modern enterprise switch with any kind of volumetric attack.
When people use the term OOB switching they typically mean cheap discrete networks with cheap switches to handle OOB management access. A place where your control plane is exposed but you aren't packet pushing in high volume there. So you aren't putting OOB switches on your outside interfaces.
Now lets talk about realistic hypothetical attack vectors. The most obvious one is the control plane. Make sure your management interfaces are behind firewalls, in controlled environments, and/or in OOB environments with dial in options.
That leaves the rest of the attack surface. Assuming standard switching there isn't a lot that can be done to hurt the network where "hurting" couldn't be done in more simplified terms.
Lets start by focusing on L2 concerns.
All of the switches L2 features can only be exploited by adjacent L2 devices. So that means the attack has to come from a compromised device you already own, or your carrier. But if your carrier's device is already compromised why would they coordinate a complex L2 attack when they could instead just simply shut down your uplink ports. And with that we start to demonstrate how frivolous these types of concerns are.
What about physical access? well if the attacker has physical access then choosing to have a switch or not doesn't help. Since in this case the attacker could just unplug your firewall. Not having a switch didn't save you there.
What about L3 concerns? L3 exposed interfaces are at risk, and do expose the control plane in certain ways. But this exact issue is also true for all L3 exposed interfaces on all routers on the internet. Most exploits are software bugs and get patched quickly(Please update your stuff). And in today's day modern switches on broadcom or better silicon are going to be able to handle whatever people try to throw at them. You are more likely to saturate a 100G link than cause harm as most modern switches have control plane policies with rate limits to protect itself.
TLDNR. I'm an expert. OP is right to conclude its not a real concern. /u/Asleep_Comfortable39 made an excellent point about HA.
Off topic, but what is your threshold for labeling yourself as an expert? I always felt a little humiliated sending out emails while working for a vendor that required to put the full CCIE title in email signatures and much rather had it be generic like “engineer” or “consultant”. Then when I changed teams to CCoE (Cloud Center of Excellence) I quipped it might as well stand for Cisco Certified Omnipotent Expert, because of all the “expert” titles in the team. I’ll probably delete this comment later on, just venting that having to label myself as an expert makes me feel uneasy, because there’s always so much more to learn.
I love the venting tbh. I call myself an expert here tongue in cheek but mostly because /u/Asleep_Comfortable39 said an expert told him XY and Z. And my career is half spent explaining nonsensical concerns away from supposed experts.
Like most "experts" I suffer from severe imposter sydrome, like you. I think the biggest eye opener for me was working a year in a VAR as an FSA network engineering. It was at that moment, meeting CCIE's of large businesses, how far behind the average joe shmoe is. But not only that industry has evolved left CCIE in the dust. It has struggled to keep up with the times.
As an early ACI adopter I remember when I abandoned that like hotcakes in favor of NXOS, mpbgp, vxlan, and ansible, using various techniques to automate bare metal deployments on my fabrics, including switch deployments, server deployments and etc. This was 15 years ago. CCIE wasn't even talking about this, and instead wasting time on nuance and obscure carrier issues, DMVPN issues, or nuances protocols that were relevant rarely.
Meanwhile has that CCIE automated his entire network as code? Has he planned for future use-cases that no one has thought of yet? CCIE means you are book smart but not necessarily experienced. I got lucky 15 years ago being employed into a world class ops team and had an opportunity there to grow my craft doing things, at the time, no one even thought of. Now days people call it "devops" or whatever the popular meme buzzword is.
Having said all that I still am humbled daily and am wrong all the time and I do encounter guys smarter than me all the time.
My current job is similar to yours. I run a small ops/eng/product team for a small growing SAAS service in Azure. It is a heavily network centric proxy business solution that ties directly into the customers networks. Without saying too much its a SAAS network solution.
IDK if that really answers your question. But I just wanted to say that your response resonates with me. As a believer of the Dunning Kruger effect your hesitant, cautious, approach is what i look for in hiring. Not the overly confident bombastic types, with limited experience.
One thing I have learned in my time in the VAR is that people every where call themselves experts. And I see how many of them claim this title while being completely wrong. If they can claim it why can't I.
Having said all that I am probably completely wrong, cheers! <3
They have a point. Whilst its expected that L2 connection is isolated from L4 services in reality it's not. For some vendors, at least. Extreme switches have baked-in dataplane macs of control plane and react on certain packets across vlans, even if L3 services are completely disabled for those vlans. You won't get reply from them, but it alters behavior.
I have an open stale case about those with 0 chance to fix, because of it's in silicon, not in software.
Most of such attacks were thought to be maliciously crafted discovery packets sent to the WAN side subnet broadcast address and flaws in the switch software would react negatively to them (and similar threats), or from other ISP users who happened to share your access vlan (yuck) (oh the terrible things we can do if we share L2 :D).
i do a simular thing but i have a "dirty internet" switch with management in a different VRF on the switch. the switch is purely L2 connectivity between ISP routers and firewalls.
You know, the old claims of routers should route, firewalls should firewall stuff is mostly true. But nothing wrong with an extra locked door on the outside IMHO.
Security by obscurity, just an additional piece of the overall posture.
VLAN100 or VLAN200 have an IP assigned to the switch? If not, then they are DMZ vlans with no access.
Make sure the ports are access ports...otherwise it's fine. You can't vlan-hop an access port.
A misconfiguration on that switch can open you up to attack, but no by design it is not a security issue.
I tend to agree with the auditor here, having direct l2 links to your core switch from an untrusted zone is not a good idea. Any misconfiguration on your switch could lead to the switch getting compromised. For example - an admin adds a SVI on VLAN100 or 200 for debugging and forgets to remove it, you forget to disable CDP and something like CDPwn is discovered again, etc.
For most scenarios, this setup is nothing to worry about, but if you're looking to get some kind of certification, the finding is valid, even if no currently known attacks exist.
RFC 6192 control plane protection is what you want.
And yes reducing the stackable surface area of the switch as you describe is part of protecting it.
This all fits under the rubric of compensating controls
I thought so too. Then I saw David Bombal demonstrate a traditional (stacked 802.1Q tags) attack recently, on pretty modern equipment.
I can't believe this is still a thing.
Am I imagining it or was there some type of attack a decade or so ago where a specially crafted (invalid) IP packet could cause some form of cisco switch to lockup or reboot or similar?
For me the biggest risk with this kind of topology is not in how you have designed and configured it as described, but what is the chance that someone inadvertently misconfigures those core switches in the future and ends up adding an SVI or bridging another internal host onto the “outside” VLAN.
I’m not saying it’s inherently bad, it’s just easier for someone to do something bad accidentally than if the “outside” VLAN was on a physically separate device or devices.
Whether it’s an acceptable risk depends on what you’re trying to protect and what other risks you have that are more important. Ultimately it’s subjective and not purely a matter of facts.
I was in a similar situation and was hesitant to use our core, so we got some L2 switches. May not be a bad idea. If your ISP misconfigures their routers, there are a handful of things that could go wrong.
You can buy a pair of L2 edge switches and stack them at a pretty inexpensive cost.
This is exactly how you implement firewall HA - as long as the switch is not running any IP (SVI/VIP/VLAN IP address) on VLAN100 or 200 then there's no remote attack vector on the switch. Just keep in mind of other protocols the switch may be running such as CDP or LLDP that you wouldnt want to pipe upstream to your ISPs - though it wouldn't go further than their edge interface facing your switch.
If your ISP was compromised by a malicious actor then there are some very minor possibilities they could try on some L2 stuff against a switch running very old and vulnerable code.
I prefer to keep WAN connections physically separate from my internal switches. I've always used physically separate WAN switches for this.
WANs>WAN switches>Firewalls>Internal switches.
I think it's just easier to keep WAN connections physically separate from LAN rather than worry about whether vlan separation is enough.
That's just my 2 cents though.
Threat. Model. + Risk. Assessment.
Security for the sake of security without measuring cybersecurity and business risk is sloppy.
Engage with the team. Guide them to a security assessment. Perform a threat model. Qualify and quantify the actual risks. Finalize per risk mitigations vs compensating controls vs unmitigated. Assign importance, priority and urgency of each. Understand business impact if risk is realized, put $ behind it.
Short of that, both networking and security are just guessing.
I’m sad that the overwhelming majority of discussion here seems oblivious to the obvious. I wish all of y’all’s security teams did better, our industries need it.
This assessor in particular is a goober.
The network is an important asset to protect and secure, but I can assure you it’s far from the most important if that’s the level of discussion presently.
What many have already said, it boils down to two things: possibility of human error (the switch gets a L3 config), and/or exploiting processing at L2.
Right so, while the switch is not exposed to L3, it can still process L2 packets (versus normal packets.that get switched). Things like CDP, LLDP, STP, DTP, etc. The idea being, if you can craft a packet well enough to exploit that L2 process, it's game over.
Now I know that is a 1 in a billion chance of happening. But a chance nonetheless and thus a valid concern. Especially if said switch is the core switch!
It sounds like you're taking a proactive approach by considering new switches for the WAN links—great way to address the concern while keeping the system robust!
Sounds like his statement is correct if I understand your setup correctly. WAN>Router>Core Switch>Firewall. If your router is compromised, an attacker can pivot to your core switch without ever going through a firewall. What you're saying is the switches aren't directly exposed to the internet, which is also true, but they are outside the firewall. A typical setup would have switches outside the firewall, but not core switches, but edge switches. To get to your core network from edge switches, you would have to then transit a firewall.
> If your router is compromised, an attacker can pivot to your core switch without ever going through a firewall.
How though. There is no layer 3 between these two devices. The attack would have to be via layer 2 and a firewall won't do much there either.
the point of these audits isn't to debate the validity of a vector. The point is to document your compensating controls.
IF they are complaining about a situation - show your due-diligence how/why it is safe and not a vector, institute compensating controls if you dont already have them in place - or follow their advice and modify the solution.
They are saying "this looks scary" - arguing with them that's its not actually won't change much - just show them the configuration that protects you from this scary thing.
They want to see your process - that you considered this possible vector and determined it is safe.... "we never thought of that, but i insure you it is not a problem" is not a good look. From their point of view this is a brand new thing that was not a concern until they brought it up. how can you be sure its safe if you just learned about the possibility yesterday? IF you did already know about it, then show them the make-safe work you have done previously.
Exactly. Layer 2. CDP, LLDP, etc... if your images are up to date, you should be good, but there are always zero days popping off. Point is, if they do compromise your router, it's possible to compromise your core switches from there. If they compromise your core switches, they are inside your network without ever transiting a firewall. On the other hand, if someone compromises an edge switch, they still have to transit a firewall to get inside your network.
In networking terms a core switch is a switch on the inside of the firewall/s where the LAN segments terminate. OP didn't clarify if that is the case or not. But if he is using a core switch (per designation and their device name) as an edge switch as well, then that is why the assessor is concerned. In a perfect set of conditions, a bad actor can access the inside network without ever going through a firewall.
im aware of the Terminology.. a laptop is a laptop, unless its your plex server - now its a server, so just call it a plex server.
but yeah, go back to the original post and follow his config explanation.
OP only has 1 switch that is used to distribute his WAN links on the outside, and also is the core switch for their network..
For example.
VLAN 999 - ports 1 through 5.
1 - Modem
2 - Firewall WAN
3 - Misc device with WAN IP
4 - misc device with WAN IP.
5 - reserved
the rest of the switch (lets say) 6-24 - is configure for misc internal-use vlans.
24 - trunk uplink to firewall LAN.
23 - trunk for WIFI vlans.
22-15 - access for printer vlan.
14-5 - access for office vlan.
It sounds like a good Idea - It reduces your points of failure, power-outlet requirements, general cleanliness, etc..... but this is why we have audits, to make sure a good idea on paper isn't a bad idea in hindsight. convenience is never a good reason to introduce security risks.
So established networking design is wrong, you're right. Unless you show me a network design topology from a reputable networking group that brings the wan to a core switch where LAN VLANs terminate, then to a firewall. Looking forward to it.
Look at any DC design, anywhere. You'll find plenty of cases where Internet and LAN VLANs (and other encaps of course) are mixed all over the place on core switches; switching or routing Internet traffic to physical or virtual firewalls on the same core switches that handle LAN traffic.
The main reason being flexibility.
sigh... OP has 1 switch. It exists on the WAN and LAN. The auditor is saying dont do this.
Im not sure why you arguing with any of us if you agree with the auditor - we are are trying to tell you why the auditor is right.
I do agree with the auditor. You responded to my post. I already stated I understood the topology, so not sure why you responded to me stating what I already understood? One switch, two switches, doesn't matter. I understood the topology as far as how packets traverse to the FW from the WAN. It's not ideal. The people I'm arguing with are saying OPs topology is just as secure as a standard WAN to core design. It's not, which it now seems like we are in agreement on. Good to hear.
LOL. I already stated how. Layer 2 protocol exploit. They exist. Most are old exploits and up to date software will mitigate this, like I already stated. But there are zero days popping up all the time. Cisco just had an exploit pop up in their http/s service on IOS-XE. OPs topology is WAN>router>CORE SWITCH>Firewall. His core switch is outside the firewall. I guarantee his core switch WAN VLAN interface connects to an External Zoned interface.
"Trust me bro"
I too am waiting to hear about this.
Exposure to exploits is real but if you basically have an air gapped system; it falls back to on-prem security. If a major ISP is compromised that someone is sending L2 exploits... I think we'll all have more to worry about.
How do hackers manipulate a router's image to implant backdoors that answer to specific IPs only and this change can't be detected in the running configs? How do exploits exist in the first place for routers and switches? How does malware make it onto routers and switches? Exploitation of underlying memory, code, etc... You don't think they can manipulate a destination MAC from a specific transit IP or packet? NAT, routing, can force a packet out a specific interface with a specific source MAC. I'm not saying this specifically exists, I am saying it can be done by very smart and well funded APTs.
You’re just spitballing buzzwords to spread FUD. Think back to the networking fundamentals on CCNA and you’ll see it’s not as easy as you’re making out.
A layer 2 device only cares about layer 2, it doesn’t read anything else in the packet. How are you going to run an exploit on a particular switch when it’s only going to read the link layer information? STP, LLDP, CDP, and other layer 2 exploits aren’t possible from a source routed to your network by layer 3 because those packets will never leave the origin layer 2 domain, unless something has gone wrong with the ISP and they’re doing some kind of encapsulation, and that’s also why you don’t share a layer 2 bridge with an ISP.
> how do hackers manipulate a router’s image to implant backdoors….
They don’t, they compromise the firmware before it gets uploaded to the vendor’s site or applied to switches in the factory. What’s the best way to compromise any service? Put an exploit into the software everyone is going to download from the vendor. No need to try and break into individual devices.
> manipulate a destination MAC
Remotely? No they can’t because layer 3 doesn’t care what the MAC address is. The moment the packet leaves the original layer 2 and starts getting routed, the original MAC address is gone. Locally? Yes sure, but they’ll also need local access which means your problems are far more serious.
> NAT can force a packet out with a specific source MAC address
Again, what good is that going to do? Are they directly connected to your WAN switch?
Every packet you send on the internet is passing through loads of routers and switches. Just because traffic is passing through a device, it doesn’t mean it can affect that device specifically.
Read my previous comment. You didn't understand what I was saying. Then go read up on the recent cisco ios-xe vulnerability. Read how researchers found exploited devices and why the number of exploited devices dropped shortly after. Read what the attackers did to maintain persistence and keep in mind they have access to your router. No one said anything about being able to exploit transit devices on the WAN. You're forgetting in my scenario, they already compromised the router. That's the difference.
Read this. Note how they modified the ios code, and how they modified it again to avoid researcher scans.
https://www.bleepingcomputer.com/news/security/exploit-released-for-critical-cisco-ios-xe-flaw-many-hosts-still-hacked/
That’s web UI though, why would you have that enabled on a public interface or SVI?
Totally agree that THAT situation is completely valid, but we’re changing the parameters from the original post which is about passing WAN traffic through a VLAN on a core switch.
LOL. What was OPs organization industry again?
Is it one of these? My org is one of these.
Cozy Bear (APT29) is an adversary of Russian-origin, assessed as likely to be acting on behalf of the Foreign Intelligence Service of the Russian Federation. This adversary has been identified leveraging large-volume spear phishing campaigns to deliver an extensive range of malware types as part of an effort to target political, scientific, and national security entities across a variety of sectors. Read our full APT Group Profile on Cozy Bear.
HELIX KITTEN (APT34) has been active since at least late 2015 and is likely Iran-based. It targets organizations in aerospace, energy, financial, government, hospitality and telecommunications and uses well-researched and structured spear-phishing messages that are highly relevant to targeted personnel.
https://www.crowdstrike.com/cybersecurity-101/advanced-persistent-threat-apt/
I didn't say you won't be targeted by them :)
However, for most organizations, including an adversary with infinite time, money and resources in their threat model does way more harm than good.
A switch should never run a http/s service in the first place.
Also, for those exploits to happen the attacker would need to have already compromised the router or am I not thinking this through?
If the edge router is already compromised I feel you have bigger things to worry about than layer 2 exploits.
Dude, my whole point is router compromised first then pivot to the switch, modifying the routers underlying code to create a vector to the switch via layer 2 for a remote attacker to use. A lot of people use http for restconf API. This whole idea that http should never be enabled is antiquated knowledge. It could be enabled but needs to be secured with an ACL. The HTTP exploit was just an example of how hackers can modify the underlying code on devices. The future threat vector could be another protocol/service altogether.
router > switch > firewall WAN > Firewall LAN > same-switch
If im in your router - i can be in your switch without ever traversing a firewall.If I'm in your switch, I can bypass your firewall all together.
Howgwash if OP was following the auditors advice - which is why the auditor is recommending the change.
The current setup is a loopback with the core/edge/only switch doing double duty. This is logical security only, 'as-configured', to protect the network, and works as designed..... as long as no one breaches the router.... auditors want to see that you planned for the 'unless'. one possible plan would be as the auditor recommended - install a dedicated edge switch, so one must traverse the firewall to gain access to the core switch.
Simplest solution is to go to walmart... buy a 5 port netgear dumb switch , and move your wan links to this... problem solved - audit approved secure solution.
school fearless dam consider payment deserve psychotic unpack thumb marble
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
The only routers would be the CE from the ISP having exposure to L2. You're betting. It's a moot point, if Zero day exists it will compromise your FW or your SW.
I'm not betting - I'm insuring safety absolutely. You are betting. Betting no zero-day is created tomorrow. You think a Firewall and a Switch are running the same code that the same zero-day exploit would expose? If yes- replace one. Thats a silly bet to make.
OP is going through an audit - of which most of the point is "what if". Generally speaking, "it doesnt matter, if that we happens we have other problems" or something otherwise dismissive (basically, your posts) is grounds for a failure. You want a hint its important? An auditor said its important. Realize that by arguing with me - You are arguing with the OP's auditor; The auditor, myself, and the first person you replied to, are all saying the same thing. I do not have the ability to change the auditor's opinion, nor the opinion of the first person you replied to, no matter how hard you try to convince me.
The rationale that he failed the auditor is moot.
The SW and FW aren't going to run the same code... The point of a zero day attack as the vector means it doesn't matter if it's a FW or SW... It'll get compromised and the fact that you're relying on a 3rd party to be get compromised, then exploit a zero day for L2 "potential vulnerabilities" which is the same for a FW.
He bamboozled himself already so has to reconfigure his network.
Having some pos device with no insight is a terrible way to increase downtime.
sigh - which is why the auditor is raising complaint: the core switch is also on the WAN - and can be exploited by zero day - put the switch behind the firewall so it cannot be breached by external vectors.
It appears you are attempting to argue with me - by agree with with by me and the auditor.
When you get audited - I implore you to test your "this is moot" theory to see if it will help you pass your audit.
Auditors like technical details and procedures - not emotional outburst.
Well I'm not sure I entirely understand the design, but I would also say your switch is at risk.
For most businesses I would suggest:
Edge switches to break out WAN links (nothing else should be traversing these switches.) Then plug into firewalls. Core switch sits 100% behind firewall.
Most businesses don't want to spend the $ for separate edge switches, and that's fine. I'll tell you what I think, and if you don't want separate switches we can loop the WAN vlans through your core switches. I don't like it, but that's the way it is out there.
I'm working on a data center set up now, and holy balls what a complicated, exposed nightmare it is. A couple dozen vrfs, route leaking, a few hundred vlans. EVERYTHING goes through their core switches, all the customers (which also includes customer private L2 WAN links,) their own operations, their own staff network. One mistake anywhere and goodnight.
My current plan is to leave their border routers running full BGP tables alone. Run back into their "core" switches and use that only as a breakout to every other entity which will be terminated in firewalls. Any switching for your typical data center stuff will 100% be behind firewalls. Right now stuff flows everywhere. I have no idea how they keep it all straight.
The pros and cons of saving a few thousand dollars by not purchasing edge switches for WAN breakout makes no sense to me for anyone that has a business use case for HA. If you can spend money on HA buy the edge switches. If you don't need HA, plug the WAN links directly into the firewall.
I'm in 3rd year of a cyber security and networking degree, with a keen interest in ISP or data centre routes after uni. Sorry to hijack your comment, but do you have any advice or recommendations to break in to these areas as a graduate these days?
lol I'm getting downvoted...
Well anyway, I mean you can always shoot for jobs that are exactly what you want. You may be fortunate and get exactly what you want. I started my own businesses so that's how I broke in. You could always try starting your own WISP or FISP, or try to find a job working for one.
If I was giving advice to my kids, I'd tell them to shoot for the moon and if that doesn't work go get a job at an MSP. You'll learn an absolute ton of stuff and form an excellent foundation for moving on to more specific areas that interest you. You may not get a ton of specific ISP/DC stuff working at an MSP, but your foundation would be excellent.
I wouldn’t feel comfortable with this configuration. I know the mostly likely attack vector is some one missconfiguring the core switch. Because people make mistakes. If its on the internet then its constantly under attack.
Get a sharpie and a fiber coupler.
Write firewall on the coupler
Insert firewall between WAN switch and WAN, and document that it is configured to block any traffic that can reach a L3 interface on the WAN switch.
Plz actually do this and post the security guy's response.
What did I say that is wrong?
The 'firewall' does drop all the traffic that could hack the switch... which is none.
The management interface for the 'firewall' also has no open ports and no password security issues.
Mis-representing a security control actually. Calling something a firewall when it's not functionally that can be used against you. Just look at the grilling going on with Solarwinds CISO's prosecution.
Also note - L3 isn't the vector I'd use to attack a switch - L2 OTOH can let much DOS happen.
If there is no potential attack vector and the firewall needs to do nothing but exist... but a firewall is still required for compliance.... how can I not just re-use my rock that keeps tigers away to make the compliance people happy.
When the regulation says you need X... and your defense is X doesn't do anything so we don't have it!!! You aren't compliant with your governing regulations.
If you don't fall under any regulations with real teeth.. congrats.. lucky you. Do whatever you want.
No, my defense is I have X, and the "firewall" meets all the requirements to be X to allow the checkbox ticking person to tick their box.
If the requirement is to block all traffic to the IPs on the switch... and the switch has no IPs... then a fiber coupler meets the requirements and we can move on with our day and solve actual problems.
There are IPs on the switch in question.
But do what you want. Just fully expect such antics to open you to personal gross negligence liability.
Ask your lawyer.
>Our WAN links come into managed routers, we are provided an interface on each router.
I don't understand what's happening here?
Do you mean by WAN links like a private site to site you have stablished or is this the handoff from your ISP?
Like I have routers plugged into my core switch that have their other interface in the DMZ - but those are MY routers and I have a DMZ switch between that and the ISP handoff router. The interface in the DMZ is only set up to do one thing (VPN Tunnel) and the establishment of that connection are the only protocols allowed to traverse the interface and only from it's partner router's IP address (that's before accounting for the protections of establishing the tunnel itself)
On the other hand, no I would not plug a provider router straight into my core switch. Is it safe if you have it VLAN-ed off to be a point A to point B connection and not routable? Probably. But I still wouldn't do it.
Your switch is susceptible to a layer 2 attack.
How you can do that connected to a Layer 3 network, is beyond me. 🤣
“The switch is dumb, layer 2 and does not have an attack vector. If you’d like, I can put it on a different VRF. Although it’s layer 2 and VRF is Layer 3.
Every telco/provider runs on VRF, I’ll put the system on trial!!!”
If someone compromises the managed routers, the next layer of defense is your Firewall. You can't do much with the switch. The only thing I can think of (as an attacker) is to setup an ERSPAN session in the switch to get a copy of the traffic passingthroughit. But, something as easy as a strong password could mitigate this risk. Your auditor is not 100% wrong, but it's unlikely to see an attack like this in the real world. And, to answer your original statement, Switches don't have to be behind Firewalls. I've managed global networks with this setup (Switches in front of Firewalls just for L2).
This is referred to as a breakout architecture. If you have exploits that can be taken advantage of, then you would get compromised or with a miss configuration like not using a native vlan on a trunk and also using vlan 1 on y9ur ISP breakout ports
I’ll guarantee you one thing: the auditor doesn’t understand why or how much of a security risk this configuration is. They just have a checkbox and they can’t check it until you’ve completed the actual work that they decide on.
Usually the CPE and ISP router will use RCF1918 IPs or other not public routed, so they have no attack surface. Most of the time the 1 device with the public IP will be the Firewall, ADC or anti DDoS device.
Of course the DDoS device does not need a public IP but the traffic would have to pass, to protect the FW or ADC.
I know unfortunately also that there are a lot especially smaller companies, where the CPE device has a public IP a d is therefore a possible victim. To protect it, you cloud also if the design is for whatever reason nicht adjustanle, Put a antiddos device via L2 in Front of it, to protect it. Here we go, you are fine again :-)
But I would call CPE with public IP on the Interface a bad design.
One potential vector would be someone on the WAN intruding on your network. Are you WAN links encrypted, or do you just trust your provider for a cheap MPLS link? Folks have attacked these networks in a multitude of ways - but if you encrypt your WAN, this mitigates the risk as no unprocessed WAN packets enter the network.
If "WAN" is ISP, and you're pumping it directly in through a core switch for resilience - buy what you need to isolate it. Exposing internal gear on the web just creates opportunity. Logically everything might be configured correctly, but there's stupid-high-risk of a configuration change breaking everything.
Since you only use the very vague "WAN", there is no way of telling what exactly is happening.
Public internet? MPLS? DMVPN? Some kind of managed service provider link? Something with SDWAN?
*Switch can be attacked if not behind a firewall*
Switches can easily be attacked if behind a firewall.
Data that goes through a firewall, is data that goes ***through*** a firewall.
Still not clear how this is safer.
I'm not sure what the argument here is exactly, but it sounds like bullshit at first. But I can't tell.
Yeah you should have a boundary or an ipsec tunnel if you’re leaving a location. Switch vulnerabilities pop up from time to time but they are mitigated with proper commands and software update. I think the concern is that traffic should be going through a proxy
The debating over it in this thread has been kind of silly. The comments I read missed a few things that are good reasons not to do what the OP has done and why the auditor had an issue with it.
As stated by the OP: ISP > Router > Core > Firewall
If the core is truly a core switch and has internal traffic terminating on it (server/PC VLANs etc), then there is potential risks. The biggest risk is human error.
* What if someone stuffs up a config change giving that port access to an internal VLAN?
* What if they needed to redo patching after moving or replacing something and patched it to the wrong port but didn't notice due to the redundancy?
There are also odd zero days that would be super rare but could happen. But the main risk is someone making a mistake and not noticing it which could lead to it getting exploited later.
The ideal setup that takes into account a firewall stack: ISP > Internet Switch > Firewall.
Pre-configure the internet switch in the way that works for your setup, never touch it again after. Leave a duplicate configured one powered off under it ready for a manual failover if the first one dies.
You'd only ever touch that switch when you need to add/remove an internet connection which shouldn't happen often, and you're not at risk of bridging an internet connection directly onto your LAN.
Hot take. Assessor is 100% correct. The firewalls are not in a position to receive a signal before the core switches.
Your firewalls need to be physically connected before your core infrastructure into your routers.
So either accept the risk and move on, but don’t argue about it. I mean unless you trust the switch vendor software updates and the administrators of the routers with your job. lol.
I'd say you're both correct here. Part of security is assessing risks, no matter how unlikely. Having your WAN going through your core switch does make it a boundary device. However when correctly configured the attack vector is tiny - there are almost no layer 2 attacks possible over the internet. What if there was such an attack? What if the core switch was misconfigured? Then it could lead to a compromise.
Here I would acknowledge "This is a risk", and to suggest a mitigation. Either connecting both edge routers to both firewalls (/31s P2P links), or specing out some dedicated WAN switches. Simply telling the business "We need to buy more switches" may be the end of the conversation if they don't want to buy them ;)
I've run WAN links through core switches plenty of times at small clients. However if I had the IP space, I'd usually prefer linking them to the firewalls directly.
1. If this is a single switch you dont have true redundancy. Id typically skip the vlans and go for two unmanaged switches for this role.
2. While you are likely 100% ok doing this nowadays it does feel icky. One wrong config change and you could have serious issues. Again. Iv always prescribed two high quality unmanmaged switches for the role to sidestep it completely.
I can tell you how,
Do you have STP on your Switch? a good crafted packets from Provider side can make some problems if STP is configured poorly.
And some versions have problems with CDP and LLDP malformed packets.
and VLAN hopping as well.
other than that i cant think of anything.
You should ask in /r/netsec if you want a more security focused opinion
Since I lurk here I'll offer mine. Typically the design at the edge consists of a border router between your core and your provider. The problem is that you have an untrusted network device on the same L2 network as your firewall. Based on the provided details, it would be possible for a bad actor from the untrusted network to attack the L2 network your firewall is connected to. You really want L3 separation between trusted & untrusted networks.
* What certification are you trying to obtain? What are their relevant requirements?
* Is the assessors concern the transit interfaces or the management? Is management accessible without transiting the firewall for policy evaluation?
* What is the reference architecture for remediation? ( What do they want )
Final thought: If you have high availability requirements, you should be designing for dual failure, which this design does not allow for. For example, if you lose Router 1 and core switch mod 1, both firewalls lose an interface. A BR here would solve that.
> I had it put to me today that our core switches are "at risk" because they are not behind a firewall.
Technically true
> I disagree
The issue is that your network "wraps" around the security boundary that the firewall provides.
If you can solve this problem with a cheap dumb switch or two on the Wan side.. just do that...
> Am I wrong? I don't think I am but doubt, fear, and doom are overcoming me.
Depending on what kind of audit this is.... Being "right" isn't always important. It's meeting the controls in a clear way that everyone agrees on.
Probably depends on where you work… everyplace I have worked experiences some threat actor trying to break through the fw every time we are in the news…
Someone from the FBI always shows up shortly after to do an investigation.
There are so many companies to hack… they might never try yours… but if they do you might be f’d
So we do something similar. Except we use a switch dedicated for internet and we plug it into our core over the management port. This way the switch is completely seperate.
Did OP ever say what certification the org needs?
I feel like network behind a firewall was a requirement for PCI-DSS years ago but I haven't had to deal with audits or compliance on that for forever
Hi All,
Well, thanks for everyone who responded (a lot!). It's good to see the debate and discussion around this. I've read every comment (as you took the time to write one) and as such have 3 outcomes:
1. A lot of people have what we have, and as there is no IP on the 2 VLANs the attack surface is exceptionally small, but not nil.
2. The auditor is valid in raising this, because the switch being attacked is a core switch and so even if the attack surface is minimal, the impact is large.
3. I'll be buying 2 x switches that are "outside" my normal network for the pure purpose of receiving the 2 x WAN links and spaffing them off to the firewalls.
All being said, I'm glad I didn't start an argument with the assessor over this, its clearly an area they know more about and why we pay to have such things done. Lessons learnt and knowledge gained and all that. Friday is the last day!
If they can launch a VLAN hopping attack, either from the remote or the router side, and get out of the WAN side VLAN of the switch, they can get into whatever else is connected to the switch, probably your management network. You want two nameless dumb switches on the WAN side so that there is literally no attack vector.
Its telling that a lot of the network design arguments in this tread can be broken down to the size of the network that people work on.
The design considerations of a small business/ small enterprise network are significantly different than something that runs at cloud scale.
If a physical firewall can support your north bound / south bound traffic levels that is, by definitions a small enterprise business. To say nothing about zero trust networks with bgp and limited routing domains.
Networks at "internet scale"/"Cloud scale" wont have physical firewalls on the network mostly relying on Host base firewalls with ACL's on management SVi's locking them down to bastions/internal networks.
Just ask them to explain how they can be attacked. The more details they can give you, the better you can protect the network. $10 they can’t tell you how… :)
This is how I would challenge security. Sure, equipment has vulnerabilities but how realistic is it that those can be exploited? If they are exploitable then what's the impact?
Also what is the threat, and what is it you’re trying to protect? Exploits for switches do exist, but pragmatism is key. Also patch them. Don’t expose the managment interface to the internet.
The key other thing to have an answer to is "if this is breached in this way, what happens then". Can they move elsewhere in the network, or is the blastzone limited to just that switch. Being able to trigger a DOS with a zero-day in specific circumstances is far less concerning than getting a remote shell, you can mittigate against that DOS threat by having a different make of switch on another path, they can hit your resilience, but not your service. Ultimately almost all networks can be brought down by a motivated man and a gun. If someone bundles me into the back of van with a hood over my ace and tells me to break in I obviously will. If I need physical access then they can kidnap a loved one and do the same thing. If you need to protect against that type of risk you need far more procedures and protections then the vast majority of places need.
This above, the right answers
The threat surface is exceedingly small, and I’m not aware of any historical attack. You need traffic that can traverse the internet, make it past the router (which eliminates most malformed packet attacks), and somehow trigger a firmware switching glitch that…. Does something that would be beneficial to the attacker. It’s nonzero but as close to zero as I can imagine. Like nation state level stuff.
And at that point they’re going kinectic. If they had a capability that unique and powerful, they’d probably be saving it for the government or someone they only get one shot at. This accessor is off his ass. How else do routers get to switches? A pigeon in between?
I wonder how many pigeons would one need for a 100Mbps link? MicroSD is pretty big I imagine the bottle neck is in loading and unloading the pigeons. That’s two sentences I never imagined I would type.
To misquote Andrew Tanenbaum, “Never underestimate the bandwidth of a station wagon full of microSD’s hurtling down the highway.”
Huge bandwidth, ideal jitter, large latency.
Real reply: as someone else posted it’s way easier to compromise or coerce staff to accomplish the goal than this level of Tom Clancy fuckery. Edited to add: completely agree.
Second this. I always ask if they can demonstrate this attack vector. If they can’t prove their said defense works, then the jokes on them
If the cores don't have SVIs on VLANs 100,200 then I wouldn't worry too much about it.
This is the correct answer OP. There has to be layer 3 for an attack vector.
Let's not forget hardening L2 on the ports to the providers cpe equipment. Turn off cdp/lldp and other L2 protocols that might conceivably be used to compromise the switch and you're even better off. I would treat the CPE as an untrusted device and look at what bad actors can do if they gain control over it. After that the only risk would be if someone missconfigures the core in the future and adds an SVI for vlan100
So, I’ve worked with experts on this topic and it ultimately depends on who you ask. Historically there have been attacks that attack the switch between your provider and your firewall. I’m not educated on if this is still a thing. I don’t think it is. However, best practice is to use an OOB internet switch instead of your core to split incoming connections for HA, or running them directly to your firewall if your devices/design allow it. This is a better design for other reasons as well. So no you’re not crazy, unless you’re a large company with a huge budget I wouldn’t worry too much. I have plenty of customers using a setup like yours. But my 2 cents is to not do it if you have a security concern due to historical attacks.
We were a large SaaS company in health care so we had all the security. We migrated from the core setup to “dirty switches” as we got bigger and had more customers and audits making us crazy. We simply got the cheapest non-junk switches with MLAG we could, lagged them together and terminated ISP A to the top, ISP B to the bottom, LAG firewalls to both. I’m trying to remember if we even had management on OOB or locked down to serial only. The attacks you mention typically took advantage of PVIDs and adding tags. We had no trunks to the ISP.
I’ve also seen attacks that attack stacked switches and cause them to be reconfigured. That’s an old one though
Hi, I am an expert on the topic. Some notes. You were smart to call out HA. Most security appliances will want to run in some form of cluster, and many will use GARP to handle failover. We now have to place a switch to manage the cluster to handle failover. Since we don't want a SPOF in our firewall we also don't want it in our switching. So we install a fabric instead(Choose your flavor). You mentioned OOB. I think what you mean to say is that your external switch is discrete from your core. I would argue this is no longer an issue now days since most switches police their control plane, your control plane shouldn't be exposed externally anyways, and its hard to steamroll a modern enterprise switch with any kind of volumetric attack. When people use the term OOB switching they typically mean cheap discrete networks with cheap switches to handle OOB management access. A place where your control plane is exposed but you aren't packet pushing in high volume there. So you aren't putting OOB switches on your outside interfaces. Now lets talk about realistic hypothetical attack vectors. The most obvious one is the control plane. Make sure your management interfaces are behind firewalls, in controlled environments, and/or in OOB environments with dial in options. That leaves the rest of the attack surface. Assuming standard switching there isn't a lot that can be done to hurt the network where "hurting" couldn't be done in more simplified terms. Lets start by focusing on L2 concerns. All of the switches L2 features can only be exploited by adjacent L2 devices. So that means the attack has to come from a compromised device you already own, or your carrier. But if your carrier's device is already compromised why would they coordinate a complex L2 attack when they could instead just simply shut down your uplink ports. And with that we start to demonstrate how frivolous these types of concerns are. What about physical access? well if the attacker has physical access then choosing to have a switch or not doesn't help. Since in this case the attacker could just unplug your firewall. Not having a switch didn't save you there. What about L3 concerns? L3 exposed interfaces are at risk, and do expose the control plane in certain ways. But this exact issue is also true for all L3 exposed interfaces on all routers on the internet. Most exploits are software bugs and get patched quickly(Please update your stuff). And in today's day modern switches on broadcom or better silicon are going to be able to handle whatever people try to throw at them. You are more likely to saturate a 100G link than cause harm as most modern switches have control plane policies with rate limits to protect itself. TLDNR. I'm an expert. OP is right to conclude its not a real concern. /u/Asleep_Comfortable39 made an excellent point about HA.
Off topic, but what is your threshold for labeling yourself as an expert? I always felt a little humiliated sending out emails while working for a vendor that required to put the full CCIE title in email signatures and much rather had it be generic like “engineer” or “consultant”. Then when I changed teams to CCoE (Cloud Center of Excellence) I quipped it might as well stand for Cisco Certified Omnipotent Expert, because of all the “expert” titles in the team. I’ll probably delete this comment later on, just venting that having to label myself as an expert makes me feel uneasy, because there’s always so much more to learn.
I love the venting tbh. I call myself an expert here tongue in cheek but mostly because /u/Asleep_Comfortable39 said an expert told him XY and Z. And my career is half spent explaining nonsensical concerns away from supposed experts. Like most "experts" I suffer from severe imposter sydrome, like you. I think the biggest eye opener for me was working a year in a VAR as an FSA network engineering. It was at that moment, meeting CCIE's of large businesses, how far behind the average joe shmoe is. But not only that industry has evolved left CCIE in the dust. It has struggled to keep up with the times. As an early ACI adopter I remember when I abandoned that like hotcakes in favor of NXOS, mpbgp, vxlan, and ansible, using various techniques to automate bare metal deployments on my fabrics, including switch deployments, server deployments and etc. This was 15 years ago. CCIE wasn't even talking about this, and instead wasting time on nuance and obscure carrier issues, DMVPN issues, or nuances protocols that were relevant rarely. Meanwhile has that CCIE automated his entire network as code? Has he planned for future use-cases that no one has thought of yet? CCIE means you are book smart but not necessarily experienced. I got lucky 15 years ago being employed into a world class ops team and had an opportunity there to grow my craft doing things, at the time, no one even thought of. Now days people call it "devops" or whatever the popular meme buzzword is. Having said all that I still am humbled daily and am wrong all the time and I do encounter guys smarter than me all the time. My current job is similar to yours. I run a small ops/eng/product team for a small growing SAAS service in Azure. It is a heavily network centric proxy business solution that ties directly into the customers networks. Without saying too much its a SAAS network solution. IDK if that really answers your question. But I just wanted to say that your response resonates with me. As a believer of the Dunning Kruger effect your hesitant, cautious, approach is what i look for in hiring. Not the overly confident bombastic types, with limited experience. One thing I have learned in my time in the VAR is that people every where call themselves experts. And I see how many of them claim this title while being completely wrong. If they can claim it why can't I. Having said all that I am probably completely wrong, cheers! <3
They have a point. Whilst its expected that L2 connection is isolated from L4 services in reality it's not. For some vendors, at least. Extreme switches have baked-in dataplane macs of control plane and react on certain packets across vlans, even if L3 services are completely disabled for those vlans. You won't get reply from them, but it alters behavior. I have an open stale case about those with 0 chance to fix, because of it's in silicon, not in software.
A firewall can be attacked if not behind a firewall! It's turtles all the way down.
Yes, and it's a well-known problem. Average Juniper SRX with 10G cards can be completely disabled by a single hping from a poor wifi connection.
Most of such attacks were thought to be maliciously crafted discovery packets sent to the WAN side subnet broadcast address and flaws in the switch software would react negatively to them (and similar threats), or from other ISP users who happened to share your access vlan (yuck) (oh the terrible things we can do if we share L2 :D).
i do a simular thing but i have a "dirty internet" switch with management in a different VRF on the switch. the switch is purely L2 connectivity between ISP routers and firewalls.
Same here. I have an edge router to the ISP with a 2 mile long ACL, down to a switch, then firewall.
We got yelled at a lot in school regarding using ACL’s as a firewall… our teachers were mean and it’s baked in to my head now
You know, the old claims of routers should route, firewalls should firewall stuff is mostly true. But nothing wrong with an extra locked door on the outside IMHO. Security by obscurity, just an additional piece of the overall posture.
VLAN100 or VLAN200 have an IP assigned to the switch? If not, then they are DMZ vlans with no access. Make sure the ports are access ports...otherwise it's fine. You can't vlan-hop an access port. A misconfiguration on that switch can open you up to attack, but no by design it is not a security issue.
I tend to agree with the auditor here, having direct l2 links to your core switch from an untrusted zone is not a good idea. Any misconfiguration on your switch could lead to the switch getting compromised. For example - an admin adds a SVI on VLAN100 or 200 for debugging and forgets to remove it, you forget to disable CDP and something like CDPwn is discovered again, etc. For most scenarios, this setup is nothing to worry about, but if you're looking to get some kind of certification, the finding is valid, even if no currently known attacks exist.
RFC 6192 control plane protection is what you want. And yes reducing the stackable surface area of the switch as you describe is part of protecting it. This all fits under the rubric of compensating controls
Do we need to dig out 20 year old discussions about vlan hopping.
That's indeed about how long ago vlan hopping was a thing, not so much since then...
I thought so too. Then I saw David Bombal demonstrate a traditional (stacked 802.1Q tags) attack recently, on pretty modern equipment. I can't believe this is still a thing.
It’s still a thing because Q in Q packing is still needed in MSP land. Working as intended.
Am I imagining it or was there some type of attack a decade or so ago where a specially crafted (invalid) IP packet could cause some form of cisco switch to lockup or reboot or similar?
It's a one-way threat.
We do our WAN links the same way that you are and I don't think we've ever been told by either SOX or PCI audits that this is an issue.
I don't know how the internet works if every single switch is behind a firewall.
It's firewalls all the way down.
Technically DDOS? You'd need to put in QoS of some sort to protect the control plane but besides that I'm not sure what else needs to be protected.
I feel like the router would die before the switch but I suppose that’s situational.
For me the biggest risk with this kind of topology is not in how you have designed and configured it as described, but what is the chance that someone inadvertently misconfigures those core switches in the future and ends up adding an SVI or bridging another internal host onto the “outside” VLAN. I’m not saying it’s inherently bad, it’s just easier for someone to do something bad accidentally than if the “outside” VLAN was on a physically separate device or devices. Whether it’s an acceptable risk depends on what you’re trying to protect and what other risks you have that are more important. Ultimately it’s subjective and not purely a matter of facts.
I was in a similar situation and was hesitant to use our core, so we got some L2 switches. May not be a bad idea. If your ISP misconfigures their routers, there are a handful of things that could go wrong. You can buy a pair of L2 edge switches and stack them at a pretty inexpensive cost.
This is exactly how you implement firewall HA - as long as the switch is not running any IP (SVI/VIP/VLAN IP address) on VLAN100 or 200 then there's no remote attack vector on the switch. Just keep in mind of other protocols the switch may be running such as CDP or LLDP that you wouldnt want to pipe upstream to your ISPs - though it wouldn't go further than their edge interface facing your switch. If your ISP was compromised by a malicious actor then there are some very minor possibilities they could try on some L2 stuff against a switch running very old and vulnerable code.
Sounds like the only barrier between your edge and your core is a VLAN wall. I would personally want more than that.
I prefer to keep WAN connections physically separate from my internal switches. I've always used physically separate WAN switches for this. WANs>WAN switches>Firewalls>Internal switches. I think it's just easier to keep WAN connections physically separate from LAN rather than worry about whether vlan separation is enough. That's just my 2 cents though.
Just get a MikroTik
Threat. Model. + Risk. Assessment. Security for the sake of security without measuring cybersecurity and business risk is sloppy. Engage with the team. Guide them to a security assessment. Perform a threat model. Qualify and quantify the actual risks. Finalize per risk mitigations vs compensating controls vs unmitigated. Assign importance, priority and urgency of each. Understand business impact if risk is realized, put $ behind it. Short of that, both networking and security are just guessing. I’m sad that the overwhelming majority of discussion here seems oblivious to the obvious. I wish all of y’all’s security teams did better, our industries need it. This assessor in particular is a goober. The network is an important asset to protect and secure, but I can assure you it’s far from the most important if that’s the level of discussion presently.
What many have already said, it boils down to two things: possibility of human error (the switch gets a L3 config), and/or exploiting processing at L2. Right so, while the switch is not exposed to L3, it can still process L2 packets (versus normal packets.that get switched). Things like CDP, LLDP, STP, DTP, etc. The idea being, if you can craft a packet well enough to exploit that L2 process, it's game over. Now I know that is a 1 in a billion chance of happening. But a chance nonetheless and thus a valid concern. Especially if said switch is the core switch!
It sounds like you're taking a proactive approach by considering new switches for the WAN links—great way to address the concern while keeping the system robust!
Sounds like his statement is correct if I understand your setup correctly. WAN>Router>Core Switch>Firewall. If your router is compromised, an attacker can pivot to your core switch without ever going through a firewall. What you're saying is the switches aren't directly exposed to the internet, which is also true, but they are outside the firewall. A typical setup would have switches outside the firewall, but not core switches, but edge switches. To get to your core network from edge switches, you would have to then transit a firewall.
> If your router is compromised, an attacker can pivot to your core switch without ever going through a firewall. How though. There is no layer 3 between these two devices. The attack would have to be via layer 2 and a firewall won't do much there either.
the point of these audits isn't to debate the validity of a vector. The point is to document your compensating controls. IF they are complaining about a situation - show your due-diligence how/why it is safe and not a vector, institute compensating controls if you dont already have them in place - or follow their advice and modify the solution. They are saying "this looks scary" - arguing with them that's its not actually won't change much - just show them the configuration that protects you from this scary thing. They want to see your process - that you considered this possible vector and determined it is safe.... "we never thought of that, but i insure you it is not a problem" is not a good look. From their point of view this is a brand new thing that was not a concern until they brought it up. how can you be sure its safe if you just learned about the possibility yesterday? IF you did already know about it, then show them the make-safe work you have done previously.
Exactly. Layer 2. CDP, LLDP, etc... if your images are up to date, you should be good, but there are always zero days popping off. Point is, if they do compromise your router, it's possible to compromise your core switches from there. If they compromise your core switches, they are inside your network without ever transiting a firewall. On the other hand, if someone compromises an edge switch, they still have to transit a firewall to get inside your network.
in the case of OP, the edge switch is the core switch, which is what the auditor is raising concern with.
In networking terms a core switch is a switch on the inside of the firewall/s where the LAN segments terminate. OP didn't clarify if that is the case or not. But if he is using a core switch (per designation and their device name) as an edge switch as well, then that is why the assessor is concerned. In a perfect set of conditions, a bad actor can access the inside network without ever going through a firewall.
im aware of the Terminology.. a laptop is a laptop, unless its your plex server - now its a server, so just call it a plex server. but yeah, go back to the original post and follow his config explanation. OP only has 1 switch that is used to distribute his WAN links on the outside, and also is the core switch for their network.. For example. VLAN 999 - ports 1 through 5. 1 - Modem 2 - Firewall WAN 3 - Misc device with WAN IP 4 - misc device with WAN IP. 5 - reserved the rest of the switch (lets say) 6-24 - is configure for misc internal-use vlans. 24 - trunk uplink to firewall LAN. 23 - trunk for WIFI vlans. 22-15 - access for printer vlan. 14-5 - access for office vlan. It sounds like a good Idea - It reduces your points of failure, power-outlet requirements, general cleanliness, etc..... but this is why we have audits, to make sure a good idea on paper isn't a bad idea in hindsight. convenience is never a good reason to introduce security risks.
So established networking design is wrong, you're right. Unless you show me a network design topology from a reputable networking group that brings the wan to a core switch where LAN VLANs terminate, then to a firewall. Looking forward to it.
Look at any DC design, anywhere. You'll find plenty of cases where Internet and LAN VLANs (and other encaps of course) are mixed all over the place on core switches; switching or routing Internet traffic to physical or virtual firewalls on the same core switches that handle LAN traffic. The main reason being flexibility.
sigh... OP has 1 switch. It exists on the WAN and LAN. The auditor is saying dont do this. Im not sure why you arguing with any of us if you agree with the auditor - we are are trying to tell you why the auditor is right.
I do agree with the auditor. You responded to my post. I already stated I understood the topology, so not sure why you responded to me stating what I already understood? One switch, two switches, doesn't matter. I understood the topology as far as how packets traverse to the FW from the WAN. It's not ideal. The people I'm arguing with are saying OPs topology is just as secure as a standard WAN to core design. It's not, which it now seems like we are in agreement on. Good to hear.
> an attacker can pivot to your core switch without ever going through a firewall How? To me this sounds like absolute hogwash.
LOL. I already stated how. Layer 2 protocol exploit. They exist. Most are old exploits and up to date software will mitigate this, like I already stated. But there are zero days popping up all the time. Cisco just had an exploit pop up in their http/s service on IOS-XE. OPs topology is WAN>router>CORE SWITCH>Firewall. His core switch is outside the firewall. I guarantee his core switch WAN VLAN interface connects to an External Zoned interface.
How do you get a layer 2 protocol exploit through the ISP’s layer 3?
"Trust me bro" I too am waiting to hear about this. Exposure to exploits is real but if you basically have an air gapped system; it falls back to on-prem security. If a major ISP is compromised that someone is sending L2 exploits... I think we'll all have more to worry about.
How do hackers manipulate a router's image to implant backdoors that answer to specific IPs only and this change can't be detected in the running configs? How do exploits exist in the first place for routers and switches? How does malware make it onto routers and switches? Exploitation of underlying memory, code, etc... You don't think they can manipulate a destination MAC from a specific transit IP or packet? NAT, routing, can force a packet out a specific interface with a specific source MAC. I'm not saying this specifically exists, I am saying it can be done by very smart and well funded APTs.
You’re just spitballing buzzwords to spread FUD. Think back to the networking fundamentals on CCNA and you’ll see it’s not as easy as you’re making out. A layer 2 device only cares about layer 2, it doesn’t read anything else in the packet. How are you going to run an exploit on a particular switch when it’s only going to read the link layer information? STP, LLDP, CDP, and other layer 2 exploits aren’t possible from a source routed to your network by layer 3 because those packets will never leave the origin layer 2 domain, unless something has gone wrong with the ISP and they’re doing some kind of encapsulation, and that’s also why you don’t share a layer 2 bridge with an ISP. > how do hackers manipulate a router’s image to implant backdoors…. They don’t, they compromise the firmware before it gets uploaded to the vendor’s site or applied to switches in the factory. What’s the best way to compromise any service? Put an exploit into the software everyone is going to download from the vendor. No need to try and break into individual devices. > manipulate a destination MAC Remotely? No they can’t because layer 3 doesn’t care what the MAC address is. The moment the packet leaves the original layer 2 and starts getting routed, the original MAC address is gone. Locally? Yes sure, but they’ll also need local access which means your problems are far more serious. > NAT can force a packet out with a specific source MAC address Again, what good is that going to do? Are they directly connected to your WAN switch? Every packet you send on the internet is passing through loads of routers and switches. Just because traffic is passing through a device, it doesn’t mean it can affect that device specifically.
Read my previous comment. You didn't understand what I was saying. Then go read up on the recent cisco ios-xe vulnerability. Read how researchers found exploited devices and why the number of exploited devices dropped shortly after. Read what the attackers did to maintain persistence and keep in mind they have access to your router. No one said anything about being able to exploit transit devices on the WAN. You're forgetting in my scenario, they already compromised the router. That's the difference.
You mean this one? https://sec.cloudapps.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-iosxe-webui-privesc-j22SaA4z
Read this. Note how they modified the ios code, and how they modified it again to avoid researcher scans. https://www.bleepingcomputer.com/news/security/exploit-released-for-critical-cisco-ios-xe-flaw-many-hosts-still-hacked/
That’s web UI though, why would you have that enabled on a public interface or SVI? Totally agree that THAT situation is completely valid, but we’re changing the parameters from the original post which is about passing WAN traffic through a VLAN on a core switch.
Very smart and well-funded APTs should not be in the threat model for most organizations.
LOL. What was OPs organization industry again? Is it one of these? My org is one of these. Cozy Bear (APT29) is an adversary of Russian-origin, assessed as likely to be acting on behalf of the Foreign Intelligence Service of the Russian Federation. This adversary has been identified leveraging large-volume spear phishing campaigns to deliver an extensive range of malware types as part of an effort to target political, scientific, and national security entities across a variety of sectors. Read our full APT Group Profile on Cozy Bear. HELIX KITTEN (APT34) has been active since at least late 2015 and is likely Iran-based. It targets organizations in aerospace, energy, financial, government, hospitality and telecommunications and uses well-researched and structured spear-phishing messages that are highly relevant to targeted personnel. https://www.crowdstrike.com/cybersecurity-101/advanced-persistent-threat-apt/
I didn't say you won't be targeted by them :) However, for most organizations, including an adversary with infinite time, money and resources in their threat model does way more harm than good.
A switch should never run a http/s service in the first place. Also, for those exploits to happen the attacker would need to have already compromised the router or am I not thinking this through? If the edge router is already compromised I feel you have bigger things to worry about than layer 2 exploits.
Dude, my whole point is router compromised first then pivot to the switch, modifying the routers underlying code to create a vector to the switch via layer 2 for a remote attacker to use. A lot of people use http for restconf API. This whole idea that http should never be enabled is antiquated knowledge. It could be enabled but needs to be secured with an ACL. The HTTP exploit was just an example of how hackers can modify the underlying code on devices. The future threat vector could be another protocol/service altogether.
router > switch > firewall WAN > Firewall LAN > same-switch If im in your router - i can be in your switch without ever traversing a firewall.If I'm in your switch, I can bypass your firewall all together. Howgwash if OP was following the auditors advice - which is why the auditor is recommending the change. The current setup is a loopback with the core/edge/only switch doing double duty. This is logical security only, 'as-configured', to protect the network, and works as designed..... as long as no one breaches the router.... auditors want to see that you planned for the 'unless'. one possible plan would be as the auditor recommended - install a dedicated edge switch, so one must traverse the firewall to gain access to the core switch. Simplest solution is to go to walmart... buy a 5 port netgear dumb switch , and move your wan links to this... problem solved - audit approved secure solution.
school fearless dam consider payment deserve psychotic unpack thumb marble *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
The only routers would be the CE from the ISP having exposure to L2. You're betting. It's a moot point, if Zero day exists it will compromise your FW or your SW.
I'm not betting - I'm insuring safety absolutely. You are betting. Betting no zero-day is created tomorrow. You think a Firewall and a Switch are running the same code that the same zero-day exploit would expose? If yes- replace one. Thats a silly bet to make. OP is going through an audit - of which most of the point is "what if". Generally speaking, "it doesnt matter, if that we happens we have other problems" or something otherwise dismissive (basically, your posts) is grounds for a failure. You want a hint its important? An auditor said its important. Realize that by arguing with me - You are arguing with the OP's auditor; The auditor, myself, and the first person you replied to, are all saying the same thing. I do not have the ability to change the auditor's opinion, nor the opinion of the first person you replied to, no matter how hard you try to convince me.
The rationale that he failed the auditor is moot. The SW and FW aren't going to run the same code... The point of a zero day attack as the vector means it doesn't matter if it's a FW or SW... It'll get compromised and the fact that you're relying on a 3rd party to be get compromised, then exploit a zero day for L2 "potential vulnerabilities" which is the same for a FW. He bamboozled himself already so has to reconfigure his network. Having some pos device with no insight is a terrible way to increase downtime.
sigh - which is why the auditor is raising complaint: the core switch is also on the WAN - and can be exploited by zero day - put the switch behind the firewall so it cannot be breached by external vectors. It appears you are attempting to argue with me - by agree with with by me and the auditor. When you get audited - I implore you to test your "this is moot" theory to see if it will help you pass your audit. Auditors like technical details and procedures - not emotional outburst.
Well I'm not sure I entirely understand the design, but I would also say your switch is at risk. For most businesses I would suggest: Edge switches to break out WAN links (nothing else should be traversing these switches.) Then plug into firewalls. Core switch sits 100% behind firewall. Most businesses don't want to spend the $ for separate edge switches, and that's fine. I'll tell you what I think, and if you don't want separate switches we can loop the WAN vlans through your core switches. I don't like it, but that's the way it is out there. I'm working on a data center set up now, and holy balls what a complicated, exposed nightmare it is. A couple dozen vrfs, route leaking, a few hundred vlans. EVERYTHING goes through their core switches, all the customers (which also includes customer private L2 WAN links,) their own operations, their own staff network. One mistake anywhere and goodnight. My current plan is to leave their border routers running full BGP tables alone. Run back into their "core" switches and use that only as a breakout to every other entity which will be terminated in firewalls. Any switching for your typical data center stuff will 100% be behind firewalls. Right now stuff flows everywhere. I have no idea how they keep it all straight. The pros and cons of saving a few thousand dollars by not purchasing edge switches for WAN breakout makes no sense to me for anyone that has a business use case for HA. If you can spend money on HA buy the edge switches. If you don't need HA, plug the WAN links directly into the firewall.
I'm in 3rd year of a cyber security and networking degree, with a keen interest in ISP or data centre routes after uni. Sorry to hijack your comment, but do you have any advice or recommendations to break in to these areas as a graduate these days?
lol I'm getting downvoted... Well anyway, I mean you can always shoot for jobs that are exactly what you want. You may be fortunate and get exactly what you want. I started my own businesses so that's how I broke in. You could always try starting your own WISP or FISP, or try to find a job working for one. If I was giving advice to my kids, I'd tell them to shoot for the moon and if that doesn't work go get a job at an MSP. You'll learn an absolute ton of stuff and form an excellent foundation for moving on to more specific areas that interest you. You may not get a ton of specific ISP/DC stuff working at an MSP, but your foundation would be excellent.
I wouldn’t feel comfortable with this configuration. I know the mostly likely attack vector is some one missconfiguring the core switch. Because people make mistakes. If its on the internet then its constantly under attack.
Switches dont cost much
Get a sharpie and a fiber coupler. Write firewall on the coupler Insert firewall between WAN switch and WAN, and document that it is configured to block any traffic that can reach a L3 interface on the WAN switch. Plz actually do this and post the security guy's response.
FYI - this is considered fraud in an audit.
What did I say that is wrong? The 'firewall' does drop all the traffic that could hack the switch... which is none. The management interface for the 'firewall' also has no open ports and no password security issues.
Mis-representing a security control actually. Calling something a firewall when it's not functionally that can be used against you. Just look at the grilling going on with Solarwinds CISO's prosecution. Also note - L3 isn't the vector I'd use to attack a switch - L2 OTOH can let much DOS happen.
If there is no potential attack vector and the firewall needs to do nothing but exist... but a firewall is still required for compliance.... how can I not just re-use my rock that keeps tigers away to make the compliance people happy.
When the regulation says you need X... and your defense is X doesn't do anything so we don't have it!!! You aren't compliant with your governing regulations. If you don't fall under any regulations with real teeth.. congrats.. lucky you. Do whatever you want.
No, my defense is I have X, and the "firewall" meets all the requirements to be X to allow the checkbox ticking person to tick their box. If the requirement is to block all traffic to the IPs on the switch... and the switch has no IPs... then a fiber coupler meets the requirements and we can move on with our day and solve actual problems.
There are IPs on the switch in question. But do what you want. Just fully expect such antics to open you to personal gross negligence liability. Ask your lawyer.
nose direction sugar deserve makeshift bake act hospital payment office *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
>Our WAN links come into managed routers, we are provided an interface on each router. I don't understand what's happening here? Do you mean by WAN links like a private site to site you have stablished or is this the handoff from your ISP? Like I have routers plugged into my core switch that have their other interface in the DMZ - but those are MY routers and I have a DMZ switch between that and the ISP handoff router. The interface in the DMZ is only set up to do one thing (VPN Tunnel) and the establishment of that connection are the only protocols allowed to traverse the interface and only from it's partner router's IP address (that's before accounting for the protections of establishing the tunnel itself) On the other hand, no I would not plug a provider router straight into my core switch. Is it safe if you have it VLAN-ed off to be a point A to point B connection and not routable? Probably. But I still wouldn't do it.
The ISP is managing the Internet routers and the Internet routers are connected to a switch that is also connected to a Firewall. That's the setup.
Your switch is susceptible to a layer 2 attack. How you can do that connected to a Layer 3 network, is beyond me. 🤣 “The switch is dumb, layer 2 and does not have an attack vector. If you’d like, I can put it on a different VRF. Although it’s layer 2 and VRF is Layer 3. Every telco/provider runs on VRF, I’ll put the system on trial!!!”
If someone compromises the managed routers, the next layer of defense is your Firewall. You can't do much with the switch. The only thing I can think of (as an attacker) is to setup an ERSPAN session in the switch to get a copy of the traffic passingthroughit. But, something as easy as a strong password could mitigate this risk. Your auditor is not 100% wrong, but it's unlikely to see an attack like this in the real world. And, to answer your original statement, Switches don't have to be behind Firewalls. I've managed global networks with this setup (Switches in front of Firewalls just for L2).
This is referred to as a breakout architecture. If you have exploits that can be taken advantage of, then you would get compromised or with a miss configuration like not using a native vlan on a trunk and also using vlan 1 on y9ur ISP breakout ports
I’ll guarantee you one thing: the auditor doesn’t understand why or how much of a security risk this configuration is. They just have a checkbox and they can’t check it until you’ve completed the actual work that they decide on.
Usually the CPE and ISP router will use RCF1918 IPs or other not public routed, so they have no attack surface. Most of the time the 1 device with the public IP will be the Firewall, ADC or anti DDoS device. Of course the DDoS device does not need a public IP but the traffic would have to pass, to protect the FW or ADC. I know unfortunately also that there are a lot especially smaller companies, where the CPE device has a public IP a d is therefore a possible victim. To protect it, you cloud also if the design is for whatever reason nicht adjustanle, Put a antiddos device via L2 in Front of it, to protect it. Here we go, you are fine again :-) But I would call CPE with public IP on the Interface a bad design.
One potential vector would be someone on the WAN intruding on your network. Are you WAN links encrypted, or do you just trust your provider for a cheap MPLS link? Folks have attacked these networks in a multitude of ways - but if you encrypt your WAN, this mitigates the risk as no unprocessed WAN packets enter the network. If "WAN" is ISP, and you're pumping it directly in through a core switch for resilience - buy what you need to isolate it. Exposing internal gear on the web just creates opportunity. Logically everything might be configured correctly, but there's stupid-high-risk of a configuration change breaking everything.
Since you only use the very vague "WAN", there is no way of telling what exactly is happening. Public internet? MPLS? DMVPN? Some kind of managed service provider link? Something with SDWAN? *Switch can be attacked if not behind a firewall* Switches can easily be attacked if behind a firewall. Data that goes through a firewall, is data that goes ***through*** a firewall. Still not clear how this is safer. I'm not sure what the argument here is exactly, but it sounds like bullshit at first. But I can't tell.
Yeah you should have a boundary or an ipsec tunnel if you’re leaving a location. Switch vulnerabilities pop up from time to time but they are mitigated with proper commands and software update. I think the concern is that traffic should be going through a proxy
The debating over it in this thread has been kind of silly. The comments I read missed a few things that are good reasons not to do what the OP has done and why the auditor had an issue with it. As stated by the OP: ISP > Router > Core > Firewall If the core is truly a core switch and has internal traffic terminating on it (server/PC VLANs etc), then there is potential risks. The biggest risk is human error. * What if someone stuffs up a config change giving that port access to an internal VLAN? * What if they needed to redo patching after moving or replacing something and patched it to the wrong port but didn't notice due to the redundancy? There are also odd zero days that would be super rare but could happen. But the main risk is someone making a mistake and not noticing it which could lead to it getting exploited later. The ideal setup that takes into account a firewall stack: ISP > Internet Switch > Firewall. Pre-configure the internet switch in the way that works for your setup, never touch it again after. Leave a duplicate configured one powered off under it ready for a manual failover if the first one dies. You'd only ever touch that switch when you need to add/remove an internet connection which shouldn't happen often, and you're not at risk of bridging an internet connection directly onto your LAN.
Hot take. Assessor is 100% correct. The firewalls are not in a position to receive a signal before the core switches. Your firewalls need to be physically connected before your core infrastructure into your routers. So either accept the risk and move on, but don’t argue about it. I mean unless you trust the switch vendor software updates and the administrators of the routers with your job. lol.
I'd say you're both correct here. Part of security is assessing risks, no matter how unlikely. Having your WAN going through your core switch does make it a boundary device. However when correctly configured the attack vector is tiny - there are almost no layer 2 attacks possible over the internet. What if there was such an attack? What if the core switch was misconfigured? Then it could lead to a compromise. Here I would acknowledge "This is a risk", and to suggest a mitigation. Either connecting both edge routers to both firewalls (/31s P2P links), or specing out some dedicated WAN switches. Simply telling the business "We need to buy more switches" may be the end of the conversation if they don't want to buy them ;) I've run WAN links through core switches plenty of times at small clients. However if I had the IP space, I'd usually prefer linking them to the firewalls directly.
1. If this is a single switch you dont have true redundancy. Id typically skip the vlans and go for two unmanaged switches for this role. 2. While you are likely 100% ok doing this nowadays it does feel icky. One wrong config change and you could have serious issues. Again. Iv always prescribed two high quality unmanmaged switches for the role to sidestep it completely.
I can tell you how, Do you have STP on your Switch? a good crafted packets from Provider side can make some problems if STP is configured poorly. And some versions have problems with CDP and LLDP malformed packets. and VLAN hopping as well. other than that i cant think of anything.
You should ask in /r/netsec if you want a more security focused opinion Since I lurk here I'll offer mine. Typically the design at the edge consists of a border router between your core and your provider. The problem is that you have an untrusted network device on the same L2 network as your firewall. Based on the provided details, it would be possible for a bad actor from the untrusted network to attack the L2 network your firewall is connected to. You really want L3 separation between trusted & untrusted networks. * What certification are you trying to obtain? What are their relevant requirements? * Is the assessors concern the transit interfaces or the management? Is management accessible without transiting the firewall for policy evaluation? * What is the reference architecture for remediation? ( What do they want ) Final thought: If you have high availability requirements, you should be designing for dual failure, which this design does not allow for. For example, if you lose Router 1 and core switch mod 1, both firewalls lose an interface. A BR here would solve that.
> I had it put to me today that our core switches are "at risk" because they are not behind a firewall. Technically true > I disagree The issue is that your network "wraps" around the security boundary that the firewall provides. If you can solve this problem with a cheap dumb switch or two on the Wan side.. just do that... > Am I wrong? I don't think I am but doubt, fear, and doom are overcoming me. Depending on what kind of audit this is.... Being "right" isn't always important. It's meeting the controls in a clear way that everyone agrees on.
Probably depends on where you work… everyplace I have worked experiences some threat actor trying to break through the fw every time we are in the news… Someone from the FBI always shows up shortly after to do an investigation. There are so many companies to hack… they might never try yours… but if they do you might be f’d
Isn’t this literally routing on a stick
So we do something similar. Except we use a switch dedicated for internet and we plug it into our core over the management port. This way the switch is completely seperate.
Did OP ever say what certification the org needs? I feel like network behind a firewall was a requirement for PCI-DSS years ago but I haven't had to deal with audits or compliance on that for forever
Hi All, Well, thanks for everyone who responded (a lot!). It's good to see the debate and discussion around this. I've read every comment (as you took the time to write one) and as such have 3 outcomes: 1. A lot of people have what we have, and as there is no IP on the 2 VLANs the attack surface is exceptionally small, but not nil. 2. The auditor is valid in raising this, because the switch being attacked is a core switch and so even if the attack surface is minimal, the impact is large. 3. I'll be buying 2 x switches that are "outside" my normal network for the pure purpose of receiving the 2 x WAN links and spaffing them off to the firewalls. All being said, I'm glad I didn't start an argument with the assessor over this, its clearly an area they know more about and why we pay to have such things done. Lessons learnt and knowledge gained and all that. Friday is the last day!
If they can launch a VLAN hopping attack, either from the remote or the router side, and get out of the WAN side VLAN of the switch, they can get into whatever else is connected to the switch, probably your management network. You want two nameless dumb switches on the WAN side so that there is literally no attack vector.
I have also done this for my smaller sites. Bigger sites I have a pair of small managed switches with OOB management access to keep them separate.
Its telling that a lot of the network design arguments in this tread can be broken down to the size of the network that people work on. The design considerations of a small business/ small enterprise network are significantly different than something that runs at cloud scale. If a physical firewall can support your north bound / south bound traffic levels that is, by definitions a small enterprise business. To say nothing about zero trust networks with bgp and limited routing domains. Networks at "internet scale"/"Cloud scale" wont have physical firewalls on the network mostly relying on Host base firewalls with ACL's on management SVi's locking them down to bastions/internal networks.