↓Skip to main content

Prisma SD-WAN Notes

·3400 words·16 mins
Networking SD-WAN Palo Alto
Author
Craig Bruenderman

Branch Design
#

Site Architecture
#

Terms
#

  • Scope (Global or Local) of an interfaces - Determines whether the subnet is advertised through the Fabric and via routing protocols
  • Link Quality Monitoring (LQM)

Control Connections
#

  • There are four sessions established between the device and the controller:
    • MRL (Message Routing Layer): Provides the infrastructure to enable connectivity between ION and controller
    • Flows/statistics
    • Logs
    • Remote Access
  • If a circuit is excluded for controller connectivity, then in the absence of any viable link, the excluded circuit can be used for minimum controller connectivity (MRL and Remote Access)
  • Once an allowed circuit comes up, statistics and logs will be exported to controller

Order (from 5.4.3)
#

  • Controller Port
  • LAN Interface
  • Unlabeled
  • WAN interfaces, sorted by cost

Reevaluation
#

  • When a lower-cost circuit is available and is enabled for controller connectivity
  • When link reachability for an allowed circuit comes back up
  • When there is a config change
  • MRL connection can stay on a circuit until the controller resets connection (normal MRL rebalancing)

Interface Types
#

  • Scope: An interface can be of local or global scope. When an interface is designated as global, the IP subnet configured on the interface gets advertised to the App Fabric. It is not advertised when the scope is local.
  • Circuit Label: Uniquely identifies a WAN circuit. The circuit labels defined in the site settings are bound to the interface of the ION that is assigned to the site. This is discussed further in this module.

Bypass Pairs
#

Controller Ports
#

  • Dedicated (fixed) integrated port used for:
    • Dynamic Probe Generation – Used for application reachability detection
    • LQM, PCM, and BFD traffic origination for only private Layer 2 deployments (e.g., no Layer 3 direct private forwarding interfaces)
    • Administration traffic
    • SSH
    • SNMP agent
    • SNMP trap source
    • NTP source
    • Controller communication
    • ION High Availability control interface (where applicable)
  • The ION 1000 does not have a controller port; it connects to the controller via any internet-connected port by default

Dynamic Probe Generation
#

  • Allows application unreachability detection as follows:
    • When an Application unreachable event occurs as a result of an “Initialization failure” (TCP 3-way handshake failed to complete) or a “Transaction failure” (server stopped responding to client requests on an established connection), the ION will mark the path as unavailable for that particular application and prefix and will allow flows to that same application and prefix to use an alternate path specified by policy
  • As such, the ION must determine when the original path is able to be used again (e.g. the application is reachable again)
  • Hence the need for dynamic probes generated by the ION sourced from the controller port

Interface Uses
#

Internet
#

  • Interface to an untrusted internet boundary
  • Automatically installs firewall rules allowing only inbound port 4500 and ESP 50
    • Can be configured for additional inbound services (SSH, Ping, Traceroute, BGP, DHCP, SNMP, Spoke HA Sync)
  • Automatic NAT boundary for application traffic configured to go direct on the internet
    • You can configure NAT for advanced capabilities including NONAT, SNAT, and DNAT
  • Automatic formation and negotiation of Prisma SD-WAN VPN tunnels
  • Can be used to establish BGP routing adjacency with provider
  • Supported as a Sub-Interface as NATIVE or with VLAN tag, also supports PPPoE
  • Can be assigned to a Port or Bypass Pair
  • Advanced Options contains field to specify External NAT address, if behind NAT

Private WAN
#

  • Used for communication to MPLS or across a P2P link
  • Can be used establish BGP routing adjacency with provider
  • Used to form zero-touch VPN tunnels to devices with the same WAN defined on an interface
  • Traffic can be forwarded directly onto the Private WAN link unencapsulated OR forwarded via VPN
  • Supported as a Sub-interface as NATIVE or with VLAN tag
  • Can be assigned to a Port or Bypass Pair

LAN
#

  • Used to interface to the local area network (client side)
  • Can be used as a
    • Layer 3 default gateway for clients
    • Transit network for environments with Layer 3 switches
  • Supported as a Sub-interface as NATIVE or with VLAN tag
  • Allows for network context definition – used in path selection policy
  • Static routing or BGP routing supported (BGP in 5.2.1 and higher)
  • Can be assigned to a Port or Bypass Pair
  • Can act as a DHCP relay or listener for local DHCP server

Private L2
#

  • Used in topologies where transparent inline operation is required
  • Is a bridge interface with ability to intercept traffic
  • Can be used for sites where router interoperability is required
  • NOT supported as a Sub-interface

Virtual Interface (VI)
#

  • Ensures physical redundancy for connectivity
  • Can have one or two member physical interfaces
  • Cannot have a mix of controller and noncontroller ports as member interfaces
  • Can be a source port for branch services – i.e., DHCP server, SNMP, Syslog, NTP, Netflow, HA control interface
  • Lower MAC of the VI member will be taken as VI MAC
  • VI member interface
  • Cannot have any configurations – IP config, site WAN interface
  • Cannot be parent of other interfaces – Sub-Interfaces, PPPOE
  • Cannot be part of other logical interfaces – Bypass Pair
  • The following attributes of a VI member interface can be configured – description, tag, speed/duplex, admin_up, MAC address

Types of VI’s
#

  • Virtual interface with controller ports
    • Supported on H/W platforms ION 3k, ION 7k, and ION 9k
    • Admin state cannot go down for VI with controller ports
    • Won’t be cleaned up automatically when device is unassigned from a site
  • Virtual interface with noncontroller ports
    • Supported on all the platforms
    • Supports used for (public | private| LAN) configuration and site WAN interface attachment
    • Can be a parent of Sub-Interface and service link
    • VIs used for LAN will be cleaned up automatically when device is unassigned from a site

VI Limitations
#

  • Cannot create virtual interface with controller ports in a single operation, if the device can connect to the controller only through the controller interfaces
  • Steps to create VI with controller ports:
    • Reset config of one of the controller interfaces (make sure that the element can connect through the other controller interface).
    • Create VI with one controller interface and configure IP. (Element should be able to connect through VI.)
    • Reset config of second controller interface and add to the VI.

Branch Labels
#

  • Circuit labels are important to establish zero-touch tunnels over Private WANs
  • When the WAN carrier defined in the circuit labels match, Private WAN VPNs come up automatically
  • When WAN carriers are different, you can also establish Private WAN VPNs manually via configuration
  • Internet VPNs come up regardless and do not require matching WAN carriers at the end sites
  • Any internet WAN can be used to form a VPN with any other internet WAN
  • For Private WAN labels, the WAN is used to establish zero-touch tunnels with IONs that have the same WAN definition in use on a particular label & interface

VPN Configs
#

  • BFD Aggressive Mode
    • By default, the system will send a BFD packet approximately every second
    • After timeouts (one-second default), the link will be marked as down and traffic moved to an alternative link
  • BFD non-agggressive Mode
    • With no traffic on the link – The system will send a BFD probe every 30 seconds
    • With traffic on the link – The system will step up to aggressive mode until there are no active flows on the link

Branch Forwarding
#

Branch HA
#

  • If static IP configuration is used for an interface, the IP address of the backup device should be configured as a duplicate of the active device
  • For remote-site devices, customers would typically use static IP addresses on LAN ports and private WAN ports
  • If the internet ports receive an IP address from DHCP and the DHCP server pool can support multiple devices, customers configure both the active and backup device to use DHCP
  • The ION devices in the HA group communicate with each other by using an HA control interface
    • Palo Alto Networks recommends the use of the controller port on ION devices as the HA control interface
    • If customers intend to use the controller port as the HA control interface, to maintain communication to the SASE portal, they should manage the ION device through an alternate controller port
    • Both HA control interfaces must be on the same IPv4 subnet
    • HA control interfaces should use unique static IP addresses
    • Both HA control interfaces are connected to an external LAN switch or to two separate external LAN switches with a common VLAN for the HA control interfaces
    • Customers should not directly interconnect the ION device control interfaces with an Ethernet cable
    • If the active device is powered off, the HA control interface link state for the backup device will be down and the backup device will not transition to the active state
  • For devices without dedicated controller ports, a Switch Virtual Interface (SVI) or sub-interface must be used
    • SVI used for devices with switched ports (1200-S series)
    • Sub-interface used for devices with routed interfaces (any non-1200-S devices from the list above)

Routing
#

Multicast
#

Protocol Independent Multicast (PIM) Any-Source Mode/Source-Specific Multicast
#

  • Prisma SD-WAN supports both any-source multicast and source-specific multicast.
  • Any-source multicasts allows multiple senders to be on the same group/channel, as opposed to source-specific multicast, where a single source is specified

Internet Group Management Protocol (IGMP) (V2/V3)
#

  • The Internet Group Management Protocol (IGMP) is a communications protocol used by hosts and adjacent routers on networks to establish multicast group memberships
  • IGMP is an integral part of IP multicast and allows the network to direct multicast transmissions only to hosts that have requested them.
  • Prisma SD-WAN ION devices support IGMPv2 and IGMPv3 and gives the administrator the ability to pick which version to use

Rendezvous Point (RP) (Static/Dynamic RP)
#

  • A Rendezvous Point (RP) is a multicast-enabled device or node in the network that serves as a meeting point for multicast traffic in the network
  • An RP receives multicast traffic from a source and forwards the traffic to receivers interested in receiving the multicast traffic
  • Prisma SD-WAN supports static RP addressing, wherein you must configure the same RP address for all the routers in the multicast network
  • The ION device can act as an RP, or there can be an external RP
  • Prisma SD-WAN supports a maximum of eight Static RPs and 240 groups
  • The groups must be unique among the RPs—that is, you cannot configure two RPs that support the same group
  • Prisma SD-WAN does not support Auto RP and BSR protocols for dynamic RP advertisement

WAN Multicast
#

  • Prisma SD-WAN supports WAN multicast over Prisma SD-WAN VPNs
  • Ensure that you have modified the cost of your LTE circuit to avoid receiving multicast traffic on your LTE/Metered circuit

LAN Multicast
#

  • Branch sites support LAN multicast senders and receivers.

Data Center Design
#

Data Center HA
#

  • Each pair of ION devices in the Prisma SD-WAN hub-redundancy design enables VPN redundancy without requiring an HA protocol
  • Each hub device is independent and does not peer, synchronize, or share other state information with the resilient hub device, and each hub is effectively unaware of the other hub’s existence
  • The ION devices at the branch negotiate VPN connections to both of the hub devices, and the branch devices designate which is the active connection, typically corresponding to the first VPN that is successfully established
  • At any point in time, all active spoke connections might use a single hub device, or some of the spokes might use one hub device while the remainder use the other hub device
  • With data center cluster, Branch IONs are distributed between multiple active Data center IONs
  • Max Data Center cluster size is 2
    • Data centers can have multiple clusters for large horizontal scale
    • A designated default cluster will be used by all new branch sites by default
  • Branch IONs form an active tunnel to one cluster member
  • The cluster member with the active VPN peer to the branch will be the member that advertises routes into the DC core
  • The backup cluster member will maintain a VPN in the Up state, but won’t be actively forwarding
  • Cluster Soft Limit is threshold of branches after which an alarm will be generated
    • Additonal branches will continue to work

Data Center BGP Routing
#

  • Injecting routes by using the split-prefixes method happens only using the core peer to the hub and with no other eBGP peer, and only when there is an edge eBGP peer configured that receives a matching prefix
  • In Prisma SD-WAN, hub devices can have three different types of eBGP peers: core, edge, and classic
  • Peers are logical associations, which typically have a dedicated physical connection, but they could share a physical connection instead
  • In the cases of core and edge peering, the only controller configuration required is BGP peering information, including local and remote ASN, the peer IP address, and any desired timer and authentication options
  • The more complex operations are configured automatically, including route-map generation, updates, and filtering
  • In the case of classic peering for more complex topologies or scenarios, the hub device uses a custom customer configuration for flexible eBGP peering, with all required behavior manually configured

BGP Peer Types
#

Classic Peer
#

  • Standard BGP implementation
  • Will learn and advertise routes
  • Classic is the ONLY deployment mode supported in Branches
  • No default filtering

Core Peer
#

  • Data center only implementation
  • In the least-complex BGP deployments, an SD-WAN hub might have eBGP peering only with the DC core
  • Because the SD-WAN hub is off-path to the DC core, the hub receives no traffic from the core unless the hub attracts the traffic by sending routing updates to the core
  • A hub directs traffic from a spoke over an active VPN connection to services available via the DC core
  • Spoke traffic must return symmetrically for successful delivery, which is accomplished using routing updates while avoiding routing loops
  • To be able to successfully deliver traffic from a spoke to private resources via the DC core, a hub must have knowledge of routes to those resources
  • To accomplish this, hubs learn prefixes from the core over the eBGP peering session and inform the controller of these learned prefixes, which are added to the forwarding table, without any additional filtering
  • SD-WAN hub routing updates over the core eBGP peering session attract traffic to spokes by advertising spoke prefixes, unchanged and without adding a BGP community, by default
  • If dual hubs are used for data center redundancy, hubs attract routes only for the prefixes associated with active VPN sessions
  • Advertises Branch prefixes to DC core (with special behavior if same routes known via Edge peer)
  • Learns routes from DC core, and advertises to Branches
  • System auto-generated outbound route map to allow only prefixes of interest
  • Match auto-generated prefix list and AS path list
    • Set AS PATH prepend
    • Set No-Advertise/Advertise Community
    • System auto-generated inbound route map sets local pref to 100

Edge Peer
#

  • Data center only implementation
  • Configured ONLY on BGP session to MPLS router
    • Learns prefixes only
    • Will not advertise anything to WAN edge peer
  • System auto-generated inbound route map to allow only prefixes coming from MPLS
    • Match auto generated AS path list to filter edge-originated routes and routes transited through core
    • Set local pref to 500

Routing CLI Tools
#

dump routing peer routes all
dump routing peer advertised-routes all
dump routing peer received-routes all
dump routing summary
dump routing peer status all
dump routing static-route config all
clear routing peer-ip <IP addr> [soft]

AS Path Regex
#

  • ? repeats the previous character one or zero times.
  • * repeats the previous character zero or many times.
  • + repeats the previous character one or more times.
  • ^ matches the beginning of a string.
  • $ matches the end of a string.
  • [] defines a range.
  • _ matches the space between AS numbers or the end of the AS Path List.

Policy
#

  • Each branch site must have an assigned policy for Path, NAT, and QoS.
  • NAT and Zone-Based Firewall policy deep dive covered in a separate module.
  • Optionally can apply a Security policy set.
  • A user can create a new/custom policy or use one of the default policies provided.
  • Policies can be cloned and edited to fit business needs

Path Policy
#

  • When configuring the network path policy, customers can choose from the following path type options:
    • Direct Internet — Physical connection to a public circuit
    • Direct private WAN — Physical connection to a private circuit
    • Internet VPN — Secure fabric link across a public circuit
    • Private WAN VPN — Secure fabric link across a private circuit
    • Standard VPN — Service link to a standard VPN endpoint using standard IPSec

Stacked Policies
#

  • Policy stacks are composed of path, performance, QoS, security, and NAT policy sets
  • A policy set contains policy rules attached to one or more sites
  • There are simple and advanced policy stacks
    • Simple stacks contain only one policy set, simplifying the management of policy stacks for customers who do not need to leverage the stacking capabilities
      • A simple stack merges the default rule policy set and individual policy rules into one set
      • An advanced stack is a collection of policy sets (path/performance/QoS/security/NAT) that are stacked in the order in which they are evaluated by a site

Path Policy
#

  • Path policy sets specify the paths allowed for an application between two branches, between a branch and a data center, or between a branch and the Internet
  • Each path-policy set contains one or more rules that match application flow (using attributes such as network context, prefixes, and applications) and define the allowed paths and services

Network Context
#

  • Network context allows segmentation of network traffic in order to apply different policy rules for the same application
  • Customers can typically use a network context to identify traffic associated with a specific user community, such as guest users

Source and Dest Prefixes
#

  • A prefix is a group of one or more individual IP addresses or IP address subnets. A prefix can be defined as either global or local in scope
  • A global prefix has the same values across all sites
  • A local prefix allows the customer to configure site-specific values
  • A source prefix is an IP prefix list that matches the traffic source, and a destination prefix is an IP prefix list that matches the traffic destination

Applications
#

  • Applications can be system or custom applications and are defined using either Layer 7 or Layer 3/Layer 4 information
  • The customer can assign each application a transfer type—real-time audio, real-time video, transactional, or bulk
  • Transfer types are used to determine the QoS queue assignment

Actions
#

  • Path-policy rules include the following actions that an administrator can define in a policy rule:
    • Path — A path defines an application’s allowed paths. It can consist of direct or VPN paths over internet, private WAN, or specific circuit categories. These paths can be designated as active, backup, and Layer 3 failure paths. Backup paths are selected only when no available active paths exist. When no available backup paths exist, Layer 3 failure paths are used as a last resort.
    • Service and data center groups — Customers can use service and DC group labels in path policy rules in order to express intent to allow or require traffic to transit through a Prisma SD-WAN data center or a cloud security service such as Prisma Access.

Default Path Policy
#

  • Path stacks have a default rule policy set configured with two rules:
    • Default rule—This rule predefines paths allowed as Direct on any public or private circuit and as VPN on any public or private circuit. It is applied to all traffic destined to prefixes outside the enterprise network for unknown applications and applications without a defined rule.
    • Enterprise-Default rule—This rule predefines paths allowed as Direct on any private circuit or as VPN on any public or private circuit. It is applied to all traffic destined to an enterprise prefix for unknown enterprise application traffic and to enterprise applications without a defined rule.

NAT and Security Policy
#

Operations and Troubleshooting
#

Incidents and Alerts
#

CLI
#

API
#

Build Sequence
#

  • Create circuit categories
  • Create data center sites
  • Create branch sites

Misc
#

  • If an ION device loses connectivity with the controller, it still maintains the Prisma SD-WAN secure VPNs and rotates the unique session keys for each VPN every hour for up to 72 hours
  • Use the default user details username elem-admin/hackle628)bags for unclaimed devices.