How Do You Operate Host-Based Networking Across Hundreds of DPUs?

 


Introduction

Moving networking functions into each host addresses a scaling issue but creates an operations issue. Instead of a single leaf carrying the state for a rack, you have a router in every server. These routers need consistent BGP, EVPN, and tenant configuration and need to be in sync with the leaf switches they peer with. This document will address the operations side of the problem. We will demonstrate how the design and implementation of a fabric leads to a running HBN fabric and discuss where automation and validation prevent the problems that would occur with manual work.

How Is HBN Turned On in ONES?

Configuration is in the Network Setup section under N-S Network Settings. Enabling DPU HBN requires just one switch, but changes how ONES models the entire fabric. VTEP termination is now done at the DPU, not the leaf. ONES programs the DPUs directly, requiring DPU IPs and credentials instead of host entries. Five coupled settings define the N-S network model. The Operating System field defines the switch NOS, either Cumulus Linux or SONiC, while the DPU always runs a lean version of Cumulus Linux. The Starting Subnet defines the first octet of the N-S address space. Tenant Aware (Storage) defines whether storage traffic gets dedicated per-tenant segments using a VRF-per-tenant model in the HBN container. Tenant Segmentation defines the encapsulation, and with HBN enabled, this can only be VXLAN.

What Does ONES Check Before Anything Deploys?

The Deploy tab serves as a pre-flight checklist. HBN requires host entries for DPU IPs and credentials in order for ONES to connect to the HBN container on the DPU Arm cores. ONES verifies the existence of the expected Virtual Functions (VFs) on each DPU. If VFs are missing, the deploy is rejected and the option to Apply Design to Fabric is hidden until all DPUs pass. It is to avoid the case of a partial deployment in which the fabric assumes VFs that are not present. ONES also executes Check Switch Inventory and Check Server Inventory to verify the devices are running the NOS versions expected by the design. In the case of HBN fabrics, it also includes verifying that the DPU-facing leaf switches are running the NOS selected in Network Setup.

How Are Border Leaf Ports Handled?

The Port Config for Border Leaf defines the two uplink ports on the border leaf to the external firewall. In an HBN deployment, outside traffic hits the border leaf, takes the fabric underlay across to the DPU VTEP for the tenant, and returns. Without this defined, ONES cannot tell how outside traffic enters the fabric.

What Happens When You Apply the Design?

Apply Design to Fabric automates the entire end-to-end deployment: configuration of EVPN on the leaf switches, configuration of DPUs using the HBN container, configuration of BGP peering between DPUs and leaves, and finally, the deployment of the VXLAN overlay. The design model is deployed to live infrastructure in one fell swoop, without any manual steps.

How Does Day-to-Day Tenant Provisioning Work?

Once the fabric is live, the operator creates a tenant and ONES generates the VNI, VRF, and subnet. The operator binds a server to that tenant. ONES then lists the VFs on that server and their status: free, provisioned, or error. Only free VFs can be allocated. When the operator assigns one, ONES updates the DPU VLAN, the leaf VTEP entry, and the FRR VRF-to-VNI binding together, and the VM connects through the host-side VF.

For example, two tenants can share one server: Tenant BLUE on vf2 and Tenant RED on vf6, with the remaining six VFs free for future tenants.

Conclusion

HBN distributes routing, tenancy enforcement, and VTEPs terminations across the DPUs, reducing the state in the leaf switch and enforcing multi-tenancy in hardware. Without orchestration, complexity is simply introduced in other areas. ONES, on the other hand, ensures that the design is entered just once, validated before deployment, and managed during the full lifecycle of the tenant including VF assignment and tenancy de-provisioning.

Comments

Post a Comment

Popular posts from this blog

Evolving Packet Brokering for Modern Network Observability

Image
  I recently reviewed a technical overview describing how packet brokering platforms are evolving to support large-scale observability in modern data centers and service provider networks. The discussion focused on scalability, automated deployment, and operational efficiency. Some practical observations: • Network visibility platforms are expanding support for 400G switching hardware used in modern data center fabrics • GRE encapsulation enables mirrored traffic to move across Layer-3 networks while preserving packet metadata • Zero-Touch Provisioning simplifies onboarding for new or factory-reset switches • Packet brokers aggregate traffic from TAPs and SPAN ports and apply filtering and distribution policies • CLI improvements help operators manage large monitoring environments more efficiently One notable element was how GRE tunneling extends observability beyond Layer-2 boundaries. By encapsulating mirrored traffic into GRE tunnels, monitoring systems can receive full packet c...
Read more

Scaling Deep Network Observability for 5G: Reflections from a Real Deployment

Image
  I recently went through a detailed implementation showing how deep network observability enhances 5G mobile user experience while reducing operational complexity. What stood out was how grounded the deployment was in real traffic and production-scale constraints. Some practical observations: • Fragmented systems increase cost and operational overhead • Frequent infrastructure upgrades create instability and repair expenses • Optimised filtering significantly reduces DPI hardware requirements • Real-time KPI computation improves SLA monitoring and service assurance • Standardised data export through streaming pipelines simplifies integration One misconception addressed during the walkthrough was that deeper observability automatically means more hardware. In reality, efficient filtering and intelligent packet processing reduce hardware footprint while improving performance. My biggest takeaway is that scalable observability in 5G environments requires architectural efficiency,...
Read more

How Network Copilot Uses Agentic AI to Correlate FortiGate and Splunk

Image
I recently went through a detailed implementation explaining how Network Copilot correlates FortiGate firewall telemetry with operational data indexed in Splunk . What stood out was how grounded the discussion was in real operational workflows rather than theoretical AI claims. Some practical observations: • Network and security logs are typically distributed across multiple platforms • Troubleshooting often requires manually correlating events between systems • Network Copilot ingests metrics, events, logs, and traces from firewall environments • Operators can interact with the system using natural language questions • Explainable root cause analysis improves operational confidence One misconception addressed during the walkthrough was that AI in operations means replacing engineers. In reality, the system augments teams by surfacing insights faster and correlating information across environments. Another interesting point was how the platform learns from real firewall deployments...
Read more