Networking in Oracle Cloud Infrastructure

Components of Networking in OCI

VCN — Virtual Cloud Network

  • A virtual, private network.
  • VCN is regional.
  • Covers a single, contiguous IPv4 CIDR block.
  • e.g. 10.0.0.0/16
  • Allowed range: /16 to /30.
  • Disallowed range:
    • 169.254.0.0/16: These addresses are used for iSCSI connections to the boot & block volumes, instance metadata, & other services.
    • Class D IPs: 224.0.0.0 to 239.255.255.255 are reserved for multicast address assignments in the IP standards.
    • Class E IPs: 240.0.0.0 to 255.255.255.255 are reserved for future use in the IP standards.
  • CIDR cannot be changed once created!
  • Recommend to use RFC 1918 CIDRs: 10/8, 172.16/12 & 192.168/16.
  • CIDRs must not overlap with on-prem or peered VCN.
  • IPv6 is only supported in US govt cloud.
  • Each VCN comes with:
    • Default route table, with no route rules.
    • Default security list, with default security rules.
    • Default set of DHCP options, with default values.
  • These default components cannot be deleted but their contents can be changed.

Subnet

  • Subdivision of VCN.
  • Covers a single, contiguous IPv4 CIDR block.
  • CIDRs of subnets in same VCN don’t overlap.
  • e.g. 10.0.0.0/24 & 10.0.1.0/24
  • Can be regional or AD-specific.
  • Regional subnets are recommended.
  • Regional & AD-specific subnets can coexist in VCN.
  • Route tables, security lists & DHCP options exist at subnet-level.
  • Can be public or private.
  • Public by default.
  • OCI reserves first 2 & last IP in each subnet (192.168.0.0/24):
    • The first IP is the network address (192.168.0.0).
    • The last IP is the broadcast address (192.168.0.255).
    • The first host IP is the default gateway address (192.168.0.1).
  • CIDR cannot be changed once created!
  • Each subnet has:
    • 1 route table.
    • 1 or more security lists.
    • 1 set of DHCP options.
  • You specify these during subnet creation. If you don’t, VCN defaults are used. You can change them later.

VNIC — Virtual Network Interface Card

  • Resides in subnet.
  • Attaches to VM.
  • All VNICs in a subnet use the same route tables, security lists & DHCP options.
  • VNICs in private subnets can’t have public IPs.
  • VNICs in public subnets can optionally have public IPs.
  • Each VM has a primary VNIC that’s created during instance launch & cannot be removed.
  • You can add/remove secondary VNICs to/from VMs.
  • Consider 2 VCNs: VCN1 & VCN2 with subnets Sub11 & Sub12 in VCN1 & Sub21 & Sub22 in VCN2. If VM is in Sub11:
    • Its primary VNIC is in Sub11.
    • Its secondary VNICs can be in Sub11, Sub12, Sub21 or Sub22.
  • VM & all its VNICs must be in the same AD.
  • A VNIC has:
    • 1 primary private IP.
    • Up to 31 optional secondary private IPs.
    • 1 optional public IP per private IP.
    • 1 optional DNS hostname per private IP.
    • 1 MAC address.
    • 1 VLAN tag for BM instances.
    • A flag to enable/disable source/destination check.
    • Optional membership in 1 or more NSGs.

Private IP

  • A private IPv4 address.
  • Has related info for addressing a VM e.g. hostname for DNS.
  • Each VM / VNIC has 1 primary private IP.
  • You can add/remove secondary private IPs.
  • A VM’s primary IP never changes & cannot be removed.

Public IP

  • A public IPv4 address.
  • Has related info for addressing a VM e.g. hostname for DNS.
  • You can optionally assign a public IP to your instances or other resources that have a private IP.
  • Public IPs can be:
    • Ephemeral: Exists only for the lifetime of the private IP it’s assigned to.
    • Reserved: Exists until deleted.

Ephemeral vs Reserved Public IPs

EPHEMERALRESERVED
Assigned to:VNIC’s primary private IP.Either primary or secondary IP.
Limits:1 per VNIC, 2 per VM, 16 per BM.32 per VNIC.
Created:During instance launch or secondary VNIC creation.Independently
When unassigned:Gets deleted.Stays in pool.
Movement:Cannot be moved to a different private IP.Can be reassigned to another private IP in same region.
Scope:ADRegion
Compartment & AD:Same as private IP.Any

DRG — Dynamic Routing Gateway

  • An optional virtual router for your VCN.
  • Provides a path for private network traffic between your VCN and:
    • On-prem.
    • Another VNC in a different region.
  • Used for IPSec VPN & FastConnect.

Internet Gateway

  • An optional virtual router for your VCN.
  • Provides direct internet access.
  • Resources behind internet gateway are reachable from internet.
  • If traffic flowing through an internet gateway is destined for a public IP that’s part of OCI, it’s routed without being sent over the internet.

NAT Gateway

  • An optional virtual router for your VCN.
  • Provides internet access to resources without public IP.
  • Doesn’t expose such resources to incoming internet connections.

Service Gateway

  • An optional virtual router for your VCN.
  • Provides a path for private network traffic between VCN & supported services in Oracle Services Network like:
    • Object & File Storage
    • Autonomous Database — ADW & ATP
    • Autoscaling
    • Yum
    • Block Volumes
    • Compute
    • Containers
    • Functions
    • IAM
    • KMS
    • etc
  • Example use case:
    • DB Systems in private subnet backup to Object Storage.

LPG — Local Peering Gateway

  • An optional virtual router for your VCN.
  • Lets you peer one VCN with another in the same region.
  • Peering means the VCNs communicate using private IPs, without traffic traversing the internet.
  • 1 LPG = 1 Peering
  • To peer with more VCNs, create more LPGs.

RPC — Remote Peering Connection

  • A component for DRG.
  • Used to peer 2 VCNs in different regions.

Route Tables

  • Virtual route tables for VCN.
  • Associated at subnet-level.
  • Contain rules to route traffic from subnets to destinations outside VCN.
  • No rule required to route traffic within VCN. It’s built into OCI.
  • VCN comes with default empty route table.

Security Rules

  • Virtual firewall rules for VCN.
  • Rules can be:
    • Ingress or Egress
    • Stateful or Stateless
  • Rules specify:
    • Protocol e.g. TCP, UDP
    • Port e.g. 22, 1521, 3306
  • Security rules can be implemented using:
    • NSGs — Network Security Groups:
      • Security rules in NSGs apply to resources in that group.
      • NSGs let you separate subnet architecture from app security requirements.
      • Supported only for:
        • Compute instances.
        • Load balancers.
        • DB systems.
        • Autonomous databases.
        • Mount targets.
    • Security Lists:
      • Associated at subnet-level. Enforced at VNIC-level.
      • Security rules in security lists apply to all resources in subnet.
      • Security lists are regional.
  • VCN comes with default security list with default security rules:
    • Allow stateful ingress TCP on port 22 (SSH) from everywhere.
    • Allow stateful ingress ICMP type 3 code 4 from everywhere.
      • To receive path MTU discovery fragmentation messages.
    • Allow stateful ingress ICMP type 3 (all codes) from VCN.
      • To receive connectivity error messages.
    • Allow all stateful egress traffic.

DHCP Options

  • Config info automatically provided to VMs at boot.
  • VCN domain name: <VCN DNS label>.oraclevcn.com
  • Subnet domain name: <subnet DNS label>.<VCN DNS label>.oraclevcn.com
  • Instance FQDN: <hostname>.<subnet DNS label>.<VCN DNS label>.oraclevcn.com

Requirements for a VM to be Accessible from the Internet

  • VCN must have internet gateway.
  • Internet gateway must be enabled.
  • Subnet must be public.
  • The subnet must have a route rule that directs traffic to the internet gateway.
  • Rules in subnet’s security list must allow the traffic.
  • Rules in VM’s NSG must allow the traffic.
  • Rules in VM’s OS’s firewall must allow the traffic.
  • VM must have a public IP.

Best Practices for Security Rules

  • Prefer NSGs over security lists:
    • In an n-tier application architecture, have 1 NSG per tier. All VNICs of a tier belong to that NSG.
    • Oracle will prioritize NSGs over security lists when implementing future enhancements.
  • Get familiar with the default security list rules:
    • They enable basic connectivity.
    • They do NOT allow ping!
  • Don’t delete default security rules indiscriminately.
  • Align OS firewall rules with security rules.
    • Use firewalld in Oracle (Autonomous) Linux 7, e.g.:
      • sudo firewall-cmd --zone=public --permanent --add-port=1521/tcp
      • sudo firewall-cmd --reload
  • Use VNIC metrics to troubleshoot packets dropped due to security rules.

Parts of a Security Rule

  • Direction: Ingress or Egress
  • Stateful or Stateless
  • Source type (CIDR, NSG, service) & source for ingress rules.
  • Destination type (CIDR, NSG, service) & destination for egress rules.
  • IP Protocol
  • Source Port
  • Destination Port
  • ICMP Type & Code
  • Description

NOTE: Security rules are not enforced for traffic involving the 169.254.0.0/16 CIDR block, which includes services such as iSCSI & instance metadata.