1 .\"
2 .\" This file and its contents are supplied under the terms of the
3 .\" Common Development and Distribution License ("CDDL"), version 1.0.
4 .\" You may only use this file in accordance with the terms of version
5 .\" 1.0 of the CDDL.
6 .\"
7 .\" A full copy of the text of the CDDL should have accompanied this
8 .\" source. A copy of the CDDL is also available via the Internet at
9 .\" http://www.illumos.org/license/CDDL.
10 .\"
11 .\"
12 .\" Copyright 2020 Joyent, Inc.
13 .\"
14 .Dd October 26, 2020
15 .Dt OVROUTE 1M
16 .Os
17 .Sh NAME
18 .Nm ovroute
19 .Nd overlay router management
20 .Sh SYNOPSIS
21 .\"
22 .\" ovroute router create ...
23 .\"
24 .Nm
25 .Cm router create
26 .Fl d Ar overlay
27 .Fl m Ar macaddr
28 .Fl v Ar vlan
29 .Fl a Ar address/mask
30 .Oo Fl a Ar address/mask Oc
31 .Oo Fl r Ar route_table_id Oc
32 .Ar router_id
33 .\"
34 .\" router delete
35 .\"
36 .Nm
37 .Cm router delete
38 .Fl d Ar overlay
39 .Ar router_id
40 .\"
41 .\" router get
42 .\"
43 .Nm
44 .Cm router get
45 .Fl d Ar overlay
46 .Oo Fl o Ar field Ns Oo ,... Oc Oo Fl p Oc Oc
47 .Oo Ar router_id Ns ... Oc
48 .\"
49 .\" set-routing-table
50 .\"
51 .Nm
52 .Cm router set-routing-table
53 .Fl d Ar overlay
54 .Fl r Ar route_table_id
55 .Ar router_id
56 .\"
57 .\" route-table create
58 .\"
59 .Nm
60 .Cm route-table create
61 .Fl d Ar overlay
62 .Ar route_table_id
63 .\"
64 .\" route-table delete
65 .\"
66 .Nm
67 .Cm route-table delete
68 .Fl d Ar overlay
69 .Ar route_table_id
70 .\"
71 .\" route-table get
72 .\"
73 .Nm
74 .Cm route-table get
75 .Fl d Ar overlay
76 .Ar route_table_id
77 .Oo Fl o Ar field Ns Oo ,... Oc Oo Fl p Oc Oc
78 .Oo Ar route_table_id Ns ... Oc
79 .\"
80 .\" route-table set-default
81 .\"
82 .Nm
83 .Cm route-table set-default
84 .Fl d Ar overlay
85 .Ar route_table_id
86 .\"
87 .\" route-table add
88 .\"
89 .Nm
90 .Cm route-table add
91 .Fl d Ar overlay
92 .Fl i Ar route_table_id
93 .Ar destination
94 .Ar target
95 .\"
96 .\" route-table del
97 .\"
98 .Nm
99 .Cm route-table del
100 .Fl d Ar overlay
101 .Fl i Ar route_table_id
102 .Ar destination
103 .Ar target
104 .\"
105 .\" ================================
106 .Sh DESCRIPTION
107 .\" ================================
108 The
109 .Nm
110 utility is used to manage routers within an
111 .Xr overlay 5
112 network.
113 .Xr overlay 5
114 routers are used to move packets within an overlay network off the virtual
115 layer two (VL2) network.
116 .Pp
117 For example, two VL2 subnets may wish to route packets to each other,
118 regardless of which physical system instances using those VL2 networks reside.
119 As long as the underlay IPs
120 .Po
121 the UL3 address ka the IP address of the underlying interface the overlay
122 instance sits on, e.g. the IP address of
123 .Ql sdc_underlay0
124 .Pc of the source and destination systems are reachable to each other,
125 and the virtual layer three (VL3) \(-> virtual layer two (VL2) mappings
126 for the source and destination VL3 IPs, the
127 .Nm
128 command can be run to setup routing between these two subnets.
129 Normally, these entries are managed automatically \(em e.g. by the Triton
130 .Ql net-agent
131 process, and should normally not need to be altered by an operator.
132 .Pp
133 To route packets between VL2 subnets, an
134 .Xr overlay 5
135 router instance is created for each VL2 network where packets are intended to
136 be routed.
137 A
138 .Xr overlay 5
139 router instance should be created on any physical machine where the
140 corresponding VL2 subnet is in use.
141 .Pp
142 For example, if a subnet with a vlan id of
143 .Ql 2112
144 and an address range of
145 .Ql 192.168.1.0/24
146 on overlay
147 .Ql sdc_underlay42
148 exists, then any machine that houses an instance with an IP address within
149 .Ql 192.168.1.0/24
150 should have an overlay router instance for the
151 .Ql 192.168.1.0/24
152 subnet created on it.
153 The
154 .Ar address/mask
155 and
156 .Ar macaddr
157 parameters should be the same for all such systems.
158 It is also recommended, but not required, that the router id used also
159 match.
160 It is not necessary to create an
161 .Xr overlay 5
162 router instance on a machine if the corresponding subnet is not in use
163 (no instances have IP assigned to the subnet) on that machine.
164 .Pp
165 Additionally, an
166 .Xr overlay 5
167 route table must be associated with each
168 .Xr overlay 5
169 router instance before any routing will occur.
170 Multiple router instances in a given
171 .Xr overlay 5
172 instance can share the same route table.
173 Both the source and destination systems must be configured with
174 .Xr overlay 5
175 router instances for any VL2 networks that wish to route packets between each
176 other.
177 Failure to do so will result in dropped packets.
178 .Pp
179 Once the router instances and router tables have been created, any
180 packets sent to the VL3 router IP by the instance (using the instance's routing
181 table), will then have the destination VL3 IP matched in the
182 .Xr overlay 5
183 routing table for the router instance that received the packet.
184 The match is a longest prefix match, similar to how
185 .Xr route 7P
186 works for traditional IP forwarding.
187 If the target address is the
188 .Dq any
189 .Po
190 .Ql 0.0.0.0
191 or
192 .Ql ::0
193 .Pc
194 address, the packets are routed to the destination subnet and the destionation
195 machine.
196 If another address in the
197 .Xr overlay 5
198 routing table is the longest prefix match for the VL3 destination IP, the
199 corresponding target IP:port value is the underlay IP address where the
200 encapsulated packet is sent.
201 This is normally used for features such as EIP, NAT or ELB.
202 If there is no matching destination, the packet is dropped.
203 .\"
204 .\" =====================================
205 .Sh OPTIONS
206 .\" =====================================
207 The following options are supported:
208 .Bl -tag -width overlay
209 .\"
210 .It Xo Nm Cm router create
211 .\" ----------------------
212 .Fl d Ar overlay
213 .Fl m Ar macaddr
214 .Fl v Ar vlan
215 .Fl a Ar address/mask
216 .Oo Fl a Ar address/mask Oc
217 .Oo Fl r route_table_id Oc
218 .Ar router_id
219 .Xc
220 .Pp
221 Create an
222 .Xr overlay 5
223 router instance on the overlay device
224 .Ar overlay
225 with an id of
226 .Ar router_id .
227 The
228 .Ar router_id
229 must be a unique identifer across all the
230 .Xr overlay 5
231 router instances for the given
232 .Xr overlay 5
233 instance.
234 Valid id values consists of characters from the set [A-Za-z0-9.-].
235 The MAC address for the router instance is specified by
236 .Ar macaddr
237 and the vlan id of the network for the router is given by
238 .Ar vlan .
239 The IP address and subnet mask the router instance listens on is given by
240 .Ar address/mask .
241 If the subnet has both an IPv4 and IPv6 address, an additional
242 .Fl a Ar address/mask
243 option may be specified, but at most only 1 IPv4 and 1 IPv6 address may be
244 given (and at least 1 address must be specified).
245 .Pp
246 Optionally, if an
247 .Xr overlay 5
248 routing table has already been defined, the
249 .Fl r Ar route_table_id
250 option can be specified to associate the the VL2 subnet to the given
251 .Xr overlay 5
252 route table.
253 If no route table is specified, the default route table for the
254 .Xr overlay 5
255 device (if set) will be used.
256 .\"
257 .It Xo Nm Cm router delete
258 .\" ----------------------
259 .Fl d Ar overlay
260 .Ar router_id
261 .Xc
262 Delete the
263 .Xr overlay 5
264 router instance denoted by
265 .Ar router_id .
266 .\"
267 .It Xo Nm Cm router get
268 .\" -------------------
269 .Fl d Ar overlay
270 .Oo Fl o Ar field Ns Oo ,... Oc Oo Fl p Oc Oc
271 .Oo Ar router_id Ns ... Oc
272 .Xc
273 Get the information about
274 .Xr overlay 5
275 router
276 .Ar router_id .
277 If no
278 .Ar router_id
279 is specified, all
280 .Xr overlay 5
281 router instances defined for
282 .Ar overlay
283 are printed.
284 If the
285 .Fl o
286 option is specified, the output is limited to the comma separated list of
287 fields given by the
288 .Ar field
289 arguments(s):
290 .Bl -tag -offset indent -width "NETWORK-V6"
291 .It id
292 The
293 .Ar router_id
294 as passed in to the
295 .Nm Cm router create
296 command.
297 .It network
298 The overlay IPv4 network and CIDR mask the router listens on, e\.g\.
299 .Ql 192.168.1.0/24 .
300 .It network-v6
301 The overlay IPv6 network and CIDR mask the router listens on.
302 .It vlan
303 The VLAN id of the subnet the overlay router instance is listening on.
304 .It mac
305 The MAC address the overlay router instance uses for the given subnet.
306 .It rtable
307 The overlay routing table associated with this subnet (if any).
308 .El
309 .Pp
310 Additionally, the
311 .Fl p
312 option may be passed with the
313 .Fl o
314 option to indicate the output should be in a machine parsable (colon delimited)
315 format.
316 The
317 .Fl p
318 option requires specifying the
319 .Fl o
320 option to indicate the fields to output.
321 .\"
322 .It Xo Nm Cm router set-routing-table
323 .\" ---------------------------------
324 .Fl d Ar overlay
325 .Fl r Ar route_table_id
326 .Ar router_id
327 .Xc
328 Set the the routing table for
329 .Ar router_id
330 on overlay
331 .Ar overlay
332 to
333 .Ar route_table_id .
334 .Ar route_table_id
335 must already exist, or the command will return an error.
336 .\"
337 .It Xo Nm Cm route-table create
338 .\" ---------------------------
339 .Fl d Ar overlay
340 .Fl r Ar route_table_id
341 .Ar router_id
342 .Xc
343 Creates an empty
344 .Xr overlay 5
345 router table with an id
346 Ar route_table_id
347 for overlay
348 .Ar overlay .
349 If the route table already exists, an error is returned.
350 .\"
351 .It Xo Nm Cm route-table delete
352 .\" ---------------------------
353 .Fl d Ar overlay
354 .Ar route_table_id
355 .Xc
356 Deletes the route table
357 .Ar route_table_id
358 on overlay
359 .Ar overlay .
360 .\"
361 .It Xo Nm Cm route-table get
362 .\" ------------------------
363 .Fl d Ar overlay
364 .Oo Fl o Ar field Ns Oo ,... Oc Oo Fl p Oc Oc
365 .Oo Ar route_table_id Ns ... Oc
366 .Xc
367 Display the routing table
368 .Ar route_table_id
369 on overlay
370 .Ar overlay .
371 If more than one
372 .Ar route_table_id
373 is specified, each route table is displayed.
374 If no route tables are specified, all route tables on
375 .Ar overlay
376 are displayed.
377 If the
378 .Fl o
379 option is specified, a comma separated list of fields must be specified to
380 indicate the fields to display.
381 The
382 .Ar field
383 values may be any combination of:
384 .Bl -tag -offset indent -width "destination"
385 .It id
386 The overlay route table id as is given in the
387 .Nm Cm route-table create
388 command.
389 .It destination
390 The virtual layer three (VL3) address of the destination in CIDR format.
391 In other words, the destination subnet and CIDR mask as seen by instances in
392 the overlay network.
393 .It target
394 The underlay layer three (UL3) address and port to send the matching traffic.
395 In other words, the destination address of the encapsulated packet.
396 .El
397 .Pp
398 Additionally, the
399 .Fl p
400 option may be given with the
401 .Fl o
402 option to generate output in machine parsable (colon delimited) format.
403 .\"
404 .It Xo Nm Cm route-table set-default
405 .\" --------------------------------
406 .Fl d Ar overlay
407 .Ar route_table_id
408 .Xc
409 Set the default routing table for
410 .Ar overlay
411 to
412 .Ar route_table_id .
413 If
414 .Ar route_table_id
415 doesn't exist, an error is returned.
416 .\"
417 .It Xo Nm Cm route-table add
418 .\" ------------------------
419 .Fl d Ar overlay
420 .Fl i Ar route_table_id
421 .Ar destination
422 .Ar target
423 .Xc
424 Adds an route table entry to the route table
425 .Ar route_table_id
426 in overlay
427 .Ar overlay .
428 .It Xo Nm Cm route-table del
429 .Fl d Ar overlay
430 .Fl i Ar route_table_id
431 .Ar destination
432 .Ar target
433 .Xc
434 Removes a route table entry from the route table
435 .Ar route_table_id
436 in overlay
437 .Ar overlay .
438 .El
439 .\"
440 .\" ================================
441 .Sh OPERANDS
442 .\" ================================
443 The following operands are supported:
444 .Bl -tag -width Ar
445 .It Ar overlay
446 .\" ----------
447 The overlay device to operate on.
448 For example
449 .Ql sdc_underlay1234 .
450 .It Ar macaddr
451 .\" ----------
452 A MAC address such as
453 .Ql 90:01:02:03:04:05 .
454 .It Ar vlan
455 .\" -------
456 A VLAN id.
457 Valid VLAN ids range from 1 - 4096.
458 .It Ar address/mask
459 An IPv4 or IPv6 address and CIDR mask length, e.g.
460 .Ql 192.168.1.0/24 .
461 .It Ar route_table_id
462 .\" -----------------
463 A routing table id.
464 .It Ar router_id
465 .\" ------------
466 An
467 .Xr overlay 5
468 router id.
469 .It Ar destination
470 .\" --------------
471 The virtual layer three (VL3) destination CIDR block for an
472 .Xr overlay 5
473 route table entry.
474 For example,
475 .Ql 10.1.2.0/24
476 or
477 .Ql 2001::1/96 .
478 .It Ar target
479 .\" ---------
480 The underlay layer three (UL3) address and port that receives the encapsulated
481 virtual packet.
482 For IPv4 addresses this in the form of
483 .Ql IP addr:port .
484 For IPv6 addresses, to disambiguate the address from the port, the address
485 must be in the form of
486 .Ql [IPv6 addr]:port .
487 The IPv4 address can optionally be given in this form as well, but it is not
488 required.
489 An
490 .Dq any
491 .Po
492 e.g.
493 .Ql 0.0.0.0/0
494 or
495 .Ql ::0/0
496 .Pc
497 address (with no port) indicates the encapsulated packet should be routed and
498 delivered to the machine that contains the VL3 destination address of the
499 packet.
500 .El
501 .\" ================================
502 .Sh EXIT STATUS
503 .\" ================================
504 .Ex -std
505 .\" ================================
506 .Sh EXAMPLES
507 .\" ================================
508 .Bl -inset
509 .It Example 1
510 .Pp
511 In this setup, we have the following hosts, all using an overlay instance
512 .Ql sdc_overlay7224243
513 with a vnet ID of
514 .Ql 7224243 :
515 .TS
516 box tab (@);
517 c | c | c
518 l | l | l.
519 HOST@VLAN@SUBNET
520 _
521 hostA@42@192.168.1.0/24
522 _
523 hostB@2112@192.168.2.0/24
524 _
525 hostC@42@192.168.1.0/24
526 @2112@192.168.2.0/24
527 .TE
528 .Pp
529 The router instances will be as follows:
530 .TS
531 box tab (@);
532 c | c | c
533 l | l | l.
534 SUBNET@ROUTER IP@ROUTER MAC
535 _
536 192.168.1.0/24@192.168.1.1@90:b8:d0:17:f7:be
537 _
538 192.168.2.0/24@192.168.2.2@90:b8:d0:7f:36:ac
539 .TE
540 .Pp
541 To enable all three vnics to route traffic between the two subnets, the
542 appropriate router instances must be created on each host.
543 Additionally, a routing table must be created to indicate that the traffic sent
544 to the router IP should be routed to the other subnets.
545 .Pp
546 The following commands are run on the respective hosts to enable this:
547 .Bd -literal
548 hostA# ovroute router create -d sdc_overlay7224243 -m 90:b8:d0:17:f7:be -v 42 -a 192.168.1.1/24 router-42
549 hostA# ovroute route-table create -d sdc_overlay7224243 rtab-default
550 hostA# ovroute route-table set-default -d sdc_overlay7224243 rtab-default
551 hostA# ovroute route-table add -d sdc_overlay7224243 -i rtab-default 0.0.0.0/0 0.0.0.0
552
553 hostB# ovroute router create -d sdc_overlay7224243 -m 90:b8:d0:7f:36:ac -v 2112 -a 192.168.2.1/24 router-2112
554 hostB# ovroute route-table create -d sdc_overlay7224243 rtab-default
555 hostB# ovroute route-table set-default -d sdc_overlay7224243 rtab-default
556 hostB# ovroute route-table add -d sdc_overlay7224243 -i rtab-default 0.0.0.0/0 0.0.0.0
557
558 hostC# ovroute router create -d sdc_overlay7224243 -m 90:b8:d0:17:f7:be -v 42 -a 192.168.1.1/24 router-42
559 hostC# ovroute router create -d sdc_overlay7224243 -m 90:b8:d0:7f:36:ac -v 2112 -a 192.168.2.1/24 router-2112
560 hostC# ovroute route-table create -d sdc_overlay7224243 rtab-default
561 hostC# ovroute route-table set-default -d sdc_overlay7224243 rtab-default
562 hostC# ovroute route-table add -d sdc_overlay7224243 -i rtab-default 0.0.0.0/0 0.0.0.0
563 .Ed
564 .Pp
565 Note that
566 .Ql 192.168.0.0/16 0.0.0.0
567 or similar could also have been used in the routing table to route the
568 traffic.
569 As long as the destination network and mask match the traffic.
570 .El
571 .\" ================================
572 .Sh INTERFACE STABILITY
573 .\" ================================
574 Private
575 .Pp
576 This is intented to be an internal Triton utility, and it's command line
577 interface may change without notice.
578 .\" ================================
579 .Sh SEE ALSO
580 .\" ================================
581 .Xr overlay 5 ,
582 .Xr route 7P