If anyone gets here from google looking for simple DHCP settings (which google doesn't turn up atm...)

• rebuild kernel per the existing docs (1)
• ensure running latest ports version of iocage (0.9.10 as of this writing)
• for each jail: iocage set bpf=yes dhcp=on devfs_ruleset=4 JAILNAME (2)
• that's it, no need to create the bridges, gateways, etc. iocage does that for you <3

(1) kernel building, specifically to change options SCTP to nooptions SCTP and add options VIMAGE. RACCT and RCTL are already default enabled in the 11.1-RELEASE generic kernel, if you start from that as a baseline (after installing the sources from the latest SVN, cd to /usr/src/sys/amd64/conf and cp GENERIC to VNET or some other name you like, then edit VNET to add those options and use VNET as your build/install target).

(2) see below about additional settings in the HEAD (1.0 alpha) version of iocage.

(1) kernel building, specifically to change options SCTP to nooptions SCTP

@RNCTX, why is it necessary to disable SCTP? (aside from a disabling a whole network stack that hardly anyone uses)

That's a fair question, it's in the original docs and I don't use it, so I disabled it per the original docs.

The only mention I can find of it is in this from a few years ago, suggesting issues with SCTP and IPV6 in 9.0...

https://lists.freebsd.org/pipermail/freebsd-net/2014-April/038403.html

I don't use IPV6 so can't confirm/deny issues there either.

I did some further poking and prodding, it seems like the configuration for iocage is very much a moving target as we come up to the official 1.0 release.

I also have a beta install of FreeNAS on a virtualbox and was comparing their default jail configs versus mine. FreeNAS beta is using the head version of iocage (which reports 1.0 alpha if you compile it yourself). There are some variances:

• when you set DHCP in FreeNAS, the dhcp request is made on the epair interface, and 1.0 (head) has an option to set vnet_interfaces=none in the config. FreeNAS uses this by default. I'm not sure what purpose this serves, but it apparently changes which interface the DHCP address binds to.
• there is no need to specify anything on the ip4_addr or ip6_addr lines, those are picked up by the dhcp client during the jail startup. edited the above to reflect this. dhcp=on is sufficient, apparently.
• I stripped all other settings other than DHCP from rc.conf on both host and jails. The sysctl settings mentioned in the guide do not seem to be required either. I stripped those as well, and everything seems to work fine.
• FreeNAS is using devfs_rules=4 rather than 5, so I changed that as well.

edited post above to reflect these tests.

@skarekrow wrote:

Specifying devfs_rules is not necessary for iocage either.

That is not true, because it's in the config by default. If you unset it (leave it blank), the jail will not boot, mount reports an invalid ruleset. If you remove or comment the configuration parameter entirely, you get an error about it not being in the config...

Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/iocage/cli/start.py", line 54, in cli
    ioc.IOCage(exit_on_error=True, jail=jail, rc=rc).start()
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/iocage.py", line 1643, in start
    exit_on_error=self.exit_on_error)
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_start.py", line 63, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_start.py", line 109, in __start_jail__
    devfs_ruleset = self.conf["devfs_ruleset"]
KeyError: 'devfs_ruleset'

I'm now on the head version as of yesterday, btw.

Here is the current networking docs that mentions all of these things.

Gotcha, thanks!

After changing it I realized that there was no documentation pointing to what the default was. I couldn't change it back to default because I didn't remember what it was (without creating and destroying another jail, anyway...)

I'll set mine back at 5.

@gronke wrote:

python-iocage uses netifaces.gateways() to obtain the interface of the default gateway:

That behavior changed between 0.9.10 and the current head version. Previously the address in the jail's ifconfig was on vnet0 for me. On the most recent head version the jail's ifconfig lists the address on the epair interface. This may not make any difference, both worked for me just fine.

Here's the output of a fresh devfs.rules from a stock 11.1-RELEASE jail....

cat /var/db/etcupdate/current/etc/defaults/devfs.rules
#
# The following are some default rules for devfs(5) mounts.
# The format is very simple. Empty lines and lines beginning
# with a hash '#' are ignored. If the hash mark occurs anywhere
# other than the beginning of a line, it and any subsequent
# characters will be ignored.  A line in between brackets '[]'
# denotes the beginning of a ruleset. In the brackets should
# be a name for the rule and its ruleset number. Any other lines
# will be considered to be the 'action' part of a rule
# passed to the devfs(8) command. These will be passed
# "as-is" to the devfs(8) command with the exception that
# any references to other rulesets will be expanded first. These
# references must include a dollar sign '$' in front of the
# name to be expanded properly.
#
# $FreeBSD: releng/11.1/etc/defaults/devfs.rules 233048 2012-03-16 21:30:26Z mm $
#

# Very basic and secure ruleset: Hide everything.
# Used as a basis for other rules.
#
[devfsrules_hide_all=1]
add hide

# Basic devices typically necessary.
# Requires: devfsrules_hide_all
#
[devfsrules_unhide_basic=2]
add path log unhide
add path null unhide
add path zero unhide
add path crypto unhide
add path random unhide
add path urandom unhide

# Devices typically needed to support logged-in users.
# Requires: devfsrules_hide_all
#
[devfsrules_unhide_login=3]
add path 'ptyp*' unhide
add path 'ptyq*' unhide
add path 'ptyr*' unhide
add path 'ptys*' unhide
add path 'ptyP*' unhide
add path 'ptyQ*' unhide
add path 'ptyR*' unhide
add path 'ptyS*' unhide
add path 'ptyl*' unhide
add path 'ptym*' unhide
add path 'ptyn*' unhide
add path 'ptyo*' unhide
add path 'ptyL*' unhide
add path 'ptyM*' unhide
add path 'ptyN*' unhide
add path 'ptyO*' unhide
add path 'ttyp*' unhide
add path 'ttyq*' unhide
add path 'ttyr*' unhide
add path 'ttys*' unhide
add path 'ttyP*' unhide
add path 'ttyQ*' unhide
add path 'ttyR*' unhide
add path 'ttyS*' unhide
add path 'ttyl*' unhide
add path 'ttym*' unhide
add path 'ttyn*' unhide
add path 'ttyo*' unhide
add path 'ttyL*' unhide
add path 'ttyM*' unhide
add path 'ttyN*' unhide
add path 'ttyO*' unhide
add path ptmx unhide
add path pts unhide
add path 'pts/*' unhide
add path fd unhide
add path 'fd/*' unhide
add path stdin unhide
add path stdout unhide
add path stderr unhide

# Devices usually found in a jail.
#
[devfsrules_jail=4]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path zfs unhide

After checking the default rules for gronke that makes sense.

So with all of this in mind, in my opinion what the docs lack is:

• Explanation to users (links to wiki or BSD handbook?) what the different considerations are for what their jail needs to do. They probably don't know the limitations of shared IP (such as the need for static routes from their router if they need inbound traffic through NAT to the jail).
• Explanation that they must read and choose on their own, iocage can't do it for them because iocage doesn't know what they are going to do in the jail (whether they have servers inside or not).
• Less examples, more "this is how the system works". Examples are great but the choices between vnet and shared ip aren't something that they can copy/paste without understanding what they are trying to accomplish and what the capabilities of their router are. I'd have a good few paragraphs explaining the differences in the two, with any examples posted at the bottom of the document.