Note: this was tested on Software Version 08.0.61 of the ICX switch firmware. If you are using a different release, you may want to verify my findings.
I recently came across an interesting problem when trying to add more than one DHCP address pool to my home laboratory’s ICX 7150-C12P switch. I wanted to add two new address pools to make attaching new x86 server hardware and Wi-Fi access points a little easier!
The rest, is history:
Setting Up Multiple Address Pools
I followed the steps to configure my DHCP address pools contained in this article and ended up with the final configuration of the new pools as follows:
ip dhcp-server enable ip dhcp-server server-identifier 172.31.255.1 ! ip dhcp-server pool mgmt_1 dhcp-default-router 172.31.255.1 dns-server 10.0.0.254 18.104.22.168 excluded-address 172.31.255.1 172.31.255.99 lease 0 6 0 network 172.31.255.0 255.255.255.0 deploy ! ! ip dhcp-server pool x86_hosts dhcp-default-router 172.31.128.1 dns-server 172.31.128.1 22.214.171.124 126.96.36.199 excluded-address 172.31.128.1 lease 1 0 0 network 172.31.128.0 255.255.255.0 deploy ! ! ip dhcp-server pool ap-default dhcp-default-router 172.31.129.1 dns-server 172.31.129.1 excluded-address 172.31.129.1 lease 0 6 0 network 172.31.129.0 255.255.255.0 deploy !
I assumed this was a perfectly simple change and that everything would work perfectly and went ahead and attached ALL THE HARDWARE! I logged into the switch and used the command show ip dhcp-server binding and saw that the DHCP server had indeed, assigned some addresses to the newly attached hardware from my newly configured address pools.
“All is well!” I smugly thought. I thought wrong.
Symptom 1: Hello? Anybody Home?
I attempted to reach the attached hardware, first by WEB UI, then by SSH, and then finally, and rather dejectedly I might add, by using a ping. Nothing was responding. This was very strange. I could reach the gateway of each of my new networks, there was no special routing required to get back to me. What on earth could it be? The DHCP server had assigned the addresses… WTF man!?
I decided to plug my laptop into one of the switch ports and check what was going on. Lo and behold, my laptop reported no assigned IP address. But the Switch reported that it had in fact assigned me an address. “The plot thickens…” I thought.
Symptom 2: Acknowledge Me!
At this point I knew that the switch had assigned the laptop an address, and I also knew that the laptop had not received one. Something, somewhere was going missing and I needed a packet trace to actually see what was happening with the DHCP packets. I disconnected from the switch, opened up Wireshark, set my filter to udp.port==68, started the trace, and reconnected to the switch.
This is what I saw:
The packet capture showed up the problem pretty quickly – the switch was not sending a DHCP ACK.
Comparing with a working Address Pool
I knew that the address pool for my management network was working fine. Plugging a client device into an interface using the mgmt_1 address pool worked perfectly! So I had a look at the configuration of the switch (above), and tried to see if there was anything that was different about the DHCP server configs for 172.31.255.1/24 vs 172.31.129.1/24. Turns out there was only one thing that seemed different:
ip dhcp-server enable ip dhcp-server server-identifier 172.31.255.1 !
What is the Server-Identifier?
The server identifier is a global configuration for the DHCP Server on the switch. It provides clients with a unique identifier for the DHCP Server in the form of the DHCP server’s IP address. The attribute is sent by the server in DHCP option 54 in the DHCP OFFER packet, and is also used by the client when sending the DHCP REQUEST packet.
This can be a useful feature in environments with multiple DHCP servers. Remember that DHCP messages sent by the client are always broadcasts. This feature provides the client with a mechanism to specify which DHCP server, it is sending the DHCP REQUEST message to. Here is a note from Microsoft explaining the same idea.
Changing the Server-Identifier
First I tried configuring the server-identifier to 172.31.129.1. I reconnected to the switch and took another packet trace. DHCP for my APs now worked, but DHCP on the interfaces dedicated to my management network stopped working.
It is my deduction that the switch’s DHCP Server sees the server-identifier in the DHCP REQUEST Packet and attempts to broadcast the DHCP ACK from the same IP Address. If you are not in the same network segment as that IP Address, you won’t see the broadcasted DHCP ACK. In this case 172.31.129.1 is not visible on layer 2 from the management network, meaning that the management network cannot receive any broadcast DHCP ACK messages from that IP.
This isn’t really a great solution, as it still only allows one address pool in the switch’s internal DHCP server to work at any given time.
Removing the Server Identifier
Secondly I tried removing the dhcp-server server-identifier completely. I can do this because I only have one DHCP server in the network; I don’t really need to worry about giving clients a unique identifier to select a specific server’s lease offer. This gave me good results! DHCP now works on all my interfaces. Each interface now responds directly to the broadcasted DHCP REQUEST messages they receive. This solution works well, but only in simple environments where you’re sure your DHCP discover messages will only be heard by one server!
What About Using a Loopback Interface?
Well, I tried this, and rather unsurprisingly, I got the same results I got in my first option. The DHCP OFFER and DHCP REQUEST packets are sent with the loopback interface’s IP Address specified as the server-identifier in Option 54. The switch fails to send back a DHCP ACK to any physical interface, breaking the DHCP exchange everywhere. So this is a no go.
Working with DHCP in More Complex Environments
Sometimes we are required to work in more complex environments that may contain multiple DHCP servers serving multiple address pools. Given what we have seen above, in these scenarios I would strongly advise NOT using the switch’s internal DHCP server. I would be more inclined to use external DHCP servers along with the switch’s DHCP relay function. You can configure any interface on your switch with an ip helper-address, that will forward any UDP broadcast messages to the specified DHCP server for that interface. This also allows you to place DHCP servers in a centralised location, define which DHCP server will respond to a request on any given interface and cuts down on the amount of broadcast traffic flying about in your network. Ideally, if you are working in a more complicated environment, this is the way to go!
As Always, I hope this was useful!