PXE Boot doesn't ask for bootcode.bin

[andig]

Similar to #764, originally reported in forums before I found my way here. I've followed tutorial, main difference that a fritzbox is running as router in the local network and serving dhcp. rpi-update has been run, didn't try BRANCH=next yet.

tcpdump:

11:13:12.397988 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9d:c1:5a (oui Unknown), length 320
11:13:12.401547 IP keller.fritz.box.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 326
11:13:26.028379 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9d:c1:5a (oui Unknown), length 320
11:13:26.030140 IP keller.fritz.box.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 326
11:13:31.028659 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9d:c1:5a (oui Unknown), length 320
11:13:31.030341 IP keller.fritz.box.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 326
11:13:34.257512 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9d:c1:5a (oui Unknown), length 320
11:13:34.259085 IP keller.fritz.box.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 326

daemon log:

Aug 27 11:12:22 keller dhcpcd[1390]: eth0: Router Advertisement from fe80::ca0e:14ff:feaa:856a
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 available DHCP subnet: 192.168.0.255/255.255.255.0
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 PXE(eth0) b8:27:eb:9d:c1:5a proxy
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 tags: eth0
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 broadcast response
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 sent size:  1 option: 53 message-type  2
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 sent size:  4 option: 54 server-identifier  192.168.0.48
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 sent size:  9 option: 60 vendor-class  50:58:45:43:6c:69:65:6e:74
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 sent size: 17 option: 97 client-machine-id  00:44:44:44:44:44:44:44:44:44:44:44:44:44...
Aug 27 11:13:12 keller dnsmasq-dhcp[4625]: 653460281 sent size: 44 option: 43 vendor-encap  06:01:03:0a:04:00:50:58:45:08:07:80:00:01...
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 available DHCP subnet: 192.168.0.255/255.255.255.0
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 PXE(eth0) b8:27:eb:9d:c1:5a proxy
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 tags: eth0
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 broadcast response
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 sent size:  1 option: 53 message-type  2
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 sent size:  4 option: 54 server-identifier  192.168.0.48
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 sent size:  9 option: 60 vendor-class  50:58:45:43:6c:69:65:6e:74
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 sent size: 17 option: 97 client-machine-id  00:44:44:44:44:44:44:44:44:44:44:44:44:44...
Aug 27 11:13:26 keller dnsmasq-dhcp[4625]: 653460281 sent size: 44 option: 43 vendor-encap  06:01:03:0a:04:00:50:58:45:08:07:80:00:01...

dnsmasq.conf:

resolv-file=/etc/resolv.conf.dnsmasq 
port=0
dhcp-range=192.168.0.255,proxy
log-dhcp
enable-tftp
tftp-root=/tftpboot
pxe-service=0,"Raspberry Pi Boot   ",192.168.0.48

The " " and appended pxe server IP were added after looking for potential solutions.

Is there any way to get this working without need for as SD card?

UPDATE Playing with dhcp-reply-delay from 1 to 5 seconds didn't help any.

UPDATE I've also tried with an SD card containing but a single FAT32 partition with just BOOTCODE.BIN. This does go into TFTP but fails with Unable to mount root fs on unknown-block(2,0), but that's a separate topic.

[andig]

I've done a wireshare here https://pastebin.com/ju4hrEa6 that should show whats going on. It's limited to the packets participating in the conversion. Can upload full version if helpful.

All help would be greatly appreciated. Without SD card I'm unable to get the PI to boot from network.

[andig]

ping @ghollingworth might this be something you could help with?

[andig]

With help from the dnsmasq mailing list I've finally managed to boot the raspi without SD card/ the help of BOOTCODE.BIN. It seems that my FritzBox router is not replying to the raspi's (potentially malformed) DHCP request. As workaround I've used dnsmasq to serve the IP:

resolv-file=/etc/resolv.conf.dnsmasq

# DNS off
port=0

# DHCP on, serve static
dhcp-range=192.168.0.255,static
dhcp-host=b8:xx:xx:xx:xx,192.168.0.66
dhcp-reply-delay=1

# TFTP on
enable-tftp
tftp-root=/tftpboot

# PXE on
pxe-service=0,"Raspberry Pi Boot"

With this setup the raspi is finally net-booting.

[ghollingworth]

Looking at your previous wireshark response there is no IP address served to the device. You are using a proxy DHCP reply to provide the pxe-service but there is no standard DHCP reply from your router.

Previously you had "dhcp-range=192.168.0.255,proxy" which means the dnsmasq will not give the device an IP address (it is assuming a separate DHCP server is going to do this for you). It only serves a DHCP offer that contains the Option 43 with the client IP address of 0.0.0.0

What you've now done is to get your fritzbox to actually serve IP addresses as well so your DHCP response should now have both the Option 43 and the client IP address in there.

Why do you think the DHCP request is malformed, we've never found a problem with getting a DHCP server to actually serve it addresses.

[andig]

Looking at your previous wireshark response there is no IP address served to the device. You are using a proxy DHCP reply to provide the pxe-service but there is no standard DHCP reply from your router.

Right.

Previously you had "dhcp-range=192.168.0.255,proxy" which means the dnsmasq will not give the device an IP address (it is assuming a separate DHCP server is going to do this for you). It only serves a DHCP offer that contains the Option 43 with the client IP address of 0.0.0.0

Correct.

This setup is working fine if the SD with bootcode is in place. Combined with the fact that not a single other device has had problems with the router that makes me think that the router reply is per se fine, potentially the raspi request is different with or without bootcode.

What you've now done is to get your fritzbox to actually serve IP addresses as well so your DHCP response should now have both the Option 43 and the client IP address in there.

Imho no. I've set my server raspi to serve the IP instead of the fritzbox router?

Why do you think the DHCP request is malformed, we've never found a problem with getting a DHCP server to actually serve it addresses.

I'm only guessing. But if the client raspi receives an IP from the router with bootcode but not without then this should not depend on the router DHCP reply (which should not change), unless bootcode is already doing things for working with "malformed" replies or similar?

[ghollingworth]

Looking at your dump again, I can't see the reply from the router. I assume this is because you've got a switch between the router and the client and you are tcpdump'ing from the server...

Would you have a managed switch that you can use between the two to enable all traffic get routed to the server, then I can understand what it is that is causing it to fail. It's clearly been fixed in bootcode.bin but I'm not sure what it is that is fixed!

Gordon

[andig]

Looking at your dump again, I can't see the reply from the router. I assume this is because you've got a switch between the router and the client and you are tcpdump'ing from the server...

Both client and server are directly attached to the router.

Would you have a managed switch that you can use between the two to enable all traffic get routed to the server.

Theoretically the router should support packet capture but apparently mine doesn't. Since I'm aiming for replacement anyway I'll try to get the replacement and provide packet capture.

From the wireshark sessions I still have the DHCP request around that didn't get an answer:

0.0.0.0 255.255.255.255 DHCP    362 DHCP Discover - Transaction ID 0x26f30339

0000   ff ff ff ff ff ff b8 27 eb 9d c1 5a 08 00 45 00  .......'...Z..E.
0010   01 5c 00 00 00 00 80 11 39 92 00 00 00 00 ff ff  .\......9.......
0020   ff ff 00 44 00 43 01 48 00 00 01 01 06 00 26 f3  ...D.C.H......&.
0030   03 39 00 00 00 00 00 00 00 00 00 00 00 00 00 00  .9..............
0040   00 00 00 00 00 00 b8 27 eb 9d c1 5a 00 00 00 00  .......'...Z....
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0090   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0100   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
0110   00 00 00 00 00 00 63 82 53 63 35 01 01 37 0c 2b  ......c.Sc5..7.+
0120   3c 43 80 81 82 83 84 85 86 87 42 5d 02 00 00 5e  <C........B]...^
0130   03 01 02 01 61 11 00 44 44 44 44 44 44 44 44 44  ....a..DDDDDDDDD
0140   44 44 44 44 44 44 44 3c 20 50 58 45 43 6c 69 65  DDDDDDD< PXEClie
0150   6e 74 3a 41 72 63 68 3a 30 30 30 30 30 3a 55 4e  nt:Arch:00000:UN
0160   44 49 3a 30 30 32 30 30 31 ff                    DI:002001.

Frame 40: 362 bytes on wire (2896 bits), 362 bytes captured (2896 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: Sep  2, 2017 17:42:46.063789000 CEST
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1504366966.063789000 seconds
    [Time delta from previous captured frame: 0.000142000 seconds]
    [Time delta from previous displayed frame: 0.000142000 seconds]
    [Time since reference or first frame: 27.995098000 seconds]
    Frame Number: 40
    Frame Length: 362 bytes (2896 bits)
    Capture Length: 362 bytes (2896 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:udp:bootp]
    [Coloring Rule Name: UDP]
    [Coloring Rule String: udp]
Ethernet II, Src: Raspberr_9d:c1:5a (b8:27:eb:9d:c1:5a), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
    Destination: Broadcast (ff:ff:ff:ff:ff:ff)
    Source: Raspberr_9d:c1:5a (b8:27:eb:9d:c1:5a)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 255.255.255.255
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 348
    Identification: 0x0000 (0)
    Flags: 0x00
    Fragment offset: 0
    Time to live: 128
    Protocol: UDP (17)
    Header checksum: 0x3992 [validation disabled]
    Source: 0.0.0.0
    Destination: 255.255.255.255
    [Source GeoIP: Unknown]
    [Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 68, Dst Port: 67
    Source Port: 68
    Destination Port: 67
    Length: 328
    Checksum: 0x0000 (none)
        [Good Checksum: False]
        [Bad Checksum: False]
    [Stream index: 7]
Bootstrap Protocol (Discover)
    Message type: Boot Request (1)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x26f30339
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
    Client IP address: 0.0.0.0
    Your (client) IP address: 0.0.0.0
    Next server IP address: 0.0.0.0
    Relay agent IP address: 0.0.0.0
    Client MAC address: Raspberr_9d:c1:5a (b8:27:eb:9d:c1:5a)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Discover)
    Option: (55) Parameter Request List
    Option: (93) Client System Architecture
    Option: (94) Client Network Device Interface
    Option: (97) UUID/GUID-based Client Identifier
    Option: (60) Vendor class identifier
    Option: (255) End

[hlev]

Is a permanent resolution for the "diskless PXE" issue possible at all on the RPi3 or this root-cause is within the boot ROM itself and will remain as is?
I think my TFTP server and network setup are fine, but I also could not make it reliably work. The occasional ping -b <broadcast_ip> solves it, without that it only works ~30% of the time.

A recent bootcode.bin from https://github.com/raspberrypi/firmware/tree/2669578d1449255edf23f38ed98d208ab73faed7 on the SD card is reliable too. It'd be so nice to spare a card though.

[andig]

@hlev The solution presented above does work reliably without SD now. The key to success is to make sure that the raspi actually receives the IP which it didn't- for not entirely clear reasons- from my FritzBox.

[hlev]

@andig thanks, good to know it works, it seems the trick is the dhcp-reply-delay

TL;DR I originally experimented with dnsmasq 2.76 which did not have this option yet, installed 2.78 and all is well.

My first "diskless" plan was, despite my router supports BOOTP/TFTP, that I'd simply use it to relay BOOTP DHCPREQUEST packets to my laptop running dnsmasq (2.76). It worked up to the point that the relay did its job and sent back the DHCPOFFER packet produced by dnsmasq to the Pi. The options may not have been 100% right though. The Pi did not take the IP.

Then I simplified this setup by putting the Pi and the laptop running dnsmasq behind an unmanaged switch downstream from the router, so the router was not involved at all. With 2.76 it worked intermittently, but with 2.78 and dhcp-reply-delay=1 it is stable.