400s Ethernet Throughput
Many people did wonder already why the 400s Ethernet Throughput is so slow, and if there is a way to improve it. Udo Eberhardt, an experienced device driver programmer from Germany did a research to the bottom to know exactly the limitations of the 400s Ethernet connection.
Here his excellent report:
With my 400s box I was able to achieve an FTP download rate of 2100..2300 KB/s using TotalCommander on WinXP as client. This was measured with the Neumoteam 2.13.6 image.
The Relook 400s box uses the DM9000E Ethernet controller from DAVICOM. I checked the data book and the schematics from DGStation. The DM9000 is a very unsophisticated Ethernet controller. It is able to support DMA transfers but this seems not to be used in the 400s design. The DM9000 is connected to an ISA-style bus and all data transferred via Ethernet needs to be moved from/to the DM9000 by the CPU via 16 bit memory mapped IO. As a consequence, moving a packet from/to the Ethernet consumes a lot of CPU cycles.
As a first step I reviewed the Ethernet driver for the DM9000 that is contained in the Linux kernel. After checking the PowerPC instruction set and the specific 400s hardware setup I found that it would be possible to reduce the CPU cycles required to move Ethernet packets from/to the DM9000. I implemented some optimizations in the DM9000 driver. The major changes were:- Speed up of packet copy operations by using loop unrolling techniques. Approximately only half as much CPU cycles are needed to copy one packet.- Elimination of a tasklet in the RX path which was scheduled for every packet. This improves latency in the RX path.
But: This did not change the download rate in any way! I came to the realization that the DM9000 driver seems not to be the bottle neck.
Next I concentrated on vsftpd. When you check the system load with the tool “top” while a download is in progress then you can see that vsftpd causes a CPU load of about 95%. I analyzed the implementation of vsftpd and tested some modifications. Mainly, I tried the following:- increasing the socket buffer size (SO_SNDBUF)- increasing the file buffer size (default: 64KB)
But: No success. Again I was not able to improve throughput.
I found that adding use_sendfile=NO to /etc/vsftpd.conf seems to improve the download rate slightly. But this can easily be within the inaccuracy of measurement.
Next I checked the data rate that can be achieved if a file is read from the IDE disk. I have a Samsung HD400LD and found that the maximum read rate is ~5MB/s only. You can test the rate with this command: hdparm -t /dev/ide/host0/bus0/target0/lun0/disc
I found that the IDE interface uses MDMA2 mode (don’t know what it exactly is) and not UDMA. You can check this with hdparm -I /dev/ide/host0/bus0/target0/lun0/disc
In the IDE driver source code I found no way to improve this. The hardware seems to be limited to that mode and does not support UDMA.
The 400s hardware design seems to have some restrictions which lead to a limitation in the throughput that is achievable for an FTP download of a recorded TS file. The main restrictions are: – Ethernet controller does not support DMA, data needs to be moved via memory mapped programmed IO (PIO) – IDE interface does not support UDMA mode
Because of these restrictions, when serving a file download the vsftpd server consumes all the CPU bandwidth that is available. So the processing power of the CPU becomes the limiting factor.
Given that the download rate I measured is about half of the HD read transfer rate there seems to be no way to improve the overall throughput further.
Best regards, Udo Eberhardt