I'll start with the last question first. If the XP has the software to
configure your router or printer, (and running McAfee) then yes, McAfee's
firewall [and likely which includes AV] could be affecting your network.
McAfee and Norton [among others] have extra aspects, such as worm protection
which MAY interfere with other connections and local settings [computer upon
which they are installed and thereby their connectability in the network].
One must be diligent to set ALL the proper settings in those programs.
Same holds true for the XP firewall.
Any must be configured to properly pass the intended protocols and have the
proper open ports (and application assignments if applicable).

Next, has the Belkin been flashed with the latest BIOS for the router?
Though this is generally associated more with connections in VISTA inclusive
networks, XP is also still receiving updates which may affect both XP
[networking] and the local network [with 9X left waving in the breeze]. A
caution: if a massive change is made to the Belkin BIOS it could potentially
effect 9X's ability to function [small chance]. Also, flashing ANY device
carries potential issues.

As you found the web page and I presume reviewed the material; which style
of network did you choose:

1. Soft set [router assigned networking/IP, which actual means hardware
set]?
If router set, did you make sure all router settings were still
properly set?
Is there another router or network involved?
If more than one router, are the extra set as switches rather than routers?

2. Hard set [manually assigned addressing for all devices and nodes]?

3. Simple networking [file and printer, guest accounts, etc.] or fine
grained {increased security} [specifically assigned user access and
passwords, special assigned networking folders, etc.]. Those types of global
settings would affect all basic networking. Regardless of which, did you
re-check all network settings, addressing, and assignments.

4. Did you check to ensure such factors as an expired IP "lease" is not
interfering/breaking connections by using ipconfig [cmd ipconfig /all from
the run/command box on XP] and winipcfg [from the run box clicking Details]
on the 9X box, and that all network settings are still properly assigned?

5. It would be condusive to know which box and OS you are using and what
firewall and AV, as well as your partner's configuration. Also which
computer controls what.

6. You indicated the printer is connected directly to the router  [network
ready]:
Which computer/node has the configuration software for the printer?
Can this computer access the printer?
Is the printer *always on* or is it intermittently shut down during
non-use?
You also stated there were two printers; does the other function properly?

7. Has this net had any changes made to it during the time period you
indicated, and was the network reset during that time period [complete
network reset] after those changes? {That includes any Microsoft updates}

8. You indicate, that the network HAS been shutdown and restarted [though
not how extensively/completely]:
IF using router soft set [router hardware TCP/IP addressing] did you follow
the proper startup routine.
Presuming this is just a simple single net:
Modem then router > router controlling computer {computer with
router interface installed} > printer controlling computer {which would be
best as the router controller also [makes that the master computer (somewhat
like a master server) far easier to maintain the network]} > other nodes.

9. Are you attempting to use VPN or remote desktop functions?

Supplying this basic information may allow the group to better handle and
proceed through diagnosis of potential issues.

Also, if possible, try connecting the problem printer directly to the
computer with the device configuration software, and check those settings.
Dealing with two printers and several nodes, it would appear a hard set
network would be more stable.