I don't understand how a directory with no files in it can cause a problem.
Did you try to rename it and then delete it, yet?
Or, you should be able to delete it in DOS. Copy down its location and
delete it in DOS.
If the directory name is more than 8 letters long, its DOS name will be the
first 6 letters with a ~1 following it.
Example, C\Windows\System\Strangefile  will become
C:\Windows\System\Strang~1
You can try the MSDOS window first.
Or you can boot to the Startup Menu and choose Safe Mode Command Prompt Only
or  Command Prompt Only and do it there.
To get to the Startup Menu, you usually have to hold down either the F8 or
the Ctrl key during reboot (sometimes tapping it continously instead of
holding it) until the StartMenu appears.
When you are finished, do a Ctrl-Alt-Del to restart in Normal Windows.

What is it's name? What is the error message? R-Clk it, & select
Properties. Is it "Read-Only"? Then, uncheck that, & delete it, if your
sure it's something expendable. Otherwise...

If you are certain it is not a system file or otherwise important,
then...

(1) "START, Programs, MS-DOS Prompt".
(2) Type "DEL " at the C:\ Prompt. (No quotes, but there is a space.)
(3) Open Explorer to that file.
(4) Grab the file with the R-Mouse button & drag it over the Prompt
button in the Taskbar.
(5) When the DOS window opens, drop the file into it.

Hit the Enter key, if you've dragged the right one into there.

If that doesn't do it (because the file is in use), then "START, Shut
down, Restart in MS-DOS mode", and DEL the short name there.