An old Problem, with the following, possible causes :

a) Windows does not recognize your Monitor ("Unknown Monitor")

Install either the correct Monitor Driver, or manually choose a compatible one (e.g. Plug&Play is usually a good & safe start)

b) small Hickup

Set Color Depth to 16bit in your Display Properties (but do NOT touch Resolution), then reboot when Prompted. In many cases, the next boot gives you 16bit in 640x480, and now you may increase Resolution as desired.

Note : whenever 16 Color Mode is set by Windows, it will actually load the default Video Driver (vga.drv) instead of the 3dfx Driver. Hence the inability to change both Color Depth and Resolution at one. Rebooting after setting 16bit  normally fixes this, and allows the 3dfx Driver to initialize.