Eh, stupid me - wrong game (I was looking at Jupiter Hell). Sorry.
Check the config.lua file in your DRL installation folder, namely these settings (default values included):
-- Windows and GFX mode only:
-- Set to false to turn off the Fullscreen query at run time. If false you
-- can use StartFullscreen to control fullscreen at startup.
FullscreenQuery = true
-- whether to start in fullscreen mode, use ALT-Enter to toggle, only
-- used when FullscreenQuery is set to false
StartFullscreen = false
-- Windowed sizes
WindowedWidth = 800
WindowedHeight = 600
Change FullScreenQuery to false, keep StartFullScreen at false, and fill out whatever WindowedWidth and WindowedHeight suits you. This should allow DoomRL to start in a window of your preferred size.
Again, sorry for the confusion.